2023-11-28T12:46:36.153 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T12:46:36.153 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --owner scheduled_neesingh@teuthology --verbose --name neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi -- /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973/orig.config.yaml 2023-11-28T12:46:36.196 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T12:46:36.283 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973 branch: wip-neesingh-testing-231123 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7469973' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-neesingh-testing-231123 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: issue_54557 sha1: 3ae99413970476692dfab1fe38dadb056700a61d owner: scheduled_neesingh@teuthology priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6754 sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 sleep_before_teardown: 0 suite: fs suite_branch: issue_54557 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 3ae99413970476692dfab1fe38dadb056700a61d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi061.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_12:38:30 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T12:46:36.284 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa; will attempt to use it 2023-11-28T12:46:36.284 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks 2023-11-28T12:46:36.284 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T12:46:36.286 INFO:teuthology.task.internal:Checking packages... 2023-11-28T12:46:36.310 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2f5f6028b893ad41357cd06e827373f0fe2a1232' 2023-11-28T12:46:36.310 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T12:46:36.310 INFO:teuthology.packaging:ref: None 2023-11-28T12:46:36.310 INFO:teuthology.packaging:tag: None 2023-11-28T12:46:36.310 INFO:teuthology.packaging:branch: wip-neesingh-testing-231123 2023-11-28T12:46:36.310 INFO:teuthology.packaging:sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:46:36.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-neesingh-testing-231123 2023-11-28T12:46:36.541 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2697.g2f5f6028 2023-11-28T12:46:36.541 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T12:46:36.548 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T12:46:36.548 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T12:46:36.596 INFO:teuthology.task.internal:Saving configuration 2023-11-28T12:46:36.628 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T12:46:36.635 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T12:46:36.660 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:41:37.001073', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T12:46:36.683 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi061.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:41:37.002518', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T12:46:36.683 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T12:46:36.689 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T12:46:36.690 INFO:teuthology.task.internal:roles: ubuntu@smithi061.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T12:46:36.690 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T12:46:36.782 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb039367a60>, signals=[15]) 2023-11-28T12:46:36.782 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T12:46:36.789 INFO:teuthology.task.internal:Opening connections... 2023-11-28T12:46:36.789 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T12:46:36.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:46:36.860 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T12:46:36.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:46:36.968 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T12:46:36.977 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2023-11-28T12:46:36.999 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2023-11-28T12:46:36.999 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2023-11-28T12:46:37.059 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T12:46:37.059 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T12:46:37.059 INFO:teuthology.orchestra.run.smithi006.stdout:ID="rhel" 2023-11-28T12:46:37.059 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="fedora" 2023-11-28T12:46:37.059 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8.6" 2023-11-28T12:46:37.059 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8" 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T12:46:37.060 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T12:46:37.061 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2023-11-28T12:46:37.082 DEBUG:teuthology.orchestra.run.smithi061:> uname -m 2023-11-28T12:46:37.103 INFO:teuthology.orchestra.run.smithi061.stdout:x86_64 2023-11-28T12:46:37.104 DEBUG:teuthology.orchestra.run.smithi061:> cat /etc/os-release 2023-11-28T12:46:37.165 INFO:teuthology.orchestra.run.smithi061.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T12:46:37.165 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T12:46:37.165 INFO:teuthology.orchestra.run.smithi061.stdout:ID="rhel" 2023-11-28T12:46:37.165 INFO:teuthology.orchestra.run.smithi061.stdout:ID_LIKE="fedora" 2023-11-28T12:46:37.166 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_ID="8.6" 2023-11-28T12:46:37.166 INFO:teuthology.orchestra.run.smithi061.stdout:PLATFORM_ID="platform:el8" 2023-11-28T12:46:37.166 INFO:teuthology.orchestra.run.smithi061.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T12:46:37.166 INFO:teuthology.orchestra.run.smithi061.stdout:ANSI_COLOR="0;31" 2023-11-28T12:46:37.166 INFO:teuthology.orchestra.run.smithi061.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T12:46:37.166 INFO:teuthology.orchestra.run.smithi061.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T12:46:37.166 INFO:teuthology.orchestra.run.smithi061.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T12:46:37.166 INFO:teuthology.orchestra.run.smithi061.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T12:46:37.167 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:46:37.167 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T12:46:37.167 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T12:46:37.167 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T12:46:37.167 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T12:46:37.167 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server 2023-11-28T12:46:37.190 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T12:46:37.200 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T12:46:37.220 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T12:46:37.220 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2023-11-28T12:46:37.222 DEBUG:teuthology.orchestra.run.smithi061:> test '!' -e /home/ubuntu/cephtest 2023-11-28T12:46:37.241 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T12:46:37.248 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T12:46:37.248 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2023-11-28T12:46:37.284 DEBUG:teuthology.orchestra.run.smithi061:> test -z $(ls -A /var/lib/ceph) 2023-11-28T12:46:37.305 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T12:46:37.362 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T12:46:37.377 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-11-28T12:46:37.377 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-11-28T12:46:37.377 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T12:46:37.378 DEBUG:teuthology.orchestra.run.smithi061:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T12:46:37.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:46:37.397 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2023-11-28T12:46:37.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:46:37.398 DEBUG:teuthology.orchestra.run.smithi061:> uname -r 2023-11-28T12:46:37.459 INFO:teuthology.orchestra.run.smithi006.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:37.459 INFO:teuthology.task.kernel:Running kernel on smithi006: 4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:37.459 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum install -y kernel 2023-11-28T12:46:37.461 INFO:teuthology.orchestra.run.smithi061.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:37.461 INFO:teuthology.task.kernel:Running kernel on smithi061: 4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:37.461 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum install -y kernel 2023-11-28T12:46:38.642 INFO:teuthology.orchestra.run.smithi061.stdout:Updating Subscription Management repositories. 2023-11-28T12:46:38.642 INFO:teuthology.orchestra.run.smithi061.stdout:Unable to read consumer identity 2023-11-28T12:46:38.652 INFO:teuthology.orchestra.run.smithi006.stdout:Updating Subscription Management repositories. 2023-11-28T12:46:38.652 INFO:teuthology.orchestra.run.smithi006.stdout:Unable to read consumer identity 2023-11-28T12:46:40.016 INFO:teuthology.orchestra.run.smithi006.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-11-28T12:46:40.018 INFO:teuthology.orchestra.run.smithi061.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-28T12:46:40.694 INFO:teuthology.orchestra.run.smithi061.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-11-28T12:46:40.967 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-11-28T12:46:44.411 INFO:teuthology.orchestra.run.smithi061.stdout:lab-extras 441 kB/s | 24 kB 00:00 2023-11-28T12:46:44.750 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 426 kB/s | 24 kB 00:00 2023-11-28T12:46:45.096 INFO:teuthology.orchestra.run.smithi061.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T12:46:45.143 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-11-28T12:46:45.144 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-11-28T12:46:45.144 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-11-28T12:46:45.262 DEBUG:teuthology.orchestra.run.smithi061:> echo no | sudo yum reinstall kernel || true 2023-11-28T12:46:45.411 INFO:teuthology.orchestra.run.smithi006.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T12:46:45.458 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-11-28T12:46:45.459 INFO:teuthology.orchestra.run.smithi006.stdout:Nothing to do. 2023-11-28T12:46:45.460 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-11-28T12:46:45.580 DEBUG:teuthology.orchestra.run.smithi006:> echo no | sudo yum reinstall kernel || true 2023-11-28T12:46:45.612 INFO:teuthology.orchestra.run.smithi061.stdout:Updating Subscription Management repositories. 2023-11-28T12:46:45.613 INFO:teuthology.orchestra.run.smithi061.stdout:Unable to read consumer identity 2023-11-28T12:46:45.893 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:00:01 ago on Tue 28 Nov 2023 12:46:44 PM UTC. 2023-11-28T12:46:45.930 INFO:teuthology.orchestra.run.smithi006.stdout:Updating Subscription Management repositories. 2023-11-28T12:46:45.931 INFO:teuthology.orchestra.run.smithi006.stdout:Unable to read consumer identity 2023-11-28T12:46:45.935 INFO:teuthology.orchestra.run.smithi061.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T12:46:45.974 INFO:teuthology.orchestra.run.smithi061.stderr:Error: No packages marked for reinstall. 2023-11-28T12:46:46.022 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum reinstall -y kernel || true 2023-11-28T12:46:46.212 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 12:46:44 PM UTC. 2023-11-28T12:46:46.258 INFO:teuthology.orchestra.run.smithi006.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T12:46:46.282 INFO:teuthology.orchestra.run.smithi006.stderr:Error: No packages marked for reinstall. 2023-11-28T12:46:46.329 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum reinstall -y kernel || true 2023-11-28T12:46:46.369 INFO:teuthology.orchestra.run.smithi061.stdout:Updating Subscription Management repositories. 2023-11-28T12:46:46.370 INFO:teuthology.orchestra.run.smithi061.stdout:Unable to read consumer identity 2023-11-28T12:46:46.636 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 12:46:44 PM UTC. 2023-11-28T12:46:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:Updating Subscription Management repositories. 2023-11-28T12:46:46.678 INFO:teuthology.orchestra.run.smithi006.stdout:Unable to read consumer identity 2023-11-28T12:46:46.681 INFO:teuthology.orchestra.run.smithi061.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T12:46:46.701 INFO:teuthology.orchestra.run.smithi061.stderr:Error: No packages marked for reinstall. 2023-11-28T12:46:46.751 DEBUG:teuthology.orchestra.run.smithi061:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T12:46:46.834 INFO:teuthology.orchestra.run.smithi061.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:46.834 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:46.835 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T12:46:46.835 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-28T12:46:46.835 DEBUG:teuthology.orchestra.run.smithi061:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-28T12:46:46.861 INFO:teuthology.orchestra.run.smithi061.stdout:ttyS1 2023-11-28T12:46:46.886 DEBUG:teuthology.parallel:result is None 2023-11-28T12:46:46.955 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 12:46:44 PM UTC. 2023-11-28T12:46:46.999 INFO:teuthology.orchestra.run.smithi006.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T12:46:47.024 INFO:teuthology.orchestra.run.smithi006.stderr:Error: No packages marked for reinstall. 2023-11-28T12:46:47.077 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T12:46:47.160 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:47.160 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T12:46:47.160 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T12:46:47.160 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-28T12:46:47.160 DEBUG:teuthology.orchestra.run.smithi006:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-28T12:46:47.187 INFO:teuthology.orchestra.run.smithi006.stdout:ttyS1 2023-11-28T12:46:47.204 DEBUG:teuthology.parallel:result is None 2023-11-28T12:46:47.204 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T12:46:47.215 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T12:46:47.215 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T12:46:47.247 DEBUG:teuthology.orchestra.run.smithi061:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T12:46:47.269 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T12:46:47.281 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T12:46:47.303 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T12:46:47.303 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T12:46:47.306 DEBUG:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T12:46:47.339 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T12:46:47.346 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T12:46:47.347 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T12:46:47.372 DEBUG:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T12:46:47.410 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:46:47.416 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:46:47.421 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:46:47.431 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T12:46:47.431 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T12:46:47.438 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T12:46:47.438 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T12:46:47.441 DEBUG:teuthology.orchestra.run.smithi061:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T12:46:47.490 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T12:46:47.499 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T12:46:47.499 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T12:46:47.509 DEBUG:teuthology.orchestra.run.smithi061:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T12:46:47.548 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:46:47.605 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:46:47.662 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:46:47.746 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:46:47.789 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:46:47.789 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T12:46:47.880 DEBUG:teuthology.orchestra.run.smithi061:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:46:47.926 DEBUG:teuthology.orchestra.run.smithi061:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T12:46:47.980 DEBUG:teuthology.orchestra.run.smithi061:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:46:48.066 DEBUG:teuthology.orchestra.run.smithi061:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T12:46:48.109 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T12:46:48.110 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T12:46:48.183 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2023-11-28T12:46:48.186 DEBUG:teuthology.orchestra.run.smithi061:> sudo service rsyslog restart 2023-11-28T12:46:48.238 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T12:46:48.287 INFO:teuthology.orchestra.run.smithi061.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T12:46:48.653 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T12:46:48.661 INFO:teuthology.task.internal:Starting timer... 2023-11-28T12:46:48.661 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T12:46:48.690 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T12:46:48.714 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T12:46:48.714 DEBUG:teuthology.orchestra.run.smithi006:> sudo service auditd rotate 2023-11-28T12:46:48.771 INFO:teuthology.orchestra.run.smithi006.stdout:Rotating logs: 2023-11-28T12:46:48.773 DEBUG:teuthology.orchestra.run.smithi061:> sudo service auditd rotate 2023-11-28T12:46:48.839 INFO:teuthology.orchestra.run.smithi061.stdout:Rotating logs: 2023-11-28T12:46:48.841 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T12:46:48.841 DEBUG:teuthology.orchestra.run.smithi006:> /usr/sbin/getenforce 2023-11-28T12:46:48.863 INFO:teuthology.orchestra.run.smithi006.stdout:Permissive 2023-11-28T12:46:48.864 DEBUG:teuthology.orchestra.run.smithi061:> /usr/sbin/getenforce 2023-11-28T12:46:48.890 INFO:teuthology.orchestra.run.smithi061.stdout:Permissive 2023-11-28T12:46:48.890 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi006.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi061.front.sepia.ceph.com': 'permissive'} 2023-11-28T12:46:48.890 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T12:46:48.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:46:48.933 DEBUG:teuthology.orchestra.run.smithi061:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T12:46:48.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T12:46:48.961 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T12:46:48.961 DEBUG:teuthology.orchestra.run.smithi006:> sudo /usr/sbin/setenforce permissive 2023-11-28T12:46:49.018 DEBUG:teuthology.orchestra.run.smithi061:> sudo /usr/sbin/setenforce permissive 2023-11-28T12:46:49.046 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T12:46:49.056 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-28T12:46:49.056 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T12:46:49.075 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T12:46:49.076 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi006.front.sepia.ceph.com,smithi061.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T12:56:53.626 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi006.front.sepia.ceph.com'), Remote(name='ubuntu@smithi061.front.sepia.ceph.com')] 2023-11-28T12:56:53.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-28T12:56:53.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:56:53.700 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-11-28T12:56:53.768 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-28T12:56:53.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-11-28T12:56:53.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T12:56:53.880 DEBUG:teuthology.orchestra.run.smithi061:> true 2023-11-28T12:56:53.954 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-11-28T12:56:53.955 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T12:56:53.964 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T12:56:53.965 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T12:56:53.965 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T12:56:53.968 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T12:56:53.968 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T12:56:54.001 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T12:56:54.016 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T12:56:54.029 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T12:56:54.045 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T12:56:54.081 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: ntpd: command not found 2023-11-28T12:56:54.094 INFO:teuthology.orchestra.run.smithi006.stdout:506 Cannot talk to daemon 2023-11-28T12:56:54.108 INFO:teuthology.orchestra.run.smithi061.stderr:sudo: ntpd: command not found 2023-11-28T12:56:54.109 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T12:56:54.121 INFO:teuthology.orchestra.run.smithi061.stdout:506 Cannot talk to daemon 2023-11-28T12:56:54.125 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T12:56:54.136 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T12:56:54.151 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T12:56:54.202 INFO:teuthology.orchestra.run.smithi006.stderr:bash: ntpq: command not found 2023-11-28T12:56:54.205 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T12:56:54.205 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2023-11-28T12:56:54.205 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:54.205 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:54.205 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:54.205 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:54.235 INFO:teuthology.orchestra.run.smithi061.stderr:bash: ntpq: command not found 2023-11-28T12:56:54.238 INFO:teuthology.orchestra.run.smithi061.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T12:56:54.238 INFO:teuthology.orchestra.run.smithi061.stdout:=============================================================================== 2023-11-28T12:56:54.238 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:54.238 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:54.239 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:54.239 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T12:56:54.240 INFO:teuthology.run_tasks:Running task install... 2023-11-28T12:56:54.248 DEBUG:teuthology.task.install:project ceph 2023-11-28T12:56:54.248 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}} 2023-11-28T12:56:54.248 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'} 2023-11-28T12:56:54.248 INFO:teuthology.task.install:Using flavor: default 2023-11-28T12:56:54.254 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T12:56:54.254 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T12:56:54.255 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T12:56:54.255 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:56:54.257 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T12:56:54.257 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:56:54.397 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T12:56:54.397 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T12:56:54.407 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T12:56:54.408 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T12:56:54.510 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T12:56:54.510 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:56:54.510 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T12:56:54.540 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T12:56:54.540 DEBUG:teuthology.orchestra.run.smithi006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T12:56:54.577 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T12:56:54.577 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T12:56:54.577 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T12:56:54.606 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T12:56:54.606 DEBUG:teuthology.orchestra.run.smithi061:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T12:56:54.609 DEBUG:teuthology.orchestra.run.smithi006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T12:56:54.676 DEBUG:teuthology.orchestra.run.smithi061:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T12:56:54.687 DEBUG:teuthology.orchestra.run.smithi006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T12:56:54.755 DEBUG:teuthology.orchestra.run.smithi061:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T12:56:54.756 INFO:teuthology.orchestra.run.smithi006.stdout:check_obsoletes = 1 2023-11-28T12:56:54.756 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum clean all 2023-11-28T12:56:54.821 INFO:teuthology.orchestra.run.smithi061.stdout:check_obsoletes = 1 2023-11-28T12:56:54.823 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum clean all 2023-11-28T12:56:55.146 INFO:teuthology.orchestra.run.smithi006.stdout:Updating Subscription Management repositories. 2023-11-28T12:56:55.214 INFO:teuthology.orchestra.run.smithi061.stdout:Updating Subscription Management repositories. 2023-11-28T12:56:56.284 INFO:teuthology.orchestra.run.smithi006.stdout:44 files removed 2023-11-28T12:56:56.336 INFO:teuthology.orchestra.run.smithi061.stdout:44 files removed 2023-11-28T12:56:56.339 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T12:56:56.391 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T12:56:56.691 INFO:teuthology.orchestra.run.smithi006.stdout:Updating Subscription Management repositories. 2023-11-28T12:56:56.747 INFO:teuthology.orchestra.run.smithi061.stdout:Updating Subscription Management repositories. 2023-11-28T12:56:58.367 INFO:teuthology.orchestra.run.smithi061.stdout:ceph packages for x86_64 213 kB/s | 80 kB 00:00 2023-11-28T12:56:58.386 INFO:teuthology.orchestra.run.smithi006.stdout:ceph packages for x86_64 174 kB/s | 80 kB 00:00 2023-11-28T12:56:58.689 INFO:teuthology.orchestra.run.smithi006.stdout:ceph noarch packages 35 kB/s | 9.4 kB 00:00 2023-11-28T12:56:58.734 INFO:teuthology.orchestra.run.smithi061.stdout:ceph noarch packages 28 kB/s | 9.4 kB 00:00 2023-11-28T12:56:58.971 INFO:teuthology.orchestra.run.smithi006.stdout:ceph source packages 5.7 kB/s | 1.5 kB 00:00 2023-11-28T12:56:59.060 INFO:teuthology.orchestra.run.smithi061.stdout:ceph source packages 4.9 kB/s | 1.5 kB 00:00 2023-11-28T12:56:59.130 INFO:teuthology.orchestra.run.smithi006.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-11-28T12:56:59.211 INFO:teuthology.orchestra.run.smithi061.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-11-28T12:57:00.791 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 16 MB 00:01 2023-11-28T12:57:00.972 INFO:teuthology.orchestra.run.smithi061.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 16 MB 00:01 2023-11-28T12:57:04.514 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 473 kB/s | 24 kB 00:00 2023-11-28T12:57:04.653 INFO:teuthology.orchestra.run.smithi061.stdout:lab-extras 557 kB/s | 24 kB 00:00 2023-11-28T12:57:05.816 INFO:teuthology.orchestra.run.smithi006.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 MB/s | 53 MB 00:01 2023-11-28T12:57:05.838 INFO:teuthology.orchestra.run.smithi061.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-11-28T12:57:14.856 INFO:teuthology.orchestra.run.smithi061.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-11-28T12:57:15.209 INFO:teuthology.orchestra.run.smithi006.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 MB/s | 47 MB 00:01 2023-11-28T12:57:26.481 INFO:teuthology.orchestra.run.smithi061.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T12:57:26.482 INFO:teuthology.orchestra.run.smithi061.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T12:57:26.497 INFO:teuthology.orchestra.run.smithi061.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T12:57:26.498 INFO:teuthology.orchestra.run.smithi061.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T12:57:26.500 INFO:teuthology.orchestra.run.smithi061.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T12:57:26.501 INFO:teuthology.orchestra.run.smithi061.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T12:57:26.503 INFO:teuthology.orchestra.run.smithi061.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T12:57:26.506 INFO:teuthology.orchestra.run.smithi061.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T12:57:26.508 INFO:teuthology.orchestra.run.smithi061.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T12:57:26.650 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-11-28T12:57:26.663 INFO:teuthology.orchestra.run.smithi061.stdout:=========================================================================================================================== 2023-11-28T12:57:26.663 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-11-28T12:57:26.663 INFO:teuthology.orchestra.run.smithi061.stdout:=========================================================================================================================== 2023-11-28T12:57:26.663 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-11-28T12:57:26.663 INFO:teuthology.orchestra.run.smithi061.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T12:57:26.663 INFO:teuthology.orchestra.run.smithi061.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T12:57:26.663 INFO:teuthology.orchestra.run.smithi061.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T12:57:26.663 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T12:57:26.664 INFO:teuthology.orchestra.run.smithi061.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T12:57:26.665 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout:Upgrading: 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T12:57:26.666 INFO:teuthology.orchestra.run.smithi061.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T12:57:26.667 INFO:teuthology.orchestra.run.smithi061.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T12:57:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T12:57:26.669 INFO:teuthology.orchestra.run.smithi061.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T12:57:26.670 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T12:57:26.671 INFO:teuthology.orchestra.run.smithi061.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T12:57:26.672 INFO:teuthology.orchestra.run.smithi061.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T12:57:26.672 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T12:57:26.672 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T12:57:26.672 INFO:teuthology.orchestra.run.smithi061.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T12:57:26.672 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T12:57:26.672 INFO:teuthology.orchestra.run.smithi061.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T12:57:26.672 INFO:teuthology.orchestra.run.smithi061.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T12:57:26.673 INFO:teuthology.orchestra.run.smithi061.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T12:57:26.674 INFO:teuthology.orchestra.run.smithi061.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T12:57:26.675 INFO:teuthology.orchestra.run.smithi061.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T12:57:26.675 INFO:teuthology.orchestra.run.smithi061.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T12:57:26.675 INFO:teuthology.orchestra.run.smithi061.stdout:Enabling module streams: 2023-11-28T12:57:26.675 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql 10 2023-11-28T12:57:26.675 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:57:26.675 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-11-28T12:57:26.675 INFO:teuthology.orchestra.run.smithi061.stdout:=========================================================================================================================== 2023-11-28T12:57:26.675 INFO:teuthology.orchestra.run.smithi061.stdout:Install 128 Packages 2023-11-28T12:57:26.675 INFO:teuthology.orchestra.run.smithi061.stdout:Upgrade 3 Packages 2023-11-28T12:57:26.675 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:57:26.676 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 203 M 2023-11-28T12:57:26.676 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-11-28T12:57:26.951 INFO:teuthology.orchestra.run.smithi006.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T12:57:26.952 INFO:teuthology.orchestra.run.smithi006.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T12:57:26.967 INFO:teuthology.orchestra.run.smithi006.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T12:57:26.968 INFO:teuthology.orchestra.run.smithi006.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T12:57:26.969 INFO:teuthology.orchestra.run.smithi006.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T12:57:26.971 INFO:teuthology.orchestra.run.smithi006.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T12:57:26.973 INFO:teuthology.orchestra.run.smithi006.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T12:57:26.976 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T12:57:26.977 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T12:57:27.023 INFO:teuthology.orchestra.run.smithi061.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2023-11-28T12:57:27.120 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-11-28T12:57:27.133 INFO:teuthology.orchestra.run.smithi006.stdout:=========================================================================================================================== 2023-11-28T12:57:27.133 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout:=========================================================================================================================== 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T12:57:27.134 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T12:57:27.135 INFO:teuthology.orchestra.run.smithi006.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T12:57:27.136 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T12:57:27.137 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T12:57:27.138 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T12:57:27.139 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T12:57:27.139 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T12:57:27.139 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T12:57:27.139 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T12:57:27.139 INFO:teuthology.orchestra.run.smithi006.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T12:57:27.139 INFO:teuthology.orchestra.run.smithi006.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T12:57:27.139 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T12:57:27.139 INFO:teuthology.orchestra.run.smithi006.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T12:57:27.139 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T12:57:27.140 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T12:57:27.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T12:57:27.142 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T12:57:27.143 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout:Enabling module streams: 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql 10 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:57:27.144 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-11-28T12:57:27.145 INFO:teuthology.orchestra.run.smithi006.stdout:=========================================================================================================================== 2023-11-28T12:57:27.145 INFO:teuthology.orchestra.run.smithi006.stdout:Install 128 Packages 2023-11-28T12:57:27.145 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 3 Packages 2023-11-28T12:57:27.145 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:57:27.145 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 203 M 2023-11-28T12:57:27.145 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-11-28T12:57:27.493 INFO:teuthology.orchestra.run.smithi006.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-11-28T12:57:28.440 INFO:teuthology.orchestra.run.smithi061.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 597 kB/s | 845 kB 00:01 2023-11-28T12:57:28.710 INFO:teuthology.orchestra.run.smithi006.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 695 kB/s | 845 kB 00:01 2023-11-28T12:57:28.799 INFO:teuthology.orchestra.run.smithi061.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 462 kB/s | 165 kB 00:00 2023-11-28T12:57:28.843 INFO:teuthology.orchestra.run.smithi006.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 1.2 MB/s | 165 kB 00:00 2023-11-28T12:57:30.960 INFO:teuthology.orchestra.run.smithi006.stdout:(4/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 1.1 MB/s | 2.3 MB 00:02 2023-11-28T12:57:31.449 INFO:teuthology.orchestra.run.smithi061.stdout:(4/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 879 kB/s | 2.3 MB 00:02 2023-11-28T12:57:32.094 INFO:teuthology.orchestra.run.smithi006.stdout:(5/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 1.4 MB/s | 1.5 MB 00:01 2023-11-28T12:57:32.365 INFO:teuthology.orchestra.run.smithi061.stdout:(5/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 977 kB/s | 5.4 MB 00:05 2023-11-28T12:57:32.662 INFO:teuthology.orchestra.run.smithi061.stdout:(6/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 1.3 MB/s | 1.5 MB 00:01 2023-11-28T12:57:33.252 INFO:teuthology.orchestra.run.smithi006.stdout:(6/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 906 kB/s | 5.4 MB 00:06 2023-11-28T12:57:33.444 INFO:teuthology.orchestra.run.smithi006.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 3.4 MB/s | 4.6 MB 00:01 2023-11-28T12:57:34.016 INFO:teuthology.orchestra.run.smithi061.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 2.8 MB/s | 4.6 MB 00:01 2023-11-28T12:57:35.744 INFO:teuthology.orchestra.run.smithi006.stdout:(8/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el8 5.7 MB/s | 13 MB 00:02 2023-11-28T12:57:35.778 INFO:teuthology.orchestra.run.smithi006.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 739 kB/s | 24 kB 00:00 2023-11-28T12:57:36.120 INFO:teuthology.orchestra.run.smithi006.stdout:(10/131): ceph-common-18.0.0-2697.g2f5f6028.el8 2.4 MB/s | 22 MB 00:08 2023-11-28T12:57:36.237 INFO:teuthology.orchestra.run.smithi006.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 2.2 MB/s | 257 kB 00:00 2023-11-28T12:57:36.295 INFO:teuthology.orchestra.run.smithi006.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 524 kB/s | 30 kB 00:00 2023-11-28T12:57:36.458 INFO:teuthology.orchestra.run.smithi061.stdout:(8/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el8 5.3 MB/s | 13 MB 00:02 2023-11-28T12:57:36.500 INFO:teuthology.orchestra.run.smithi061.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 588 kB/s | 24 kB 00:00 2023-11-28T12:57:36.504 INFO:teuthology.orchestra.run.smithi006.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 3.6 MB/s | 767 kB 00:00 2023-11-28T12:57:36.572 INFO:teuthology.orchestra.run.smithi006.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 2.0 MB/s | 137 kB 00:00 2023-11-28T12:57:36.630 INFO:teuthology.orchestra.run.smithi006.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 2.3 MB/s | 139 kB 00:00 2023-11-28T12:57:36.805 INFO:teuthology.orchestra.run.smithi006.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 2.6 MB/s | 474 kB 00:00 2023-11-28T12:57:38.330 INFO:teuthology.orchestra.run.smithi006.stdout:(17/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 4.3 MB/s | 6.5 MB 00:01 2023-11-28T12:57:38.406 INFO:teuthology.orchestra.run.smithi006.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 596 kB/s | 45 kB 00:00 2023-11-28T12:57:38.489 INFO:teuthology.orchestra.run.smithi006.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 1.4 MB/s | 117 kB 00:00 2023-11-28T12:57:38.564 INFO:teuthology.orchestra.run.smithi006.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 2.5 MB/s | 190 kB 00:00 2023-11-28T12:57:38.732 INFO:teuthology.orchestra.run.smithi006.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 2.2 MB/s | 371 kB 00:00 2023-11-28T12:57:38.882 INFO:teuthology.orchestra.run.smithi006.stdout:(22/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 2.4 MB/s | 364 kB 00:00 2023-11-28T12:57:38.974 INFO:teuthology.orchestra.run.smithi006.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.1 MB/s | 107 kB 00:00 2023-11-28T12:57:39.066 INFO:teuthology.orchestra.run.smithi006.stdout:(24/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.0 MB/s | 94 kB 00:00 2023-11-28T12:57:39.950 INFO:teuthology.orchestra.run.smithi061.stdout:(10/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 2.4 MB/s | 18 MB 00:07 2023-11-28T12:57:40.158 INFO:teuthology.orchestra.run.smithi061.stdout:(11/131): ceph-common-18.0.0-2697.g2f5f6028.el8 1.6 MB/s | 22 MB 00:13 2023-11-28T12:57:40.183 INFO:teuthology.orchestra.run.smithi006.stdout:(25/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 2.6 MB/s | 18 MB 00:06 2023-11-28T12:57:40.183 INFO:teuthology.orchestra.run.smithi061.stdout:(12/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 1.1 MB/s | 257 kB 00:00 2023-11-28T12:57:40.217 INFO:teuthology.orchestra.run.smithi061.stdout:(13/131): libcephfs-devel-18.0.0-2697.g2f5f6028 522 kB/s | 30 kB 00:00 2023-11-28T12:57:40.258 INFO:teuthology.orchestra.run.smithi006.stdout:(26/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 2.9 MB/s | 3.4 MB 00:01 2023-11-28T12:57:40.284 INFO:teuthology.orchestra.run.smithi061.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 2.0 MB/s | 137 kB 00:00 2023-11-28T12:57:40.316 INFO:teuthology.orchestra.run.smithi006.stdout:(27/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 1.3 MB/s | 180 kB 00:00 2023-11-28T12:57:40.342 INFO:teuthology.orchestra.run.smithi006.stdout:(28/131): ceph-grafana-dashboards-18.0.0-2697.g 276 kB/s | 23 kB 00:00 2023-11-28T12:57:40.367 INFO:teuthology.orchestra.run.smithi061.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 1.6 MB/s | 139 kB 00:00 2023-11-28T12:57:40.383 INFO:teuthology.orchestra.run.smithi006.stdout:(29/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.8 MB/s | 123 kB 00:00 2023-11-28T12:57:40.393 INFO:teuthology.orchestra.run.smithi061.stdout:(16/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 3.6 MB/s | 767 kB 00:00 2023-11-28T12:57:40.543 INFO:teuthology.orchestra.run.smithi061.stdout:(17/131): libradosstriper1-18.0.0-2697.g2f5f602 2.6 MB/s | 474 kB 00:00 2023-11-28T12:57:40.618 INFO:teuthology.orchestra.run.smithi061.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 595 kB/s | 45 kB 00:00 2023-11-28T12:57:40.702 INFO:teuthology.orchestra.run.smithi061.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 1.4 MB/s | 117 kB 00:00 2023-11-28T12:57:40.767 INFO:teuthology.orchestra.run.smithi006.stdout:(30/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 4.1 MB/s | 1.7 MB 00:00 2023-11-28T12:57:40.786 INFO:teuthology.orchestra.run.smithi061.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 2.2 MB/s | 190 kB 00:00 2023-11-28T12:57:40.834 INFO:teuthology.orchestra.run.smithi006.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 3.5 MB/s | 236 kB 00:00 2023-11-28T12:57:40.918 INFO:teuthology.orchestra.run.smithi006.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 568 kB/s | 47 kB 00:00 2023-11-28T12:57:40.919 INFO:teuthology.orchestra.run.smithi061.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 2.7 MB/s | 371 kB 00:00 2023-11-28T12:57:41.001 INFO:teuthology.orchestra.run.smithi006.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 174 kB/s | 15 kB 00:00 2023-11-28T12:57:41.090 INFO:teuthology.orchestra.run.smithi061.stdout:(22/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 2.1 MB/s | 364 kB 00:00 2023-11-28T12:57:41.127 INFO:teuthology.orchestra.run.smithi006.stdout:(34/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 2.1 MB/s | 263 kB 00:00 2023-11-28T12:57:41.170 INFO:teuthology.orchestra.run.smithi061.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.3 MB/s | 107 kB 00:00 2023-11-28T12:57:41.211 INFO:teuthology.orchestra.run.smithi006.stdout:(35/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 993 kB/s | 83 kB 00:00 2023-11-28T12:57:41.254 INFO:teuthology.orchestra.run.smithi061.stdout:(24/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.1 MB/s | 94 kB 00:00 2023-11-28T12:57:41.277 INFO:teuthology.orchestra.run.smithi006.stdout:(36/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 272 kB/s | 18 kB 00:00 2023-11-28T12:57:41.411 INFO:teuthology.orchestra.run.smithi006.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.2 MB/s | 306 kB 00:00 2023-11-28T12:57:41.476 INFO:teuthology.orchestra.run.smithi006.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 903 kB/s | 59 kB 00:00 2023-11-28T12:57:41.513 INFO:teuthology.orchestra.run.smithi006.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 2.1 MB/s | 75 kB 00:00 2023-11-28T12:57:41.596 INFO:teuthology.orchestra.run.smithi006.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 5.6 MB/s | 476 kB 00:00 2023-11-28T12:57:41.638 INFO:teuthology.orchestra.run.smithi006.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-11-28T12:57:41.671 INFO:teuthology.orchestra.run.smithi006.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 5.4 MB/s | 185 kB 00:00 2023-11-28T12:57:41.722 INFO:teuthology.orchestra.run.smithi006.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 662 kB/s | 33 kB 00:00 2023-11-28T12:57:41.755 INFO:teuthology.orchestra.run.smithi006.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 491 kB/s | 16 kB 00:00 2023-11-28T12:57:41.789 INFO:teuthology.orchestra.run.smithi006.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 5.0 MB/s | 173 kB 00:00 2023-11-28T12:57:41.848 INFO:teuthology.orchestra.run.smithi006.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 6.4 MB/s | 384 kB 00:00 2023-11-28T12:57:41.881 INFO:teuthology.orchestra.run.smithi006.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 2.4 MB/s | 81 kB 00:00 2023-11-28T12:57:41.937 INFO:teuthology.orchestra.run.smithi061.stdout:(25/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 4.2 MB/s | 6.5 MB 00:01 2023-11-28T12:57:41.990 INFO:teuthology.orchestra.run.smithi006.stdout:(48/131): ceph-mgr-diskprediction-local-18.0.0- 4.6 MB/s | 7.4 MB 00:01 2023-11-28T12:57:42.013 INFO:teuthology.orchestra.run.smithi061.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 2.4 MB/s | 180 kB 00:00 2023-11-28T12:57:42.015 INFO:teuthology.orchestra.run.smithi006.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-11-28T12:57:42.040 INFO:teuthology.orchestra.run.smithi006.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 351 kB/s | 18 kB 00:00 2023-11-28T12:57:42.080 INFO:teuthology.orchestra.run.smithi061.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 346 kB/s | 23 kB 00:00 2023-11-28T12:57:42.082 INFO:teuthology.orchestra.run.smithi006.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-11-28T12:57:42.124 INFO:teuthology.orchestra.run.smithi006.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-11-28T12:57:42.130 INFO:teuthology.orchestra.run.smithi061.stdout:(28/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 3.9 MB/s | 3.4 MB 00:00 2023-11-28T12:57:42.155 INFO:teuthology.orchestra.run.smithi061.stdout:(29/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.6 MB/s | 123 kB 00:00 2023-11-28T12:57:42.216 INFO:teuthology.orchestra.run.smithi006.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 657 kB/s | 60 kB 00:00 2023-11-28T12:57:42.250 INFO:teuthology.orchestra.run.smithi006.stdout:(54/131): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2023-11-28T12:57:42.275 INFO:teuthology.orchestra.run.smithi006.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-11-28T12:57:42.301 INFO:teuthology.orchestra.run.smithi006.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-11-28T12:57:42.326 INFO:teuthology.orchestra.run.smithi006.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 669 kB/s | 34 kB 00:00 2023-11-28T12:57:42.343 INFO:teuthology.orchestra.run.smithi006.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 4.6 MB/s | 196 kB 00:00 2023-11-28T12:57:42.368 INFO:teuthology.orchestra.run.smithi006.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2023-11-28T12:57:42.393 INFO:teuthology.orchestra.run.smithi006.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2023-11-28T12:57:42.419 INFO:teuthology.orchestra.run.smithi006.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 476 kB/s | 24 kB 00:00 2023-11-28T12:57:42.444 INFO:teuthology.orchestra.run.smithi006.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2023-11-28T12:57:42.469 INFO:teuthology.orchestra.run.smithi006.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 549 kB/s | 27 kB 00:00 2023-11-28T12:57:42.494 INFO:teuthology.orchestra.run.smithi006.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-11-28T12:57:42.497 INFO:teuthology.orchestra.run.smithi061.stdout:(30/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 4.7 MB/s | 1.7 MB 00:00 2023-11-28T12:57:42.528 INFO:teuthology.orchestra.run.smithi006.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.2 MB/s | 251 kB 00:00 2023-11-28T12:57:42.554 INFO:teuthology.orchestra.run.smithi006.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-11-28T12:57:42.579 INFO:teuthology.orchestra.run.smithi006.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-11-28T12:57:42.589 INFO:teuthology.orchestra.run.smithi061.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 2.5 MB/s | 236 kB 00:00 2023-11-28T12:57:42.604 INFO:teuthology.orchestra.run.smithi006.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2023-11-28T12:57:42.664 INFO:teuthology.orchestra.run.smithi061.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 634 kB/s | 47 kB 00:00 2023-11-28T12:57:42.806 INFO:teuthology.orchestra.run.smithi061.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 102 kB/s | 15 kB 00:00 2023-11-28T12:57:42.982 INFO:teuthology.orchestra.run.smithi061.stdout:(34/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 1.5 MB/s | 263 kB 00:00 2023-11-28T12:57:43.049 INFO:teuthology.orchestra.run.smithi061.stdout:(35/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.2 MB/s | 83 kB 00:00 2023-11-28T12:57:43.054 INFO:teuthology.orchestra.run.smithi006.stdout:(69/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 6.4 MB/s | 47 MB 00:07 2023-11-28T12:57:43.071 INFO:teuthology.orchestra.run.smithi006.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 149 kB/s | 70 kB 00:00 2023-11-28T12:57:43.107 INFO:teuthology.orchestra.run.smithi061.stdout:(36/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 310 kB/s | 18 kB 00:00 2023-11-28T12:57:43.113 INFO:teuthology.orchestra.run.smithi006.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 3.2 MB/s | 1.7 MB 00:00 2023-11-28T12:57:43.256 INFO:teuthology.orchestra.run.smithi006.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 234 kB/s | 47 kB 00:00 2023-11-28T12:57:43.281 INFO:teuthology.orchestra.run.smithi006.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 284 kB/s | 43 kB 00:00 2023-11-28T12:57:43.306 INFO:teuthology.orchestra.run.smithi006.stdout:(74/131): libselinux-devel-2.9-5.el8.x86_64.rpm 854 kB/s | 200 kB 00:00 2023-11-28T12:57:43.316 INFO:teuthology.orchestra.run.smithi061.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 6.4 MB/s | 7.4 MB 00:01 2023-11-28T12:57:43.431 INFO:teuthology.orchestra.run.smithi006.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 695 kB/s | 87 kB 00:00 2023-11-28T12:57:43.448 INFO:teuthology.orchestra.run.smithi006.stdout:(76/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2023-11-28T12:57:43.499 INFO:teuthology.orchestra.run.smithi006.stdout:(77/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 221 kB/s | 48 kB 00:00 2023-11-28T12:57:43.532 INFO:teuthology.orchestra.run.smithi006.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.3 MB/s | 109 kB 00:00 2023-11-28T12:57:43.633 INFO:teuthology.orchestra.run.smithi006.stdout:(79/131): python3-requests-oauthlib-1.0.0-1.el8 212 kB/s | 43 kB 00:00 2023-11-28T12:57:43.674 INFO:teuthology.orchestra.run.smithi006.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.9 MB/s | 560 kB 00:00 2023-11-28T12:57:43.716 INFO:teuthology.orchestra.run.smithi006.stdout:(81/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.0 MB/s | 229 kB 00:00 2023-11-28T12:57:43.733 INFO:teuthology.orchestra.run.smithi061.stdout:(38/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 6.4 MB/s | 47 MB 00:07 2023-11-28T12:57:43.750 INFO:teuthology.orchestra.run.smithi006.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 160 kB/s | 19 kB 00:00 2023-11-28T12:57:43.784 INFO:teuthology.orchestra.run.smithi006.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2023-11-28T12:57:43.841 INFO:teuthology.orchestra.run.smithi061.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 417 kB/s | 306 kB 00:00 2023-11-28T12:57:43.859 INFO:teuthology.orchestra.run.smithi006.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 165 kB/s | 18 kB 00:00 2023-11-28T12:57:43.901 INFO:teuthology.orchestra.run.smithi006.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 3.0 MB/s | 559 kB 00:00 2023-11-28T12:57:43.934 INFO:teuthology.orchestra.run.smithi006.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2023-11-28T12:57:43.960 INFO:teuthology.orchestra.run.smithi006.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2023-11-28T12:57:43.975 INFO:teuthology.orchestra.run.smithi061.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 89 kB/s | 59 kB 00:00 2023-11-28T12:57:44.025 INFO:teuthology.orchestra.run.smithi061.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.5 MB/s | 476 kB 00:00 2023-11-28T12:57:44.052 INFO:teuthology.orchestra.run.smithi006.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 618 kB/s | 72 kB 00:00 2023-11-28T12:57:44.067 INFO:teuthology.orchestra.run.smithi061.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 225 kB/s | 75 kB 00:00 2023-11-28T12:57:44.084 INFO:teuthology.orchestra.run.smithi061.stdout:(43/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 400 kB/s | 44 kB 00:00 2023-11-28T12:57:44.085 INFO:teuthology.orchestra.run.smithi006.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 661 kB/s | 83 kB 00:00 2023-11-28T12:57:44.143 INFO:teuthology.orchestra.run.smithi061.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.5 MB/s | 185 kB 00:00 2023-11-28T12:57:44.144 INFO:teuthology.orchestra.run.smithi006.stdout:(90/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 9.6 MB/s | 2.3 MB 00:00 2023-11-28T12:57:44.160 INFO:teuthology.orchestra.run.smithi006.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 551 kB/s | 60 kB 00:00 2023-11-28T12:57:44.168 INFO:teuthology.orchestra.run.smithi061.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 328 kB/s | 33 kB 00:00 2023-11-28T12:57:44.186 INFO:teuthology.orchestra.run.smithi006.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 975 kB/s | 98 kB 00:00 2023-11-28T12:57:44.193 INFO:teuthology.orchestra.run.smithi061.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-11-28T12:57:44.252 INFO:teuthology.orchestra.run.smithi061.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-11-28T12:57:44.261 INFO:teuthology.orchestra.run.smithi006.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 492 kB/s | 58 kB 00:00 2023-11-28T12:57:44.294 INFO:teuthology.orchestra.run.smithi061.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 810 kB/s | 81 kB 00:00 2023-11-28T12:57:44.312 INFO:teuthology.orchestra.run.smithi006.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 5.0 MB/s | 644 kB 00:00 2023-11-28T12:57:44.329 INFO:teuthology.orchestra.run.smithi006.stdout:(95/131): libcom_err-devel-1.45.6-4.el8.x86_64. 230 kB/s | 39 kB 00:00 2023-11-28T12:57:44.353 INFO:teuthology.orchestra.run.smithi061.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-11-28T12:57:44.394 INFO:teuthology.orchestra.run.smithi061.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-11-28T12:57:44.429 INFO:teuthology.orchestra.run.smithi006.stdout:(96/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.0 MB/s | 171 kB 00:00 2023-11-28T12:57:44.455 INFO:teuthology.orchestra.run.smithi006.stdout:(97/131): NetworkManager-initscripts-updown-1.3 966 kB/s | 138 kB 00:00 2023-11-28T12:57:44.495 INFO:teuthology.orchestra.run.smithi061.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 487 kB/s | 49 kB 00:00 2023-11-28T12:57:44.521 INFO:teuthology.orchestra.run.smithi006.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2023-11-28T12:57:44.562 INFO:teuthology.orchestra.run.smithi061.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 5.6 MB/s | 1.2 MB 00:00 2023-11-28T12:57:44.587 INFO:teuthology.orchestra.run.smithi061.stdout:(53/131): python3-cherrypy-18.4.0-1.el8.noarch. 918 kB/s | 384 kB 00:00 2023-11-28T12:57:44.612 INFO:teuthology.orchestra.run.smithi061.stdout:(54/131): python3-more-itertools-7.2.0-3.el8.no 500 kB/s | 59 kB 00:00 2023-11-28T12:57:44.613 INFO:teuthology.orchestra.run.smithi006.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 342 kB/s | 54 kB 00:00 2023-11-28T12:57:44.638 INFO:teuthology.orchestra.run.smithi006.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2023-11-28T12:57:44.663 INFO:teuthology.orchestra.run.smithi061.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 599 kB/s | 60 kB 00:00 2023-11-28T12:57:44.713 INFO:teuthology.orchestra.run.smithi061.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-11-28T12:57:44.763 INFO:teuthology.orchestra.run.smithi061.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 335 kB/s | 34 kB 00:00 2023-11-28T12:57:44.814 INFO:teuthology.orchestra.run.smithi006.stdout:(101/131): python3-scipy-1.0.0-21.module+el8.5. 28 MB/s | 14 MB 00:00 2023-11-28T12:57:44.830 INFO:teuthology.orchestra.run.smithi061.stdout:(58/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-11-28T12:57:44.839 INFO:teuthology.orchestra.run.smithi006.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 194 kB/s | 39 kB 00:00 2023-11-28T12:57:44.864 INFO:teuthology.orchestra.run.smithi006.stdout:(103/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2023-11-28T12:57:44.889 INFO:teuthology.orchestra.run.smithi061.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 513 kB/s | 64 kB 00:00 2023-11-28T12:57:44.939 INFO:teuthology.orchestra.run.smithi061.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-11-28T12:57:44.965 INFO:teuthology.orchestra.run.smithi006.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 4.2 MB/s | 538 kB 00:00 2023-11-28T12:57:44.990 INFO:teuthology.orchestra.run.smithi061.stdout:(61/131): python3-routes-2.4.1-12.el8.noarch.rp 712 kB/s | 196 kB 00:00 2023-11-28T12:57:44.998 INFO:teuthology.orchestra.run.smithi006.stdout:(105/131): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2023-11-28T12:57:45.015 INFO:teuthology.orchestra.run.smithi061.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 190 kB/s | 24 kB 00:00 2023-11-28T12:57:45.023 INFO:teuthology.orchestra.run.smithi006.stdout:(106/131): python3-mako-1.0.6-13.el8.noarch.rpm 988 kB/s | 157 kB 00:00 2023-11-28T12:57:45.040 INFO:teuthology.orchestra.run.smithi061.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2023-11-28T12:57:45.090 INFO:teuthology.orchestra.run.smithi061.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-11-28T12:57:45.115 INFO:teuthology.orchestra.run.smithi006.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 6.0 MB/s | 713 kB 00:00 2023-11-28T12:57:45.116 INFO:teuthology.orchestra.run.smithi061.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-11-28T12:57:45.141 INFO:teuthology.orchestra.run.smithi006.stdout:(108/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-11-28T12:57:45.191 INFO:teuthology.orchestra.run.smithi061.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 610 kB/s | 61 kB 00:00 2023-11-28T12:57:45.216 INFO:teuthology.orchestra.run.smithi061.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 852 kB/s | 85 kB 00:00 2023-11-28T12:57:45.274 INFO:teuthology.orchestra.run.smithi006.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-11-28T12:57:45.291 INFO:teuthology.orchestra.run.smithi061.stdout:(68/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.0 MB/s | 251 kB 00:00 2023-11-28T12:57:45.299 INFO:teuthology.orchestra.run.smithi006.stdout:(110/131): python3-jsonpatch-1.21-2.el8.noarch. 144 kB/s | 27 kB 00:00 2023-11-28T12:57:45.316 INFO:teuthology.orchestra.run.smithi061.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 183 kB/s | 23 kB 00:00 2023-11-28T12:57:45.325 INFO:teuthology.orchestra.run.smithi006.stdout:(111/131): flex-2.6.1-9.el8.x86_64.rpm 1.7 MB/s | 320 kB 00:00 2023-11-28T12:57:45.342 INFO:teuthology.orchestra.run.smithi061.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-11-28T12:57:45.433 INFO:teuthology.orchestra.run.smithi006.stdout:(112/131): python3-prettytable-0.7.2-14.el8.noa 277 kB/s | 44 kB 00:00 2023-11-28T12:57:45.467 INFO:teuthology.orchestra.run.smithi006.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 3.2 MB/s | 457 kB 00:00 2023-11-28T12:57:45.509 INFO:teuthology.orchestra.run.smithi061.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 5.9 MB/s | 1.7 MB 00:00 2023-11-28T12:57:45.551 INFO:teuthology.orchestra.run.smithi061.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 200 kB/s | 47 kB 00:00 2023-11-28T12:57:45.567 INFO:teuthology.orchestra.run.smithi006.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-11-28T12:57:45.584 INFO:teuthology.orchestra.run.smithi061.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 857 kB/s | 200 kB 00:00 2023-11-28T12:57:45.592 INFO:teuthology.orchestra.run.smithi006.stdout:(115/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.4 MB/s | 225 kB 00:00 2023-11-28T12:57:45.659 INFO:teuthology.orchestra.run.smithi006.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.3 MB/s | 126 kB 00:00 2023-11-28T12:57:45.685 INFO:teuthology.orchestra.run.smithi006.stdout:(117/131): uuid-1.6.2-43.el8.x86_64.rpm 294 kB/s | 64 kB 00:00 2023-11-28T12:57:45.710 INFO:teuthology.orchestra.run.smithi006.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 170 kB/s | 20 kB 00:00 2023-11-28T12:57:45.718 INFO:teuthology.orchestra.run.smithi061.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 204 kB/s | 43 kB 00:00 2023-11-28T12:57:45.743 INFO:teuthology.orchestra.run.smithi061.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 303 kB/s | 48 kB 00:00 2023-11-28T12:57:45.768 INFO:teuthology.orchestra.run.smithi061.stdout:(76/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.0 MB/s | 220 kB 00:00 2023-11-28T12:57:45.785 INFO:teuthology.orchestra.run.smithi006.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 824 kB/s | 103 kB 00:00 2023-11-28T12:57:45.810 INFO:teuthology.orchestra.run.smithi061.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 640 kB/s | 43 kB 00:00 2023-11-28T12:57:45.819 INFO:teuthology.orchestra.run.smithi006.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2023-11-28T12:57:45.836 INFO:teuthology.orchestra.run.smithi061.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-11-28T12:57:45.886 INFO:teuthology.orchestra.run.smithi061.stdout:(79/131): libsepol-devel-2.9-3.el8.x86_64.rpm 519 kB/s | 87 kB 00:00 2023-11-28T12:57:45.886 INFO:teuthology.orchestra.run.smithi006.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.9 MB/s | 198 kB 00:00 2023-11-28T12:57:45.920 INFO:teuthology.orchestra.run.smithi061.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2023-11-28T12:57:45.953 INFO:teuthology.orchestra.run.smithi061.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.7 MB/s | 560 kB 00:00 2023-11-28T12:57:45.970 INFO:teuthology.orchestra.run.smithi061.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-11-28T12:57:45.978 INFO:teuthology.orchestra.run.smithi006.stdout:(122/131): openblas-threads-0.3.15-3.el8.x86_64 18 MB/s | 4.9 MB 00:00 2023-11-28T12:57:45.987 INFO:teuthology.orchestra.run.smithi061.stdout:(83/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 185 kB/s | 19 kB 00:00 2023-11-28T12:57:46.028 INFO:teuthology.orchestra.run.smithi006.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2023-11-28T12:57:46.079 INFO:teuthology.orchestra.run.smithi061.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 4.3 MB/s | 559 kB 00:00 2023-11-28T12:57:46.104 INFO:teuthology.orchestra.run.smithi061.stdout:(85/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 134 kB/s | 18 kB 00:00 2023-11-28T12:57:46.112 INFO:teuthology.orchestra.run.smithi006.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-11-28T12:57:46.170 INFO:teuthology.orchestra.run.smithi006.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.8 MB/s | 354 kB 00:00 2023-11-28T12:57:46.188 INFO:teuthology.orchestra.run.smithi061.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2023-11-28T12:57:46.230 INFO:teuthology.orchestra.run.smithi061.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 2.4 MB/s | 605 kB 00:00 2023-11-28T12:57:46.263 INFO:teuthology.orchestra.run.smithi006.stdout:(126/131): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2023-11-28T12:57:46.288 INFO:teuthology.orchestra.run.smithi061.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2023-11-28T12:57:46.304 INFO:teuthology.orchestra.run.smithi006.stdout:(127/131): postgresql-contrib-10.21-2.module+el 4.1 MB/s | 808 kB 00:00 2023-11-28T12:57:46.346 INFO:teuthology.orchestra.run.smithi006.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 8.6 MB/s | 1.5 MB 00:00 2023-11-28T12:57:46.363 INFO:teuthology.orchestra.run.smithi061.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 620 kB/s | 83 kB 00:00 2023-11-28T12:57:46.397 INFO:teuthology.orchestra.run.smithi061.stdout:(90/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 346 kB/s | 72 kB 00:00 2023-11-28T12:57:46.423 INFO:teuthology.orchestra.run.smithi061.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 447 kB/s | 60 kB 00:00 2023-11-28T12:57:46.481 INFO:teuthology.orchestra.run.smithi061.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 690 kB/s | 58 kB 00:00 2023-11-28T12:57:46.523 INFO:teuthology.orchestra.run.smithi061.stdout:(93/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 616 kB/s | 98 kB 00:00 2023-11-28T12:57:46.538 INFO:teuthology.orchestra.run.smithi006.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.1 MB/s | 798 kB 00:00 2023-11-28T12:57:46.605 INFO:teuthology.orchestra.run.smithi006.stdout:(130/131): librados2-18.0.0-2697.g2f5f6028.el8. 11 MB/s | 3.7 MB 00:00 2023-11-28T12:57:46.615 INFO:teuthology.orchestra.run.smithi061.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 201 kB/s | 39 kB 00:00 2023-11-28T12:57:46.649 INFO:teuthology.orchestra.run.smithi061.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 3.8 MB/s | 644 kB 00:00 2023-11-28T12:57:46.674 INFO:teuthology.orchestra.run.smithi061.stdout:(96/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.1 MB/s | 171 kB 00:00 2023-11-28T12:57:46.733 INFO:teuthology.orchestra.run.smithi061.stdout:(97/131): NetworkManager-initscripts-updown-1.3 1.1 MB/s | 138 kB 00:00 2023-11-28T12:57:46.808 INFO:teuthology.orchestra.run.smithi061.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 823 kB/s | 110 kB 00:00 2023-11-28T12:57:46.850 INFO:teuthology.orchestra.run.smithi061.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 463 kB/s | 54 kB 00:00 2023-11-28T12:57:46.925 INFO:teuthology.orchestra.run.smithi061.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2023-11-28T12:57:46.984 INFO:teuthology.orchestra.run.smithi061.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.4 MB/s | 323 kB 00:00 2023-11-28T12:57:47.089 INFO:teuthology.orchestra.run.smithi006.stdout:(131/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 4.5 MB/s | 3.6 MB 00:00 2023-11-28T12:57:47.092 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-11-28T12:57:47.092 INFO:teuthology.orchestra.run.smithi006.stdout:Total 10 MB/s | 203 MB 00:19 2023-11-28T12:57:47.126 INFO:teuthology.orchestra.run.smithi061.stdout:(102/131): python3-scipy-1.0.0-21.module+el8.5. 29 MB/s | 14 MB 00:00 2023-11-28T12:57:47.151 INFO:teuthology.orchestra.run.smithi061.stdout:(103/131): python3-markupsafe-0.23-19.el8.x86_6 172 kB/s | 39 kB 00:00 2023-11-28T12:57:47.185 INFO:teuthology.orchestra.run.smithi061.stdout:(104/131): bison-3.0.4-10.el8.x86_64.rpm 3.4 MB/s | 688 kB 00:00 2023-11-28T12:57:47.268 INFO:teuthology.orchestra.run.smithi061.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2023-11-28T12:57:47.302 INFO:teuthology.orchestra.run.smithi061.stdout:(106/131): python3-jinja2-2.10.1-3.el8.noarch.r 3.0 MB/s | 538 kB 00:00 2023-11-28T12:57:47.327 INFO:teuthology.orchestra.run.smithi061.stdout:(107/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2023-11-28T12:57:47.436 INFO:teuthology.orchestra.run.smithi061.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 245 kB/s | 27 kB 00:00 2023-11-28T12:57:47.469 INFO:teuthology.orchestra.run.smithi061.stdout:(109/131): automake-1.16.1-7.el8.noarch.rpm 3.5 MB/s | 713 kB 00:00 2023-11-28T12:57:47.653 INFO:teuthology.orchestra.run.smithi061.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-11-28T12:57:47.678 INFO:teuthology.orchestra.run.smithi061.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 211 kB/s | 44 kB 00:00 2023-11-28T12:57:47.729 INFO:teuthology.orchestra.run.smithi061.stdout:(112/131): flex-2.6.1-9.el8.x86_64.rpm 1.1 MB/s | 320 kB 00:00 2023-11-28T12:57:47.904 INFO:teuthology.orchestra.run.smithi061.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-11-28T12:57:47.929 INFO:teuthology.orchestra.run.smithi061.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.1 MB/s | 225 kB 00:00 2023-11-28T12:57:47.934 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-11-28T12:57:47.954 INFO:teuthology.orchestra.run.smithi061.stdout:(115/131): python3-werkzeug-0.12.2-4.el8.noarch 1.6 MB/s | 457 kB 00:00 2023-11-28T12:57:48.022 INFO:teuthology.orchestra.run.smithi061.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 547 kB/s | 64 kB 00:00 2023-11-28T12:57:48.065 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-11-28T12:57:48.065 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-11-28T12:57:48.088 INFO:teuthology.orchestra.run.smithi061.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 796 kB/s | 126 kB 00:00 2023-11-28T12:57:48.139 INFO:teuthology.orchestra.run.smithi061.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 109 kB/s | 20 kB 00:00 2023-11-28T12:57:48.189 INFO:teuthology.orchestra.run.smithi061.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 6.9 MB/s | 709 kB 00:00 2023-11-28T12:57:48.222 INFO:teuthology.orchestra.run.smithi061.stdout:(120/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 516 kB/s | 103 kB 00:00 2023-11-28T12:57:48.298 INFO:teuthology.orchestra.run.smithi061.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.8 MB/s | 198 kB 00:00 2023-11-28T12:57:48.482 INFO:teuthology.orchestra.run.smithi061.stdout:(122/131): openblas-threads-0.3.15-3.el8.x86_64 14 MB/s | 4.9 MB 00:00 2023-11-28T12:57:48.498 INFO:teuthology.orchestra.run.smithi061.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 1.0 MB/s | 202 kB 00:00 2023-11-28T12:57:48.624 INFO:teuthology.orchestra.run.smithi061.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-11-28T12:57:48.657 INFO:teuthology.orchestra.run.smithi061.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.0 MB/s | 354 kB 00:00 2023-11-28T12:57:48.782 INFO:teuthology.orchestra.run.smithi061.stdout:(126/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-11-28T12:57:48.825 INFO:teuthology.orchestra.run.smithi061.stdout:(127/131): postgresql-contrib-10.21-2.module+el 3.9 MB/s | 808 kB 00:00 2023-11-28T12:57:48.866 INFO:teuthology.orchestra.run.smithi061.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 7.2 MB/s | 1.5 MB 00:00 2023-11-28T12:57:48.992 INFO:teuthology.orchestra.run.smithi061.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 6.2 MB/s | 798 kB 00:00 2023-11-28T12:57:49.184 INFO:teuthology.orchestra.run.smithi061.stdout:(130/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 9.9 MB/s | 3.6 MB 00:00 2023-11-28T12:57:49.233 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-11-28T12:57:49.237 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-11-28T12:57:49.392 INFO:teuthology.orchestra.run.smithi061.stdout:(131/131): librados2-18.0.0-2697.g2f5f6028.el8. 6.0 MB/s | 3.7 MB 00:00 2023-11-28T12:57:49.395 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-11-28T12:57:49.395 INFO:teuthology.orchestra.run.smithi061.stdout:Total 8.9 MB/s | 203 MB 00:22 2023-11-28T12:57:50.232 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-11-28T12:57:50.363 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-11-28T12:57:50.363 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-11-28T12:57:51.535 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-11-28T12:57:51.539 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-11-28T12:57:54.086 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-11-28T12:57:54.245 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T12:57:54.414 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T12:57:54.556 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T12:57:54.696 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T12:57:54.897 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T12:57:55.325 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T12:57:55.555 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T12:57:55.981 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T12:57:56.155 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:57:56.267 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:57:56.463 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:57:56.562 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:57:56.646 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-11-28T12:57:56.719 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T12:57:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T12:57:56.890 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-28T12:57:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T12:57:57.115 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T12:57:57.175 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T12:57:57.316 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-28T12:57:57.359 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T12:57:57.491 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T12:57:57.751 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T12:57:57.901 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-28T12:57:57.985 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T12:57:58.125 INFO:teuthology.orchestra.run.smithi061.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T12:57:58.155 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T12:57:58.254 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T12:57:58.454 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T12:57:58.539 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 5/134 2023-11-28T12:57:58.613 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T12:57:58.733 INFO:teuthology.orchestra.run.smithi061.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:57:58.867 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:57:58.885 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-28T12:57:59.041 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:57:59.054 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-28T12:57:59.145 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:57:59.199 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-28T12:57:59.338 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T12:57:59.438 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T12:57:59.466 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T12:57:59.597 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T12:57:59.747 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-28T12:57:59.994 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T12:58:00.195 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T12:58:00.260 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T12:58:00.413 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T12:58:00.596 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T12:58:00.837 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T12:58:01.024 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T12:58:01.040 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T12:58:01.132 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T12:58:01.238 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T12:58:01.341 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T12:58:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T12:58:01.695 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-28T12:58:01.865 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-28T12:58:01.977 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T12:58:02.043 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-28T12:58:02.183 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T12:58:02.217 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-28T12:58:02.337 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-28T12:58:02.399 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-28T12:58:02.466 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T12:58:02.609 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T12:58:02.761 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T12:58:02.882 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T12:58:03.100 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-28T12:58:03.107 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T12:58:03.279 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T12:58:03.299 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T12:58:03.932 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-28T12:58:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T12:58:04.857 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-28T12:58:05.100 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-28T12:58:05.168 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T12:58:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-28T12:58:05.545 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T12:58:05.714 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T12:58:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T12:58:06.218 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-28T12:58:06.219 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T12:58:06.407 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T12:58:06.553 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-28T12:58:06.810 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-28T12:58:07.006 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-28T12:58:07.202 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T12:58:07.375 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T12:58:07.569 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T12:58:07.711 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T12:58:07.829 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-28T12:58:07.949 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-28T12:58:08.104 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T12:58:08.180 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T12:58:08.313 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-28T12:58:08.406 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-28T12:58:08.635 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-28T12:58:08.812 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T12:58:08.892 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T12:58:09.114 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-28T12:58:09.266 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-28T12:58:09.366 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-28T12:58:09.423 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-28T12:58:09.600 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-28T12:58:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-28T12:58:09.757 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-28T12:58:09.901 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-28T12:58:09.946 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-28T12:58:10.081 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-28T12:58:10.170 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-28T12:58:10.290 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-28T12:58:10.371 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T12:58:10.560 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T12:58:10.659 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-28T12:58:10.740 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T12:58:10.907 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-28T12:58:10.913 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T12:58:11.100 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-28T12:58:11.181 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-28T12:58:11.308 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-28T12:58:11.325 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-28T12:58:11.498 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T12:58:11.544 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-28T12:58:11.573 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-28T12:58:11.787 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-28T12:58:11.842 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-28T12:58:11.994 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-28T12:58:12.079 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-28T12:58:12.139 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T12:58:12.201 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-28T12:58:12.248 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-28T12:58:12.414 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-28T12:58:12.461 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-28T12:58:12.553 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-28T12:58:12.610 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-28T12:58:12.718 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-28T12:58:12.800 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-28T12:58:12.926 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-28T12:58:13.067 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-28T12:58:13.117 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-28T12:58:13.228 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-28T12:58:13.342 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-28T12:58:13.417 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-28T12:58:13.570 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-28T12:58:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-28T12:58:13.763 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-28T12:58:13.911 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T12:58:13.976 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-28T12:58:14.108 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T12:58:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-28T12:58:14.372 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T12:58:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-28T12:58:14.561 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T12:58:14.691 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T12:58:14.709 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-28T12:58:14.832 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T12:58:14.929 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-28T12:58:15.000 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T12:58:15.169 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T12:58:15.248 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-28T12:58:15.317 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T12:58:15.456 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-28T12:58:15.490 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T12:58:15.626 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T12:58:15.650 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-28T12:58:15.838 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T12:58:15.847 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-28T12:58:16.013 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-28T12:58:16.042 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T12:58:16.177 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-28T12:58:16.226 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T12:58:16.474 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T12:58:16.507 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-28T12:58:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-28T12:58:16.778 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T12:58:16.921 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T12:58:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-28T12:58:17.102 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T12:58:17.216 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-28T12:58:17.364 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T12:58:17.568 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T12:58:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T12:58:18.054 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T12:58:18.218 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T12:58:18.253 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T12:58:18.400 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T12:58:18.484 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T12:58:18.600 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T12:58:18.696 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T12:58:18.793 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T12:58:18.967 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T12:58:18.990 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T12:58:19.125 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T12:58:19.262 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T12:58:19.341 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T12:58:19.472 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T12:58:19.541 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T12:58:19.727 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T12:58:19.911 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T12:58:20.159 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T12:58:20.343 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T12:58:20.413 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T12:58:20.519 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T12:58:20.580 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T12:58:20.597 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T12:58:20.769 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T12:58:20.792 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T12:58:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T12:58:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T12:58:22.414 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T12:58:22.731 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T12:58:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T12:58:23.187 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T12:58:23.994 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T12:58:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T12:58:24.231 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T12:58:24.388 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T12:58:24.681 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:24.878 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:25.500 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:25.743 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T12:58:25.853 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T12:58:25.993 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T12:58:28.211 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:28.405 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:29.018 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T12:58:29.307 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T12:58:29.395 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T12:58:29.579 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T12:58:45.493 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T12:58:45.493 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /sys 2023-11-28T12:58:45.493 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /proc 2023-11-28T12:58:45.493 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /dev 2023-11-28T12:58:45.493 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /run 2023-11-28T12:58:45.493 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /mnt 2023-11-28T12:58:45.493 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /var/tmp 2023-11-28T12:58:45.493 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /home 2023-11-28T12:58:45.493 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /tmp 2023-11-28T12:58:45.494 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /dev 2023-11-28T12:58:45.494 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:58:45.858 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T12:58:47.543 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T12:58:47.698 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T12:58:47.929 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T12:58:48.066 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T12:58:48.556 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T12:58:48.732 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T12:58:48.750 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T12:58:48.944 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T12:58:48.944 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /sys 2023-11-28T12:58:48.944 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /proc 2023-11-28T12:58:48.944 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /dev 2023-11-28T12:58:48.944 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /run 2023-11-28T12:58:48.944 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /mnt 2023-11-28T12:58:48.944 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /var/tmp 2023-11-28T12:58:48.944 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /home 2023-11-28T12:58:48.944 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /tmp 2023-11-28T12:58:48.944 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /dev 2023-11-28T12:58:48.945 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:58:49.269 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T12:58:50.601 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T12:58:50.759 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T12:58:50.872 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T12:58:50.944 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T12:58:51.065 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T12:58:51.104 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T12:58:51.232 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T12:58:51.334 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T12:58:51.455 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T12:58:51.485 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T12:58:51.756 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T12:58:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T12:58:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T12:58:52.219 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T12:58:52.786 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T12:58:52.931 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T12:58:53.026 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T12:58:53.401 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T12:58:53.614 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T12:58:53.650 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T12:58:53.807 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T12:58:53.818 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T12:58:53.953 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T12:58:54.131 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T12:58:54.257 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T12:58:54.441 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T12:58:54.600 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T12:58:54.822 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T12:58:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T12:58:55.136 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T12:58:55.320 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T12:58:55.356 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T12:58:55.498 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T12:58:55.542 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T12:58:55.665 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T12:58:56.034 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T12:58:56.146 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T12:58:56.207 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T12:58:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T12:58:56.378 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T12:58:56.757 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T12:58:56.928 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T12:58:56.965 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T12:58:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T12:58:57.128 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T12:58:58.457 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T12:58:58.663 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T12:58:58.698 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T12:58:58.834 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T12:58:58.877 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T12:58:59.025 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T12:58:59.392 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T12:58:59.576 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T12:59:00.940 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T12:59:01.851 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T12:59:02.139 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-28T12:59:02.294 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 117/134 2023-11-28T12:59:02.526 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-28T12:59:02.705 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T12:59:02.911 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T12:59:03.118 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-28T12:59:03.284 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T12:59:03.415 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T12:59:04.278 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T12:59:05.207 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T12:59:05.528 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-28T12:59:05.687 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 117/134 2023-11-28T12:59:05.813 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:59:05.944 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-28T12:59:06.030 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:59:06.049 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:59:06.074 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T12:59:06.198 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T12:59:06.289 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-28T12:59:06.506 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T12:59:06.537 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-28T12:59:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T12:59:06.802 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T12:59:06.859 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T12:59:07.120 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T12:59:07.296 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T12:59:07.382 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T12:59:07.588 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T12:59:07.601 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T12:59:07.771 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T12:59:07.780 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T12:59:07.891 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-28T12:59:07.891 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:59:08.100 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:59:08.100 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T12:59:08.311 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T12:59:08.488 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:59:09.049 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:59:09.049 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T12:59:09.049 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T12:59:09.049 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T12:59:09.049 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T12:59:09.050 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T12:59:09.050 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:59:09.050 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:59:09.050 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T12:59:09.050 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T12:59:09.050 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T12:59:09.050 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T12:59:09.050 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T12:59:09.050 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T12:59:09.051 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T12:59:09.052 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T12:59:09.053 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T12:59:09.053 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T12:59:09.053 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T12:59:09.053 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T12:59:09.053 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T12:59:09.053 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T12:59:09.053 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T12:59:09.053 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T12:59:09.054 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T12:59:09.054 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T12:59:09.054 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T12:59:09.054 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T12:59:09.054 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T12:59:09.054 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T12:59:09.055 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T12:59:09.055 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T12:59:09.055 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T12:59:09.055 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T12:59:09.055 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T12:59:09.055 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T12:59:09.055 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T12:59:09.056 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T12:59:09.056 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T12:59:09.056 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T12:59:09.056 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T12:59:09.056 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T12:59:09.056 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T12:59:09.056 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T12:59:09.057 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T12:59:09.057 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T12:59:09.057 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T12:59:09.057 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T12:59:09.057 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T12:59:09.057 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T12:59:09.057 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T12:59:09.058 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T12:59:09.058 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-28T12:59:09.058 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-28T12:59:09.058 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-28T12:59:09.058 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-28T12:59:09.058 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-28T12:59:09.059 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-28T12:59:09.059 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-28T12:59:09.059 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-28T12:59:09.059 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-28T12:59:09.059 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-28T12:59:09.059 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-28T12:59:09.060 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-28T12:59:09.060 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-28T12:59:09.060 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-28T12:59:09.060 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-28T12:59:09.060 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-28T12:59:09.060 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-28T12:59:09.060 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-28T12:59:09.061 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-28T12:59:09.061 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-28T12:59:09.061 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-28T12:59:09.061 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-28T12:59:09.061 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-28T12:59:09.061 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-28T12:59:09.062 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-28T12:59:09.062 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-28T12:59:09.062 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-28T12:59:09.062 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-28T12:59:09.062 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-28T12:59:09.062 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-28T12:59:09.063 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-28T12:59:09.063 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-28T12:59:09.063 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-28T12:59:09.063 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-28T12:59:09.063 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-28T12:59:09.064 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-28T12:59:09.064 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-28T12:59:09.064 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-28T12:59:09.064 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-28T12:59:09.064 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-28T12:59:09.064 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-28T12:59:09.065 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-28T12:59:09.065 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-28T12:59:09.065 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-28T12:59:09.065 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-28T12:59:09.065 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-28T12:59:09.065 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-28T12:59:09.065 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-28T12:59:09.065 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-28T12:59:09.065 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-28T12:59:09.065 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-28T12:59:09.066 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-28T12:59:09.066 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-28T12:59:09.066 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-28T12:59:09.066 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-28T12:59:09.066 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-28T12:59:09.066 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-28T12:59:09.066 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T12:59:09.066 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T12:59:09.066 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T12:59:09.066 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:59:09.067 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T12:59:09.197 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:59:09.432 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:59:09.451 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T12:59:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T12:59:09.900 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T12:59:10.223 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T12:59:10.548 INFO:teuthology.orchestra.run.smithi061.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T12:59:10.715 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T12:59:10.800 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-28T12:59:10.998 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T12:59:11.011 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-28T12:59:11.174 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T12:59:11.182 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-28T12:59:11.326 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-28T12:59:11.326 INFO:teuthology.orchestra.run.smithi061.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:59:11.527 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:59:11.527 INFO:teuthology.orchestra.run.smithi061.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T12:59:11.660 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:59:11.660 INFO:teuthology.orchestra.run.smithi006.stdout:Last metadata expiration check: 0:01:57 ago on Tue 28 Nov 2023 12:57:14 PM UTC. 2023-11-28T12:59:11.761 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T12:59:11.906 INFO:teuthology.orchestra.run.smithi061.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:59:12.432 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T12:59:12.433 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T12:59:12.434 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T12:59:12.434 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T12:59:12.434 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T12:59:12.434 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T12:59:12.434 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T12:59:12.434 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T12:59:12.434 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T12:59:12.434 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T12:59:12.434 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T12:59:12.434 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T12:59:12.435 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T12:59:12.435 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T12:59:12.435 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T12:59:12.435 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T12:59:12.435 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T12:59:12.435 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T12:59:12.436 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T12:59:12.436 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T12:59:12.436 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T12:59:12.436 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T12:59:12.436 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T12:59:12.436 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T12:59:12.437 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T12:59:12.437 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T12:59:12.437 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T12:59:12.437 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T12:59:12.437 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T12:59:12.437 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T12:59:12.437 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T12:59:12.438 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T12:59:12.438 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T12:59:12.438 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T12:59:12.438 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T12:59:12.438 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T12:59:12.438 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T12:59:12.438 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T12:59:12.439 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T12:59:12.439 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T12:59:12.439 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T12:59:12.439 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T12:59:12.439 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T12:59:12.439 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T12:59:12.439 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T12:59:12.440 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T12:59:12.440 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T12:59:12.440 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T12:59:12.440 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T12:59:12.440 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T12:59:12.440 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T12:59:12.440 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T12:59:12.441 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T12:59:12.441 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T12:59:12.441 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T12:59:12.441 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T12:59:12.441 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T12:59:12.441 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T12:59:12.441 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T12:59:12.442 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T12:59:12.443 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T12:59:12.443 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T12:59:12.443 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-28T12:59:12.443 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-28T12:59:12.443 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-28T12:59:12.443 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-28T12:59:12.444 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-28T12:59:12.444 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-28T12:59:12.444 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-28T12:59:12.444 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-28T12:59:12.444 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-28T12:59:12.444 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-28T12:59:12.444 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-28T12:59:12.445 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-28T12:59:12.445 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-28T12:59:12.445 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-28T12:59:12.445 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-28T12:59:12.445 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-28T12:59:12.445 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-28T12:59:12.445 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-28T12:59:12.445 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-28T12:59:12.446 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-28T12:59:12.446 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-28T12:59:12.446 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-28T12:59:12.446 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-28T12:59:12.446 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-28T12:59:12.446 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-28T12:59:12.446 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-28T12:59:12.446 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-28T12:59:12.447 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-28T12:59:12.447 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-28T12:59:12.447 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-28T12:59:12.447 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-28T12:59:12.447 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-28T12:59:12.447 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-28T12:59:12.448 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-28T12:59:12.448 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-28T12:59:12.448 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-28T12:59:12.448 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-28T12:59:12.448 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-28T12:59:12.448 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-28T12:59:12.448 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-28T12:59:12.449 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-28T12:59:12.450 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-28T12:59:12.450 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-28T12:59:12.450 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-28T12:59:12.450 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-28T12:59:12.450 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-28T12:59:12.450 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-28T12:59:12.450 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T12:59:12.450 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T12:59:12.451 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T12:59:12.451 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T12:59:12.451 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T12:59:15.036 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T12:59:15.036 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:02:01 ago on Tue 28 Nov 2023 12:57:14 PM UTC. 2023-11-28T12:59:17.740 INFO:teuthology.orchestra.run.smithi006.stdout:Installed products updated. 2023-11-28T12:59:17.984 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:17.985 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2023-11-28T12:59:17.985 INFO:teuthology.orchestra.run.smithi006.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T12:59:17.985 INFO:teuthology.orchestra.run.smithi006.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.985 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.986 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-11-28T12:59:17.986 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T12:59:17.986 INFO:teuthology.orchestra.run.smithi006.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T12:59:17.986 INFO:teuthology.orchestra.run.smithi006.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T12:59:17.986 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.986 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.987 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.987 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.987 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:17.987 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.987 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.987 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.987 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:17.988 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:17.988 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:17.988 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:17.988 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:17.988 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.988 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.989 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:17.989 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.989 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.989 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.989 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:17.989 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:17.990 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.990 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:17.990 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T12:59:17.990 INFO:teuthology.orchestra.run.smithi006.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T12:59:17.990 INFO:teuthology.orchestra.run.smithi006.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T12:59:17.990 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T12:59:17.991 INFO:teuthology.orchestra.run.smithi006.stdout: jq-1.6-3.el8.x86_64 2023-11-28T12:59:17.991 INFO:teuthology.orchestra.run.smithi006.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T12:59:17.991 INFO:teuthology.orchestra.run.smithi006.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T12:59:17.991 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T12:59:17.991 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.991 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.992 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.992 INFO:teuthology.orchestra.run.smithi006.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T12:59:17.992 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T12:59:17.992 INFO:teuthology.orchestra.run.smithi006.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T12:59:17.992 INFO:teuthology.orchestra.run.smithi006.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T12:59:17.993 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T12:59:17.993 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T12:59:17.993 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.993 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.993 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T12:59:17.993 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.994 INFO:teuthology.orchestra.run.smithi006.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T12:59:17.994 INFO:teuthology.orchestra.run.smithi006.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T12:59:17.994 INFO:teuthology.orchestra.run.smithi006.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T12:59:17.994 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T12:59:17.994 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T12:59:17.994 INFO:teuthology.orchestra.run.smithi006.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T12:59:17.994 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T12:59:17.995 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T12:59:17.995 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T12:59:17.995 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T12:59:17.995 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T12:59:17.995 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T12:59:17.995 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T12:59:17.995 INFO:teuthology.orchestra.run.smithi006.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T12:59:17.996 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:17.996 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:17.996 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:17.996 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T12:59:17.996 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T12:59:17.996 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T12:59:17.997 INFO:teuthology.orchestra.run.smithi006.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T12:59:17.997 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T12:59:17.997 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.997 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.997 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:17.998 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T12:59:17.998 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T12:59:17.998 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T12:59:17.998 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T12:59:17.998 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T12:59:17.998 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T12:59:17.998 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T12:59:17.999 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T12:59:18.000 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T12:59:18.000 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T12:59:18.000 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T12:59:18.000 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T12:59:18.000 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T12:59:18.000 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T12:59:18.000 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T12:59:18.000 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T12:59:18.000 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T12:59:18.000 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:18.001 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:18.001 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T12:59:18.001 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T12:59:18.001 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:18.001 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T12:59:18.001 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T12:59:18.001 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T12:59:18.001 INFO:teuthology.orchestra.run.smithi006.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T12:59:18.001 INFO:teuthology.orchestra.run.smithi006.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T12:59:18.002 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T12:59:18.002 INFO:teuthology.orchestra.run.smithi006.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T12:59:18.002 INFO:teuthology.orchestra.run.smithi006.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T12:59:18.002 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T12:59:18.002 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T12:59:18.002 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T12:59:18.002 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T12:59:18.003 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T12:59:18.003 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:18.003 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:18.003 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:18.003 INFO:teuthology.orchestra.run.smithi006.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T12:59:18.003 INFO:teuthology.orchestra.run.smithi006.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T12:59:18.003 INFO:teuthology.orchestra.run.smithi006.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T12:59:18.003 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T12:59:18.004 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T12:59:18.004 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T12:59:18.004 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:18.004 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-11-28T12:59:18.190 DEBUG:teuthology.parallel:result is None 2023-11-28T12:59:24.185 INFO:teuthology.orchestra.run.smithi061.stdout:Installed products updated. 2023-11-28T12:59:24.494 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:24.495 INFO:teuthology.orchestra.run.smithi061.stdout:Upgraded: 2023-11-28T12:59:24.495 INFO:teuthology.orchestra.run.smithi061.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T12:59:24.495 INFO:teuthology.orchestra.run.smithi061.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.495 INFO:teuthology.orchestra.run.smithi061.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.495 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-11-28T12:59:24.495 INFO:teuthology.orchestra.run.smithi061.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T12:59:24.495 INFO:teuthology.orchestra.run.smithi061.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T12:59:24.495 INFO:teuthology.orchestra.run.smithi061.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T12:59:24.495 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:24.496 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:24.497 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.497 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.497 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:24.497 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.497 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.497 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.497 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:24.497 INFO:teuthology.orchestra.run.smithi061.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:24.497 INFO:teuthology.orchestra.run.smithi061.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.497 INFO:teuthology.orchestra.run.smithi061.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: jq-1.6-3.el8.x86_64 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.498 INFO:teuthology.orchestra.run.smithi061.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T12:59:24.499 INFO:teuthology.orchestra.run.smithi061.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T12:59:24.500 INFO:teuthology.orchestra.run.smithi061.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T12:59:24.500 INFO:teuthology.orchestra.run.smithi061.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T12:59:24.500 INFO:teuthology.orchestra.run.smithi061.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T12:59:24.500 INFO:teuthology.orchestra.run.smithi061.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T12:59:24.500 INFO:teuthology.orchestra.run.smithi061.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T12:59:24.500 INFO:teuthology.orchestra.run.smithi061.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T12:59:24.500 INFO:teuthology.orchestra.run.smithi061.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T12:59:24.500 INFO:teuthology.orchestra.run.smithi061.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T12:59:24.500 INFO:teuthology.orchestra.run.smithi061.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T12:59:24.501 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T12:59:24.501 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T12:59:24.501 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.502 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T12:59:24.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T12:59:24.504 INFO:teuthology.orchestra.run.smithi061.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T12:59:24.505 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T12:59:24.506 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.507 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.507 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T12:59:24.507 INFO:teuthology.orchestra.run.smithi061.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T12:59:24.507 INFO:teuthology.orchestra.run.smithi061.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T12:59:24.507 INFO:teuthology.orchestra.run.smithi061.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T12:59:24.507 INFO:teuthology.orchestra.run.smithi061.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T12:59:24.507 INFO:teuthology.orchestra.run.smithi061.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T12:59:24.507 INFO:teuthology.orchestra.run.smithi061.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T12:59:24.508 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:24.508 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-11-28T12:59:24.745 DEBUG:teuthology.parallel:result is None 2023-11-28T12:59:24.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:59:24.887 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T12:59:24.915 INFO:teuthology.orchestra.run.smithi006.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T12:59:24.916 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T12:59:24.916 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T12:59:24.916 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T12:59:25.055 DEBUG:teuthology.orchestra.run.smithi061:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T12:59:25.079 INFO:teuthology.orchestra.run.smithi061.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T12:59:25.080 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T12:59:25.080 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T12:59:25.080 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T12:59:25.080 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:25.080 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T12:59:25.129 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T12:59:25.130 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T12:59:25.172 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T12:59:25.172 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:25.172 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T12:59:25.204 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T12:59:25.274 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T12:59:25.275 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T12:59:25.302 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T12:59:25.369 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T12:59:25.369 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:25.369 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T12:59:25.400 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T12:59:25.469 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T12:59:25.470 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T12:59:25.496 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T12:59:25.563 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T12:59:25.564 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:25.564 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T12:59:25.593 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T12:59:25.661 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T12:59:25.661 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T12:59:25.688 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T12:59:25.755 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T12:59:25.818 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T12:59:25.819 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /var/log/ceph 2023-11-28T12:59:25.821 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 777 /var/log/ceph 2023-11-28T12:59:25.847 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T12:59:25.847 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T12:59:25.890 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T12:59:25.916 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T12:59:25.917 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T12:59:25.958 DEBUG:teuthology.orchestra.run.smithi061:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T12:59:25.997 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T12:59:25.998 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T12:59:25.998 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T12:59:25.998 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:25.998 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T12:59:26.042 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:26.172 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:26.199 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T12:59:26.199 DEBUG:teuthology.orchestra.run.smithi061:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T12:59:26.216 DEBUG:teuthology.orchestra.run.smithi061:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:26.383 DEBUG:teuthology.orchestra.run.smithi061:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:26.415 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T12:59:26.415 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T12:59:26.415 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973', 'branch': 'wip-neesingh-testing-231123', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7469973', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-231123'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'issue_54557', 'sha1': '3ae99413970476692dfab1fe38dadb056700a61d'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6754, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'issue_54557', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '3ae99413970476692dfab1fe38dadb056700a61d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi006.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi061.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_12:38:30', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T12:59:26.416 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T12:59:26.418 DEBUG:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T12:59:26.470 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T12:59:26.476 DEBUG:teuthology.orchestra.run.smithi061:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T12:59:26.538 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:26.538 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T12:59:26.561 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T12:59:26.561 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2023-11-28T12:59:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T12:59:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:26.619 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 384953 Links: 1 2023-11-28T12:59:26.619 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:26.619 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:26.619 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-28 12:59:08.717516532 +0000 2023-11-28T12:59:26.619 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-28 12:56:22.514630152 +0000 2023-11-28T12:59:26.619 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-28 12:56:22.514630152 +0000 2023-11-28T12:59:26.619 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2023-11-28T12:59:26.619 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T12:59:26.685 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-28T12:59:26.685 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-28T12:59:26.685 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000237205 s, 2.2 MB/s 2023-11-28T12:59:26.686 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T12:59:26.744 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2023-11-28T12:59:26.802 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T12:59:26.803 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:26.803 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 387040 Links: 1 2023-11-28T12:59:26.803 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:26.803 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:26.803 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-28 12:59:08.717516532 +0000 2023-11-28T12:59:26.803 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-28 12:56:23.042622901 +0000 2023-11-28T12:59:26.803 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-28 12:56:23.042622901 +0000 2023-11-28T12:59:26.803 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2023-11-28T12:59:26.803 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T12:59:26.869 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-28T12:59:26.869 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-28T12:59:26.869 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000218945 s, 2.3 MB/s 2023-11-28T12:59:26.870 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T12:59:26.929 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2023-11-28T12:59:26.986 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T12:59:26.987 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:26.987 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 394277 Links: 1 2023-11-28T12:59:26.987 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:26.987 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:26.987 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-28 12:59:08.717516532 +0000 2023-11-28T12:59:26.987 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-28 12:56:23.561615774 +0000 2023-11-28T12:59:26.987 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-28 12:56:23.561615774 +0000 2023-11-28T12:59:26.987 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2023-11-28T12:59:26.987 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T12:59:27.053 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-28T12:59:27.053 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-28T12:59:27.053 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000200706 s, 2.6 MB/s 2023-11-28T12:59:27.054 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T12:59:27.112 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2023-11-28T12:59:27.169 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T12:59:27.170 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:27.170 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 6h/6d Inode: 391896 Links: 1 2023-11-28T12:59:27.170 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:27.170 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:27.171 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-28 12:59:08.717516532 +0000 2023-11-28T12:59:27.171 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-28 12:56:24.083608606 +0000 2023-11-28T12:59:27.171 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-28 12:56:24.083608606 +0000 2023-11-28T12:59:27.171 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: - 2023-11-28T12:59:27.171 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T12:59:27.235 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-28T12:59:27.236 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-28T12:59:27.236 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000224578 s, 2.3 MB/s 2023-11-28T12:59:27.237 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T12:59:27.294 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:27.295 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T12:59:27.295 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T12:59:27.314 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T12:59:27.315 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_1 2023-11-28T12:59:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T12:59:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:27.374 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 6h/6d Inode: 390241 Links: 1 2023-11-28T12:59:27.374 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:27.374 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:27.374 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-11-28 12:59:12.108288808 +0000 2023-11-28T12:59:27.374 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-11-28 12:56:18.181544416 +0000 2023-11-28T12:59:27.374 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-11-28 12:56:18.181544416 +0000 2023-11-28T12:59:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: - 2023-11-28T12:59:27.375 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T12:59:27.443 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-11-28T12:59:27.443 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-11-28T12:59:27.443 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000253554 s, 2.0 MB/s 2023-11-28T12:59:27.445 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T12:59:27.505 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_2 2023-11-28T12:59:27.565 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T12:59:27.565 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:27.565 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 6h/6d Inode: 394269 Links: 1 2023-11-28T12:59:27.566 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:27.566 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:27.566 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-11-28 12:59:12.108288808 +0000 2023-11-28T12:59:27.566 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-11-28 12:56:18.714535372 +0000 2023-11-28T12:59:27.566 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-11-28 12:56:18.714535372 +0000 2023-11-28T12:59:27.566 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: - 2023-11-28T12:59:27.566 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T12:59:27.635 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-11-28T12:59:27.635 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-11-28T12:59:27.635 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000240442 s, 2.1 MB/s 2023-11-28T12:59:27.636 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T12:59:27.696 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_3 2023-11-28T12:59:27.755 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T12:59:27.755 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:27.755 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 6h/6d Inode: 388424 Links: 1 2023-11-28T12:59:27.755 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:27.755 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:27.756 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-11-28 12:59:12.108288808 +0000 2023-11-28T12:59:27.756 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-11-28 12:56:19.241526431 +0000 2023-11-28T12:59:27.756 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-11-28 12:56:19.241526431 +0000 2023-11-28T12:59:27.756 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: - 2023-11-28T12:59:27.756 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T12:59:27.823 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-11-28T12:59:27.823 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-11-28T12:59:27.823 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000235962 s, 2.2 MB/s 2023-11-28T12:59:27.825 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T12:59:27.885 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_4 2023-11-28T12:59:27.944 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T12:59:27.944 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T12:59:27.944 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 6h/6d Inode: 387535 Links: 1 2023-11-28T12:59:27.944 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T12:59:27.944 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T12:59:27.944 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-11-28 12:59:12.108288808 +0000 2023-11-28T12:59:27.944 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-11-28 12:56:19.758517659 +0000 2023-11-28T12:59:27.944 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-11-28 12:56:19.758517659 +0000 2023-11-28T12:59:27.945 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: - 2023-11-28T12:59:27.945 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T12:59:28.012 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-11-28T12:59:28.012 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-11-28T12:59:28.012 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000232478 s, 2.2 MB/s 2023-11-28T12:59:28.013 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T12:59:28.072 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:28.072 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi006.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi061.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T12:59:28.073 INFO:tasks.ceph:Generating config... 2023-11-28T12:59:28.074 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T12:59:28.074 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T12:59:28.074 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T12:59:28.074 INFO:tasks.ceph:[client] pid file = 2023-11-28T12:59:28.074 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T12:59:28.074 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T12:59:28.074 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T12:59:28.074 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T12:59:28.074 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T12:59:28.075 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T12:59:28.075 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T12:59:28.075 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T12:59:28.075 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T12:59:28.075 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T12:59:28.075 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T12:59:28.075 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T12:59:28.075 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T12:59:28.075 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T12:59:28.076 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T12:59:28.076 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T12:59:28.076 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T12:59:28.076 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T12:59:28.076 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T12:59:28.076 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T12:59:28.076 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-28T12:59:28.076 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-28T12:59:28.077 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T12:59:28.077 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T12:59:28.077 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-28T12:59:28.077 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T12:59:28.077 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T12:59:28.077 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T12:59:28.077 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T12:59:28.077 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T12:59:28.077 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T12:59:28.077 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T12:59:28.077 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T12:59:28.187 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T12:59:28.190 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T12:59:28.247 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T12:59:28.276 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.6'), ('mon.b', '172.21.15.61'), ('mon.c', '[v2:172.21.15.61:3301,v1:172.21.15.61:6790]')] 2023-11-28T12:59:28.276 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.6,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T12:59:28.277 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:28.277 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T12:59:28.369 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.6 --add b 172.21.15.61 --addv c '[v2:172.21.15.61:3301,v1:172.21.15.61:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:28.460 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:28.460 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool: generated fsid dc643718-e974-4d8a-b829-ff6ced554687 2023-11-28T12:59:28.461 INFO:teuthology.orchestra.run.smithi006.stdout:setting min_mon_release = pacific 2023-11-28T12:59:28.461 INFO:teuthology.orchestra.run.smithi006.stdout:epoch 0 2023-11-28T12:59:28.461 INFO:teuthology.orchestra.run.smithi006.stdout:fsid dc643718-e974-4d8a-b829-ff6ced554687 2023-11-28T12:59:28.461 INFO:teuthology.orchestra.run.smithi006.stdout:last_changed 2023-11-28T12:59:28.460068+0000 2023-11-28T12:59:28.461 INFO:teuthology.orchestra.run.smithi006.stdout:created 2023-11-28T12:59:28.460068+0000 2023-11-28T12:59:28.461 INFO:teuthology.orchestra.run.smithi006.stdout:min_mon_release 16 (pacific) 2023-11-28T12:59:28.462 INFO:teuthology.orchestra.run.smithi006.stdout:election_strategy: 1 2023-11-28T12:59:28.462 INFO:teuthology.orchestra.run.smithi006.stdout:0: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.a 2023-11-28T12:59:28.462 INFO:teuthology.orchestra.run.smithi006.stdout:1: [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] mon.b 2023-11-28T12:59:28.462 INFO:teuthology.orchestra.run.smithi006.stdout:2: [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] mon.c 2023-11-28T12:59:28.462 INFO:teuthology.orchestra.run.smithi006.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T12:59:28.464 DEBUG:teuthology.orchestra.run.smithi006:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T12:59:28.482 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID dc643718-e974-4d8a-b829-ff6ced554687... 2023-11-28T12:59:28.483 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T12:59:28.524 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T12:59:28.569 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2023-11-28T12:59:28.569 INFO:teuthology.orchestra.run.smithi006.stdout: chdir = "" 2023-11-28T12:59:28.570 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T12:59:28.570 INFO:teuthology.orchestra.run.smithi006.stdout: auth supported = cephx 2023-11-28T12:59:28.570 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.570 INFO:teuthology.orchestra.run.smithi006.stdout: filestore xattr use omap = true 2023-11-28T12:59:28.570 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.570 INFO:teuthology.orchestra.run.smithi006.stdout: mon clock drift allowed = 1.000 2023-11-28T12:59:28.570 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.570 INFO:teuthology.orchestra.run.smithi006.stdout: osd crush chooseleaf type = 0 2023-11-28T12:59:28.570 INFO:teuthology.orchestra.run.smithi006.stdout: auth debug = true 2023-11-28T12:59:28.570 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on old message = true 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on bug = true 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: mon pg warn max object skew = 0 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default size = 2 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow primary affinity = true 2023-11-28T12:59:28.571 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow pg remap = true 2023-11-28T12:59:28.573 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on legacy crush tunables = false 2023-11-28T12:59:28.573 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on crush straw calc version zero = false 2023-11-28T12:59:28.573 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on no sortbitwise = false 2023-11-28T12:59:28.574 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on osd down out interval zero = false 2023-11-28T12:59:28.574 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on too few osds = false 2023-11-28T12:59:28.574 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T12:59:28.574 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T12:59:28.574 INFO:teuthology.orchestra.run.smithi006.stdout: mon_allow_pool_size_one = true 2023-11-28T12:59:28.574 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.574 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: osd default data pool replay window = 5 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: mon allow pool delete = true 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: mon cluster log file level = debug 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: debug asserts on shutdown = true 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: mon health detail to clog = false 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: mon host = "172.21.15.6,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = dc643718-e974-4d8a-b829-ff6ced554687 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout:[osd] 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: osd journal size = 100 2023-11-28T12:59:28.575 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.576 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub load threshold = 5.0 2023-11-28T12:59:28.576 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub max interval = 600 2023-11-28T12:59:28.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.576 INFO:teuthology.orchestra.run.smithi006.stdout: osd recover clone overlap = true 2023-11-28T12:59:28.576 INFO:teuthology.orchestra.run.smithi006.stdout: osd recovery max chunk = 1048576 2023-11-28T12:59:28.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.576 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug shutdown = true 2023-11-28T12:59:28.576 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug op order = true 2023-11-28T12:59:28.576 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify stray on activate = true 2023-11-28T12:59:28.577 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.577 INFO:teuthology.orchestra.run.smithi006.stdout: osd open classes on start = true 2023-11-28T12:59:28.577 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug pg log writeout = true 2023-11-28T12:59:28.577 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.577 INFO:teuthology.orchestra.run.smithi006.stdout: osd deep scrub update digest min age = 30 2023-11-28T12:59:28.578 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.578 INFO:teuthology.orchestra.run.smithi006.stdout: osd map max advance = 10 2023-11-28T12:59:28.578 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.578 INFO:teuthology.orchestra.run.smithi006.stdout: journal zero on create = true 2023-11-28T12:59:28.578 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.579 INFO:teuthology.orchestra.run.smithi006.stdout: filestore ondisk finisher threads = 3 2023-11-28T12:59:28.579 INFO:teuthology.orchestra.run.smithi006.stdout: filestore apply finisher threads = 3 2023-11-28T12:59:28.579 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.579 INFO:teuthology.orchestra.run.smithi006.stdout: bdev debug aio = true 2023-11-28T12:59:28.579 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug misdirected ops = true 2023-11-28T12:59:28.579 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore block size = 96636764160 2023-11-28T12:59:28.579 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore fsck on mount = True 2023-11-28T12:59:28.579 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluefs = 20 2023-11-28T12:59:28.579 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluestore = 20 2023-11-28T12:59:28.579 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: debug osd = 20 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: debug rocksdb = 10 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd full ratio = 0.9 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd nearfull ratio = 0.8 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: osd failsafe full ratio = 0.95 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: osd objectstore = bluestore 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr] 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-11-28T12:59:28.580 INFO:teuthology.orchestra.run.smithi006.stdout: debug mgr = 20 2023-11-28T12:59:28.581 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2023-11-28T12:59:28.581 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2023-11-28T12:59:28.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:59:28.581 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:59:28.581 INFO:teuthology.orchestra.run.smithi006.stdout: mgr/telemetry/nag = false 2023-11-28T12:59:28.581 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.581 INFO:teuthology.orchestra.run.smithi006.stdout:[mon] 2023-11-28T12:59:28.581 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-11-28T12:59:28.581 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2023-11-28T12:59:28.581 INFO:teuthology.orchestra.run.smithi006.stdout: debug paxos = 20 2023-11-28T12:59:28.582 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2023-11-28T12:59:28.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon data avail warn = 5 2023-11-28T12:59:28.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon mgr mkfs grace = 240 2023-11-28T12:59:28.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:59:28.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd reporter subtree level = osd 2023-11-28T12:59:28.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd prime pg temp = true 2023-11-28T12:59:28.582 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:59:28.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.583 INFO:teuthology.orchestra.run.smithi006.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T12:59:28.583 INFO:teuthology.orchestra.run.smithi006.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T12:59:28.583 INFO:teuthology.orchestra.run.smithi006.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T12:59:28.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.583 INFO:teuthology.orchestra.run.smithi006.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T12:59:28.583 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T12:59:28.583 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T12:59:28.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.583 INFO:teuthology.orchestra.run.smithi006.stdout: # 1m isn't quite enough 2023-11-28T12:59:28.583 INFO:teuthology.orchestra.run.smithi006.stdout: mon_down_mkfs_grace = 2m 2023-11-28T12:59:28.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.584 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_filestore_osds = false 2023-11-28T12:59:28.584 INFO:teuthology.orchestra.run.smithi006.stdout: mon op complaint time = 120 2023-11-28T12:59:28.584 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T12:59:28.584 INFO:teuthology.orchestra.run.smithi006.stdout:[client] 2023-11-28T12:59:28.584 INFO:teuthology.orchestra.run.smithi006.stdout: rgw cache enabled = true 2023-11-28T12:59:28.584 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable ops log = true 2023-11-28T12:59:28.584 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable usage log = true 2023-11-28T12:59:28.584 INFO:teuthology.orchestra.run.smithi006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T12:59:28.584 INFO:teuthology.orchestra.run.smithi006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T12:59:28.585 INFO:teuthology.orchestra.run.smithi006.stdout: client mount timeout = 600 2023-11-28T12:59:28.585 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2023-11-28T12:59:28.585 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-11-28T12:59:28.585 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = "" 2023-11-28T12:59:28.585 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2023-11-28T12:59:28.585 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2023-11-28T12:59:28.585 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.a] 2023-11-28T12:59:28.585 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.b] 2023-11-28T12:59:28.586 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.c] 2023-11-28T12:59:28.586 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a] 2023-11-28T12:59:28.586 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c] 2023-11-28T12:59:28.586 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b] 2023-11-28T12:59:28.586 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d] 2023-11-28T12:59:28.586 INFO:teuthology.orchestra.run.smithi006.stdout:[mds] 2023-11-28T12:59:28.586 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds = 20 2023-11-28T12:59:28.586 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds balancer = 20 2023-11-28T12:59:28.586 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-11-28T12:59:28.586 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug frag = True 2023-11-28T12:59:28.587 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug scatterstat = True 2023-11-28T12:59:28.587 INFO:teuthology.orchestra.run.smithi006.stdout: mds op complaint time = 180 2023-11-28T12:59:28.587 INFO:teuthology.orchestra.run.smithi006.stdout: mds verify scatter = True 2023-11-28T12:59:28.587 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2023-11-28T12:59:28.587 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2023-11-28T12:59:28.587 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2023-11-28T12:59:28.587 INFO:teuthology.orchestra.run.smithi061.stdout:[global] 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: chdir = "" 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: auth supported = cephx 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: filestore xattr use omap = true 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: mon clock drift allowed = 1.000 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: osd crush chooseleaf type = 0 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: auth debug = true 2023-11-28T12:59:28.588 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: ms die on old message = true 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: ms die on bug = true 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: mon pg warn max object skew = 0 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: osd pool default size = 2 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd allow primary affinity = true 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd allow pg remap = true 2023-11-28T12:59:28.589 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on legacy crush tunables = false 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on crush straw calc version zero = false 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on no sortbitwise = false 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on osd down out interval zero = false 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on too few osds = false 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: mon_allow_pool_size_one = true 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: osd default data pool replay window = 5 2023-11-28T12:59:28.590 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout: mon allow pool delete = true 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout: mon cluster log file level = debug 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout: debug asserts on shutdown = true 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout: mon health detail to clog = false 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout: mon host = "172.21.15.6,172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790]" 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout: fsid = dc643718-e974-4d8a-b829-ff6ced554687 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout:[osd] 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout: osd journal size = 100 2023-11-28T12:59:28.591 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: osd scrub load threshold = 5.0 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: osd scrub max interval = 600 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: osd recover clone overlap = true 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: osd recovery max chunk = 1048576 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug shutdown = true 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug op order = true 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug verify stray on activate = true 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: osd open classes on start = true 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug pg log writeout = true 2023-11-28T12:59:28.592 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: osd deep scrub update digest min age = 30 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: osd map max advance = 10 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: journal zero on create = true 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: filestore ondisk finisher threads = 3 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: filestore apply finisher threads = 3 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: bdev debug aio = true 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug misdirected ops = true 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore block size = 96636764160 2023-11-28T12:59:28.593 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore fsck on mount = True 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: debug bluefs = 20 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: debug bluestore = 20 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: debug osd = 20 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: debug rocksdb = 10 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd full ratio = 0.9 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd nearfull ratio = 0.8 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: osd failsafe full ratio = 0.95 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: osd objectstore = bluestore 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: osd op complaint time = 180 2023-11-28T12:59:28.594 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr] 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: debug mgr = 20 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: debug mon = 20 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: debug auth = 20 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: mgr/telemetry/nag = false 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout:[mon] 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: debug mon = 20 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: debug paxos = 20 2023-11-28T12:59:28.595 INFO:teuthology.orchestra.run.smithi061.stdout: debug auth = 20 2023-11-28T12:59:28.596 INFO:teuthology.orchestra.run.smithi061.stdout: mon data avail warn = 5 2023-11-28T12:59:28.596 INFO:teuthology.orchestra.run.smithi061.stdout: mon mgr mkfs grace = 240 2023-11-28T12:59:28.596 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min pgs per osd = 4 2023-11-28T12:59:28.596 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd reporter subtree level = osd 2023-11-28T12:59:28.596 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd prime pg temp = true 2023-11-28T12:59:28.596 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min bytes per osd = 10 2023-11-28T12:59:28.596 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.596 INFO:teuthology.orchestra.run.smithi061.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T12:59:28.597 INFO:teuthology.orchestra.run.smithi061.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T12:59:28.597 INFO:teuthology.orchestra.run.smithi061.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T12:59:28.597 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.597 INFO:teuthology.orchestra.run.smithi061.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T12:59:28.597 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T12:59:28.597 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T12:59:28.598 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.598 INFO:teuthology.orchestra.run.smithi061.stdout: # 1m isn't quite enough 2023-11-28T12:59:28.598 INFO:teuthology.orchestra.run.smithi061.stdout: mon_down_mkfs_grace = 2m 2023-11-28T12:59:28.598 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.598 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_filestore_osds = false 2023-11-28T12:59:28.598 INFO:teuthology.orchestra.run.smithi061.stdout: mon op complaint time = 120 2023-11-28T12:59:28.598 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-11-28T12:59:28.598 INFO:teuthology.orchestra.run.smithi061.stdout:[client] 2023-11-28T12:59:28.599 INFO:teuthology.orchestra.run.smithi061.stdout: rgw cache enabled = true 2023-11-28T12:59:28.599 INFO:teuthology.orchestra.run.smithi061.stdout: rgw enable ops log = true 2023-11-28T12:59:28.599 INFO:teuthology.orchestra.run.smithi061.stdout: rgw enable usage log = true 2023-11-28T12:59:28.599 INFO:teuthology.orchestra.run.smithi061.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T12:59:28.599 INFO:teuthology.orchestra.run.smithi061.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T12:59:28.599 INFO:teuthology.orchestra.run.smithi061.stdout: client mount timeout = 600 2023-11-28T12:59:28.600 INFO:teuthology.orchestra.run.smithi061.stdout: debug client = 20 2023-11-28T12:59:28.600 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2023-11-28T12:59:28.600 INFO:teuthology.orchestra.run.smithi061.stdout: pid file = "" 2023-11-28T12:59:28.600 INFO:teuthology.orchestra.run.smithi061.stdout: rados mon op timeout = 900 2023-11-28T12:59:28.600 INFO:teuthology.orchestra.run.smithi061.stdout: rados osd op timeout = 900 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.a] 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.b] 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.c] 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.a] 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.c] 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.b] 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.d] 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout:[mds] 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout: debug mds = 20 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout: debug mds balancer = 20 2023-11-28T12:59:28.601 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2023-11-28T12:59:28.602 INFO:teuthology.orchestra.run.smithi061.stdout: mds debug frag = True 2023-11-28T12:59:28.602 INFO:teuthology.orchestra.run.smithi061.stdout: mds debug scatterstat = True 2023-11-28T12:59:28.602 INFO:teuthology.orchestra.run.smithi061.stdout: mds op complaint time = 180 2023-11-28T12:59:28.602 INFO:teuthology.orchestra.run.smithi061.stdout: mds verify scatter = True 2023-11-28T12:59:28.602 INFO:teuthology.orchestra.run.smithi061.stdout: osd op complaint time = 180 2023-11-28T12:59:28.602 INFO:teuthology.orchestra.run.smithi061.stdout: rados mon op timeout = 900 2023-11-28T12:59:28.602 INFO:teuthology.orchestra.run.smithi061.stdout: rados osd op timeout = 900 2023-11-28T12:59:28.603 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T12:59:28.604 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T12:59:28.701 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T12:59:28.701 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:28.702 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T12:59:28.719 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:28.719 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T12:59:28.775 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T12:59:28.777 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:28.777 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T12:59:28.777 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T12:59:28.854 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T12:59:28.854 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:28.913 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T12:59:28.913 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T12:59:28.913 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T12:59:28.913 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T12:59:28.958 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T12:59:28.959 DEBUG:teuthology.orchestra.run.smithi061:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:29.017 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T12:59:29.018 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T12:59:29.018 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T12:59:29.073 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T12:59:29.075 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T12:59:29.140 INFO:teuthology.orchestra.run.smithi061.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T12:59:29.143 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T12:59:29.143 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T12:59:29.198 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T12:59:29.200 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T12:59:29.227 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T12:59:29.322 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T12:59:29.325 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T12:59:29.351 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T12:59:29.411 INFO:teuthology.orchestra.run.smithi061.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T12:59:29.413 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T12:59:29.441 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T12:59:29.537 INFO:teuthology.orchestra.run.smithi061.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T12:59:29.539 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T12:59:29.566 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T12:59:29.566 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T12:59:29.611 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T12:59:29.625 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T12:59:29.668 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T12:59:29.682 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T12:59:29.728 INFO:teuthology.orchestra.run.smithi061.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T12:59:29.741 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T12:59:29.825 INFO:teuthology.orchestra.run.smithi061.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T12:59:29.839 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T12:59:29.839 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi006.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi061.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T12:59:29.840 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T12:59:29.867 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:29.867 INFO:tasks.ceph:role: osd.0 2023-11-28T12:59:29.867 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T12:59:29.867 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T12:59:29.941 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:29.941 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:29.942 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:29.942 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:29.942 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:29.942 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:29.942 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:29.942 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:29.942 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:29.942 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:30.374 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-11-28T12:59:30.376 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-11-28T12:59:30.377 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T12:59:30.551 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T12:59:30.586 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T12:59:30.656 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:30.656 INFO:tasks.ceph:role: osd.1 2023-11-28T12:59:30.656 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T12:59:30.656 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T12:59:30.732 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:30.732 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:30.732 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:30.732 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:30.732 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:30.732 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:30.732 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:30.732 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:30.732 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:30.732 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:31.151 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-11-28T12:59:31.153 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-11-28T12:59:31.153 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T12:59:31.259 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T12:59:31.294 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T12:59:31.364 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:31.364 INFO:tasks.ceph:role: osd.2 2023-11-28T12:59:31.364 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T12:59:31.364 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T12:59:31.441 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:31.441 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:31.441 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:31.442 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:31.442 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:31.442 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:31.442 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:31.442 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:31.442 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:31.442 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:31.886 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-11-28T12:59:31.889 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-11-28T12:59:31.889 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T12:59:32.031 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T12:59:32.066 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T12:59:32.136 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:32.136 INFO:tasks.ceph:role: osd.3 2023-11-28T12:59:32.137 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T12:59:32.137 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T12:59:32.214 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:32.215 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:32.215 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:32.215 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:32.215 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:32.215 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:32.215 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:32.215 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:32.215 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:32.215 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:32.662 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-11-28T12:59:32.664 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-11-28T12:59:32.664 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T12:59:32.855 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T12:59:32.891 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:33.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:33.022+0000 7f1c6b9e0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T12:59:33.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:33.022+0000 7f1c6b9e0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T12:59:33.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:33.022+0000 7f1c6b9e0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T12:59:33.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:33.022+0000 7f1c6b9e0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T12:59:33.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:33.022+0000 7f1c6b9e0640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T12:59:33.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:33.023+0000 7f1c6b9e0640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T12:59:35.897 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T12:59:35.929 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:36.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:36.022+0000 7fefac6df640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T12:59:36.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:36.022+0000 7fefac6df640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T12:59:36.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:36.022+0000 7fefac6df640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T12:59:36.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:36.022+0000 7fefac6df640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T12:59:36.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:36.022+0000 7fefac6df640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T12:59:36.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:36.022+0000 7fefac6df640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T12:59:38.894 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T12:59:38.926 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:39.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:39.018+0000 7f108f7a0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T12:59:39.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:39.018+0000 7f108f7a0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T12:59:39.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:39.018+0000 7f108f7a0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T12:59:39.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:39.018+0000 7f108f7a0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T12:59:39.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:39.018+0000 7f108f7a0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T12:59:39.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:39.018+0000 7f108f7a0640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T12:59:41.891 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T12:59:41.923 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:42.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:42.017+0000 7ff2ffd9c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T12:59:42.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:42.018+0000 7ff2ffd9c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T12:59:42.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:42.018+0000 7ff2ffd9c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T12:59:42.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:42.018+0000 7ff2ffd9c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T12:59:42.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:42.018+0000 7ff2ffd9c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T12:59:42.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T12:59:42.018+0000 7ff2ffd9c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T12:59:44.890 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T12:59:44.923 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T12:59:44.957 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:44.957 INFO:tasks.ceph:role: osd.4 2023-11-28T12:59:44.957 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T12:59:44.957 DEBUG:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T12:59:45.036 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:45.037 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:45.037 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:45.037 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:45.037 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:45.463 INFO:teuthology.orchestra.run.smithi061.stdout:Discarding blocks...Done. 2023-11-28T12:59:45.465 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2023-11-28T12:59:45.465 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T12:59:45.625 DEBUG:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T12:59:45.661 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T12:59:45.732 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:45.732 INFO:tasks.ceph:role: osd.5 2023-11-28T12:59:45.732 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T12:59:45.732 DEBUG:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T12:59:45.809 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:45.809 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:45.809 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:45.809 INFO:teuthology.orchestra.run.smithi061.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:45.809 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:45.809 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:45.809 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:45.809 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:45.809 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:45.809 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:46.236 INFO:teuthology.orchestra.run.smithi061.stdout:Discarding blocks...Done. 2023-11-28T12:59:46.238 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2023-11-28T12:59:46.238 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T12:59:46.358 DEBUG:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T12:59:46.393 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T12:59:46.463 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:46.463 INFO:tasks.ceph:role: osd.6 2023-11-28T12:59:46.463 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T12:59:46.463 DEBUG:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T12:59:46.540 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:46.540 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:46.540 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:46.540 INFO:teuthology.orchestra.run.smithi061.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:46.540 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:46.540 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:46.540 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:46.540 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:46.540 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:46.541 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:46.993 INFO:teuthology.orchestra.run.smithi061.stdout:Discarding blocks...Done. 2023-11-28T12:59:46.995 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2023-11-28T12:59:46.996 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T12:59:47.180 DEBUG:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T12:59:47.219 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T12:59:47.293 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T12:59:47.293 INFO:tasks.ceph:role: osd.7 2023-11-28T12:59:47.293 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T12:59:47.294 DEBUG:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T12:59:47.371 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T12:59:47.372 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T12:59:47.372 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T12:59:47.372 INFO:teuthology.orchestra.run.smithi061.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T12:59:47.372 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T12:59:47.372 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2023-11-28T12:59:47.372 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T12:59:47.373 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T12:59:47.373 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T12:59:47.373 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T12:59:47.805 INFO:teuthology.orchestra.run.smithi061.stdout:Discarding blocks...Done. 2023-11-28T12:59:47.807 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2023-11-28T12:59:47.807 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T12:59:47.964 DEBUG:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T12:59:48.003 DEBUG:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:48.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:48.120+0000 7fe303977640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T12:59:48.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:48.120+0000 7fe303977640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T12:59:48.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:48.120+0000 7fe303977640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T12:59:48.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:48.120+0000 7fe303977640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T12:59:48.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:48.120+0000 7fe303977640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T12:59:48.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:48.120+0000 7fe303977640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T12:59:50.995 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T12:59:51.028 DEBUG:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:51.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:51.119+0000 7f45f4000640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T12:59:51.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:51.120+0000 7f45f4000640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T12:59:51.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:51.120+0000 7f45f4000640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T12:59:51.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:51.120+0000 7f45f4000640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T12:59:51.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:51.120+0000 7f45f4000640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T12:59:51.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:51.120+0000 7f45f4000640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T12:59:53.987 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T12:59:54.020 DEBUG:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:54.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:54.114+0000 7fe1a6916640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T12:59:54.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:54.114+0000 7fe1a6916640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T12:59:54.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:54.114+0000 7fe1a6916640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T12:59:54.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:54.114+0000 7fe1a6916640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T12:59:54.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:54.114+0000 7fe1a6916640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T12:59:54.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:54.114+0000 7fe1a6916640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T12:59:56.418 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:56.422 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T12:59:56.992 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T12:59:57.024 DEBUG:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T12:59:57.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:57.116+0000 7fbe35367640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T12:59:57.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:57.117+0000 7fbe35367640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T12:59:57.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:57.117+0000 7fbe35367640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T12:59:57.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:57.117+0000 7fbe35367640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T12:59:57.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:57.117+0000 7fbe35367640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T12:59:57.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T12:59:57.117+0000 7fbe35367640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T12:59:59.997 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T13:00:00.031 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T13:00:00.031 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:00.031 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T13:00:00.065 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:00.065 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T13:00:00.137 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:00.137 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T13:00:00.207 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:00.207 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T13:00:00.276 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:00.276 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T13:00:00.345 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:00.345 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T13:00:00.414 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:00.414 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T13:00:00.483 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:00.483 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T13:00:00.522 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:00.523 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T13:00:00.593 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:00.593 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T13:00:00.664 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:00.664 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T13:00:00.734 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:00.734 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T13:00:00.804 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:00.804 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T13:00:00.873 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:00.873 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T13:00:00.942 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:00.942 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T13:00:00.965 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:00.965 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T13:00:01.026 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:01.026 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T13:00:01.044 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:01.044 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T13:00:01.102 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T13:00:01.102 DEBUG:teuthology.orchestra.run.smithi006:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T13:00:01.105 DEBUG:teuthology.orchestra.run.smithi061:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T13:00:01.145 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr.y] 2023-11-28T13:00:01.145 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAx5GVl4uVeBBAACyO9UoUP+RyJxFxE8rsFoQ== 2023-11-28T13:00:01.145 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a] 2023-11-28T13:00:01.146 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAx5GVl8nvUCxAA8XF3IdK1G4diiM0tQ5npVA== 2023-11-28T13:00:01.146 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c] 2023-11-28T13:00:01.146 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAx5GVl/bQ5ExAAerGXuTwAj5uC6cL7ScyCZA== 2023-11-28T13:00:01.146 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.0] 2023-11-28T13:00:01.146 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQA15GVlma1kARAA50lI2mIZlVoGT0ks8qcvzw== 2023-11-28T13:00:01.146 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.1] 2023-11-28T13:00:01.146 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQA45GVl4YRiARAAZ7Qktx0lxuvP0H4ceAeVig== 2023-11-28T13:00:01.146 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.2] 2023-11-28T13:00:01.146 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQA75GVlqfAkARAAFyiIXHxyw6s3bPBB2+KINA== 2023-11-28T13:00:01.146 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.3] 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQA+5GVlkjURARAANkhMOAGKdlDfEF9L29hxgA== 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr.x] 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAx5GVlIRViCBAADpzuU8y4HYepMLT1WkF08g== 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b] 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAx5GVlN9+EGBAA8WROC+gNBWJbQmf9ch69iA== 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d] 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAx5GVlo58FIBAAMZ4QjC/4ODXdO7T8j7SEYA== 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.4] 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBE5GVlNMQ9BxAA/gFfGDxTZ9TNAMQcCV2k9w== 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.5] 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBH5GVl91Q8BxAAlVlRbZqylgmq9LkQp/O6jA== 2023-11-28T13:00:01.147 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.6] 2023-11-28T13:00:01.148 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBK5GVlDoTjBhAAOXrgM8Q5MfN3I3aMZgjVvw== 2023-11-28T13:00:01.148 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.7] 2023-11-28T13:00:01.148 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBN5GVlFzINBxAAN6E1A938lHTLYA1GamwdwQ== 2023-11-28T13:00:01.148 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2023-11-28T13:00:01.148 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAx5GVlRYR3JBAADaTghk6Ix3IQVes6gTE56A== 2023-11-28T13:00:01.148 INFO:teuthology.orchestra.run.smithi006.stdout:[client.2] 2023-11-28T13:00:01.149 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAx5GVlDlnhJxAAQG5yCtKJUjh6KGJ3Ennxjg== 2023-11-28T13:00:01.149 INFO:teuthology.orchestra.run.smithi006.stdout:[client.1] 2023-11-28T13:00:01.149 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAx5GVloippKxAAsZK5iYerb15VlfiZ4svbdg== 2023-11-28T13:00:01.149 INFO:teuthology.orchestra.run.smithi006.stdout:[client.3] 2023-11-28T13:00:01.149 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQAx5GVlbJU6MRAAd4WTJLkmaaL+ATXGlIH6ig== 2023-11-28T13:00:01.169 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr.y] 2023-11-28T13:00:01.169 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQAx5GVl4uVeBBAACyO9UoUP+RyJxFxE8rsFoQ== 2023-11-28T13:00:01.169 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.a] 2023-11-28T13:00:01.169 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQAx5GVl8nvUCxAA8XF3IdK1G4diiM0tQ5npVA== 2023-11-28T13:00:01.169 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.c] 2023-11-28T13:00:01.169 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQAx5GVl/bQ5ExAAerGXuTwAj5uC6cL7ScyCZA== 2023-11-28T13:00:01.169 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.0] 2023-11-28T13:00:01.169 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQA15GVlma1kARAA50lI2mIZlVoGT0ks8qcvzw== 2023-11-28T13:00:01.170 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.1] 2023-11-28T13:00:01.170 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQA45GVl4YRiARAAZ7Qktx0lxuvP0H4ceAeVig== 2023-11-28T13:00:01.170 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.2] 2023-11-28T13:00:01.170 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQA75GVlqfAkARAAFyiIXHxyw6s3bPBB2+KINA== 2023-11-28T13:00:01.170 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.3] 2023-11-28T13:00:01.170 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQA+5GVlkjURARAANkhMOAGKdlDfEF9L29hxgA== 2023-11-28T13:00:01.170 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr.x] 2023-11-28T13:00:01.170 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQAx5GVlIRViCBAADpzuU8y4HYepMLT1WkF08g== 2023-11-28T13:00:01.170 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.b] 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQAx5GVlN9+EGBAA8WROC+gNBWJbQmf9ch69iA== 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.d] 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQAx5GVlo58FIBAAMZ4QjC/4ODXdO7T8j7SEYA== 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.4] 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQBE5GVlNMQ9BxAA/gFfGDxTZ9TNAMQcCV2k9w== 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.5] 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQBH5GVl91Q8BxAAlVlRbZqylgmq9LkQp/O6jA== 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.6] 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQBK5GVlDoTjBhAAOXrgM8Q5MfN3I3aMZgjVvw== 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.7] 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQBN5GVlFzINBxAAN6E1A938lHTLYA1GamwdwQ== 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout:[client.0] 2023-11-28T13:00:01.171 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQAx5GVlRYR3JBAADaTghk6Ix3IQVes6gTE56A== 2023-11-28T13:00:01.172 INFO:teuthology.orchestra.run.smithi061.stdout:[client.2] 2023-11-28T13:00:01.172 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQAx5GVlDlnhJxAAQG5yCtKJUjh6KGJ3Ennxjg== 2023-11-28T13:00:01.172 INFO:teuthology.orchestra.run.smithi061.stdout:[client.1] 2023-11-28T13:00:01.172 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQAx5GVloippKxAAsZK5iYerb15VlfiZ4svbdg== 2023-11-28T13:00:01.172 INFO:teuthology.orchestra.run.smithi061.stdout:[client.3] 2023-11-28T13:00:01.172 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQAx5GVlbJU6MRAAd4WTJLkmaaL+ATXGlIH6ig== 2023-11-28T13:00:01.173 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T13:00:01.194 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T13:00:01.268 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:00:01.270 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:00:01.329 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:00:01.331 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:00:01.387 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.390 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.443 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.445 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.499 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.501 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.558 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.560 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.614 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T13:00:01.616 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T13:00:01.669 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:00:01.671 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:00:01.728 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:00:01.730 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T13:00:01.787 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.789 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.843 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.845 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.899 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.901 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.955 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:01.957 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T13:00:02.011 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:00:02.013 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:00:02.069 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:00:02.072 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:00:02.129 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:00:02.131 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:00:02.188 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:00:02.190 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T13:00:02.247 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T13:00:02.248 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T13:00:02.275 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T13:00:02.375 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T13:00:02.403 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T13:00:02.431 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T13:00:02.531 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T13:00:02.558 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T13:00:02.625 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T13:00:02.725 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T13:00:02.753 DEBUG:teuthology.orchestra.run.smithi006:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:00:02.755 DEBUG:teuthology.orchestra.run.smithi061:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T13:00:02.810 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T13:00:02.811 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T13:00:02.811 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T13:00:02.813 INFO:tasks.ceph.mon.a:Started 2023-11-28T13:00:02.813 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T13:00:02.813 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T13:00:02.852 INFO:tasks.ceph.mon.b:Started 2023-11-28T13:00:02.853 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T13:00:02.853 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T13:00:02.854 INFO:tasks.ceph.mon.c:Started 2023-11-28T13:00:02.855 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T13:00:02.855 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T13:00:02.855 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T13:00:02.857 INFO:tasks.ceph.mgr.y:Started 2023-11-28T13:00:02.857 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T13:00:02.858 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T13:00:02.860 INFO:tasks.ceph.mgr.x:Started 2023-11-28T13:00:02.861 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T13:00:02.861 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T13:00:02.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.987+0000 7f74ce668700 1 Processor -- start 2023-11-28T13:00:02.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.987+0000 7f74ce668700 1 -- start start 2023-11-28T13:00:02.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74ce668700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f74c806bf30 0x7f74c806c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:02.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74ce668700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80b6350 con 0x7f74c815c640 2023-11-28T13:00:02.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74ce668700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80d1470 con 0x7f74c80d2820 2023-11-28T13:00:02.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74ce668700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c815ca10 con 0x7f74c806bf30 2023-11-28T13:00:02.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74c7fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f74c815c640 0x7f74c80cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44168/0 (socket says 172.21.15.6:44168) 2023-11-28T13:00:02.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74c7fff700 1 -- 172.21.15.6:0/2560173281 learned_addr learned my addr 172.21.15.6:0/2560173281 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:02.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74ccc05700 1 --2- 172.21.15.6:0/2560173281 >> v2:172.21.15.61:3300/0 conn(0x7f74c806bf30 0x7f74c806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:02.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74ccc05700 1 -- 172.21.15.6:0/2560173281 >> v2:172.21.15.61:3300/0 conn(0x7f74c806bf30 msgr2=0x7f74c806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T13:00:02.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74ccc05700 1 -- 172.21.15.6:0/2560173281 >> v2:172.21.15.61:3300/0 conn(0x7f74c806bf30 msgr2=0x7f74c806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T13:00:02.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74ccc05700 1 --2- 172.21.15.6:0/2560173281 >> v2:172.21.15.61:3300/0 conn(0x7f74c806bf30 0x7f74c806c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T13:00:02.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:02.988+0000 7f74ccc05700 1 --2- 172.21.15.6:0/2560173281 >> v2:172.21.15.61:3300/0 conn(0x7f74c806bf30 0x7f74c806c300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T13:00:03.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.183+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866388025 0 0) 0x7f74c80b6350 con 0x7f74c815c640 2023-11-28T13:00:03.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.183+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74b40035b0 con 0x7f74c815c640 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1098258347 0 0) 0x7f74b40035b0 con 0x7f74c815c640 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 >> v2:172.21.15.61:3300/0 conn(0x7f74c806bf30 msgr2=0x7f74c806c300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 --2- 172.21.15.6:0/2560173281 >> v2:172.21.15.61:3300/0 conn(0x7f74c806bf30 0x7f74c806c300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 >> v1:172.21.15.61:6789/0 conn(0x7f74c80d2820 legacy=0x7f74c80b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c806c840 con 0x7f74c815c640 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f74b0001670 con 0x7f74c815c640 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 >> v1:172.21.15.6:6789/0 conn(0x7f74c815c640 legacy=0x7f74c80cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74b4003a00 0x7f74b4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b4004850 0x7f74b4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b4005200 0x7f74b400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c815ca10 con 0x7f74b4003a00 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b400c170 con 0x7f74b4004850 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b400c330 con 0x7f74b4005200 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.184+0000 7f74c77fe700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b4004850 0x7f74b4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.185+0000 7f74ccc05700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b4005200 0x7f74b400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.185+0000 7f74c7fff700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74b4003a00 0x7f74b4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.185+0000 7f74c77fe700 1 -- 172.21.15.6:0/2560173281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b4005200 msgr2=0x7f74b400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.185+0000 7f74c77fe700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b4005200 0x7f74b400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.185+0000 7f74c77fe700 1 -- 172.21.15.6:0/2560173281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74b4003a00 msgr2=0x7f74b4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.185+0000 7f74c77fe700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74b4003a00 0x7f74b4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.185+0000 7f74ccc05700 1 -- 172.21.15.6:0/2560173281 reap_dead start 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.185+0000 7f74c77fe700 1 -- 172.21.15.6:0/2560173281 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40035b0 con 0x7f74b4004850 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.185+0000 7f74c7fff700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74b4003a00 0x7f74b4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:03.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.185+0000 7f74c77fe700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b4004850 0x7f74b4004cc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74b80017c0 tx=0x7f74b4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ab92c685b7cdfc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.188+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80052d0 con 0x7f74b4004850 2023-11-28T13:00:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.188+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8005450 con 0x7f74b4004850 2023-11-28T13:00:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.188+0000 7f74c6ffd700 1 -- 172.21.15.6:0/2560173281 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80038c0 con 0x7f74b4004850 2023-11-28T13:00:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.189+0000 7f74ce668700 1 -- 172.21.15.6:0/2560173281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b4004850 msgr2=0x7f74b4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.189+0000 7f74ce668700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b4004850 0x7f74b4004cc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74b80017c0 tx=0x7f74b4005880 comp rx=0 tx=0).stop 2023-11-28T13:00:03.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.189+0000 7f74ce668700 1 -- 172.21.15.6:0/2560173281 shutdown_connections 2023-11-28T13:00:03.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.189+0000 7f74ce668700 1 --2- 172.21.15.6:0/2560173281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74b4004850 0x7f74b4004cc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.189+0000 7f74ce668700 1 -- 172.21.15.6:0/2560173281 >> 172.21.15.6:0/2560173281 conn(0x7f74c80d2df0 msgr2=0x7f74c80d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:03.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.190+0000 7f74ce668700 1 -- 172.21.15.6:0/2560173281 shutdown_connections 2023-11-28T13:00:03.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.190+0000 7f74ce668700 1 -- 172.21.15.6:0/2560173281 wait complete. 2023-11-28T13:00:03.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.191+0000 7f74ce668700 1 Processor -- start 2023-11-28T13:00:03.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.192+0000 7f74ce668700 1 -- start start 2023-11-28T13:00:03.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.192+0000 7f74ce668700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b4004850 0x7f74c80cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.192+0000 7f74ce668700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74c8158b90 0x7f74c8158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74ce668700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74c80ca8b0 0x7f74c80cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74ce668700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80cb290 con 0x7f74c8158b90 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74ce668700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80ccc80 con 0x7f74c80ca8b0 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74ce668700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80cced0 con 0x7f74b4004850 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74c8158b90 0x7f74c8158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74c8158b90 0x7f74c8158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56880/0 (socket says 172.21.15.6:56880) 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c77fe700 1 -- 172.21.15.6:0/657329499 learned_addr learned my addr 172.21.15.6:0/657329499 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b4004850 0x7f74c80cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74ccc05700 1 --2- 172.21.15.6:0/657329499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74c80ca8b0 0x7f74c80cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c77fe700 1 -- 172.21.15.6:0/657329499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b4004850 msgr2=0x7f74c80cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c77fe700 1 --2- 172.21.15.6:0/657329499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b4004850 0x7f74c80cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c77fe700 1 -- 172.21.15.6:0/657329499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74c80ca8b0 msgr2=0x7f74c80cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c77fe700 1 --2- 172.21.15.6:0/657329499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74c80ca8b0 0x7f74c80cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c77fe700 1 -- 172.21.15.6:0/657329499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80cd7c0 con 0x7f74c8158b90 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c7fff700 1 --2- 172.21.15.6:0/657329499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b4004850 0x7f74c80cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c77fe700 1 --2- 172.21.15.6:0/657329499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74c8158b90 0x7f74c8158350 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f74b400c300 tx=0x7f74b8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c4ff9700 1 -- 172.21.15.6:0/657329499 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8006ea0 con 0x7f74c8158b90 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c4ff9700 1 -- 172.21.15.6:0/657329499 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8001290 con 0x7f74c8158b90 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74c4ff9700 1 -- 172.21.15.6:0/657329499 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8007280 con 0x7f74c8158b90 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74ce668700 1 -- 172.21.15.6:0/657329499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c80cd4e0 con 0x7f74c8158b90 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.193+0000 7f74ce668700 1 -- 172.21.15.6:0/657329499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c80d1710 con 0x7f74c8158b90 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.194+0000 7f74c4ff9700 1 -- 172.21.15.6:0/657329499 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f74b802d070 con 0x7f74c8158b90 2023-11-28T13:00:03.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.194+0000 7f74c4ff9700 1 -- 172.21.15.6:0/657329499 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f74b80038b0 con 0x7f74c8158b90 2023-11-28T13:00:03.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.194+0000 7f74ce668700 1 -- 172.21.15.6:0/657329499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c8066940 con 0x7f74c8158b90 2023-11-28T13:00:03.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.198+0000 7f74c4ff9700 1 -- 172.21.15.6:0/657329499 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f74b8045b00 con 0x7f74c8158b90 2023-11-28T13:00:03.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.275+0000 7f74ce668700 1 -- 172.21.15.6:0/657329499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f74c80739f0 con 0x7f74c8158b90 2023-11-28T13:00:03.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.278+0000 7f74c4ff9700 1 -- 172.21.15.6:0/657329499 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f74b8034070 con 0x7f74c8158b90 2023-11-28T13:00:03.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.279+0000 7f74ce668700 1 -- 172.21.15.6:0/657329499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74c8158b90 msgr2=0x7f74c8158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.279+0000 7f74ce668700 1 --2- 172.21.15.6:0/657329499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74c8158b90 0x7f74c8158350 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f74b400c300 tx=0x7f74b8001120 comp rx=0 tx=0).stop 2023-11-28T13:00:03.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.279+0000 7f74ce668700 1 -- 172.21.15.6:0/657329499 shutdown_connections 2023-11-28T13:00:03.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.279+0000 7f74ce668700 1 --2- 172.21.15.6:0/657329499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f74c80ca8b0 0x7f74c80cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.279+0000 7f74ce668700 1 --2- 172.21.15.6:0/657329499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f74b4004850 0x7f74c80cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.279+0000 7f74ce668700 1 --2- 172.21.15.6:0/657329499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f74c8158b90 0x7f74c8158350 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.279+0000 7f74ce668700 1 -- 172.21.15.6:0/657329499 >> 172.21.15.6:0/657329499 conn(0x7f74c80d2df0 msgr2=0x7f74c806a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:03.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.279+0000 7f74ce668700 1 -- 172.21.15.6:0/657329499 shutdown_connections 2023-11-28T13:00:03.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:03.279+0000 7f74ce668700 1 -- 172.21.15.6:0/657329499 wait complete. 2023-11-28T13:00:03.281 INFO:teuthology.orchestra.run.smithi006.stderr:adjusted tunables profile to default 2023-11-28T13:00:03.292 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T13:00:03.293 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T13:00:03.293 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:03.293 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T13:00:03.307 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:03.306+0000 7f53b3660200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T13:00:03.311 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:03.310+0000 7f1d19520200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T13:00:03.320 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:03.320 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T13:00:03.377 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:03.376+0000 7f53b3660200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T13:00:03.380 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:03.378+0000 7f1d19520200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T13:00:03.387 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:03.387 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T13:00:03.454 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:03.454 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T13:00:03.457 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:03.456+0000 7f53b3660200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T13:00:03.458 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:03.457+0000 7f1d19520200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T13:00:03.484 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:03.484 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T13:00:03.512 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:03.512 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T13:00:03.542 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:03.541+0000 7f1d19520200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T13:00:03.543 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:03.542+0000 7f53b3660200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T13:00:03.569 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:03.569 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T13:00:03.637 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:00:03.637 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T13:00:03.692 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:03.691+0000 7f53b3660200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T13:00:03.694 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:03.693+0000 7f1d19520200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T13:00:03.704 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 9217a667-026f-41e4-9f6a-4a9bf892353e 0 2023-11-28T13:00:03.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.867+0000 7f63b298f700 1 Processor -- start 2023-11-28T13:00:03.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.867+0000 7f63b298f700 1 -- start start 2023-11-28T13:00:03.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.868+0000 7f63b298f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f63ac0b62d0 0x7f63ac0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.868+0000 7f63b298f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac06f8b0 0x7f63ac0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.868+0000 7f63b298f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ac0b3020 con 0x7f63ac0b2bc0 2023-11-28T13:00:03.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.868+0000 7f63b298f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac0cd2a0 con 0x7f63ac0b62d0 2023-11-28T13:00:03.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.868+0000 7f63b298f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0d0400 con 0x7f63ac06f8b0 2023-11-28T13:00:03.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.868+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac06f8b0 0x7f63ac0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.868+0000 7f63abfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f63ac0b62d0 0x7f63ac0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.868+0000 7f63abfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f63ac0b62d0 0x7f63ac0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57022/0 (socket says 172.21.15.61:57022) 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.868+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac06f8b0 0x7f63ac0b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:59700/0 (socket says 172.21.15.61:59700) 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.868+0000 7f63abfff700 1 -- 172.21.15.61:0/3853785380 learned_addr learned my addr 172.21.15.61:0/3853785380 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.869+0000 7f63abfff700 1 -- 172.21.15.61:0/3853785380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac06f8b0 msgr2=0x7f63ac0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.869+0000 7f63abfff700 1 --2- 172.21.15.61:0/3853785380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac06f8b0 0x7f63ac0b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.869+0000 7f63abfff700 1 -- 172.21.15.61:0/3853785380 >> v1:172.21.15.61:6789/0 conn(0x7f63ac0b2bc0 legacy=0x7f63ac0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.869+0000 7f63abfff700 1 -- 172.21.15.61:0/3853785380 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac0d06e0 con 0x7f63ac0b62d0 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.869+0000 7f63ab7fe700 1 --2- 172.21.15.61:0/3853785380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac06f8b0 0x7f63ac0b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.869+0000 7f63abfff700 1 --2- 172.21.15.61:0/3853785380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b62d0 0x7f63ac0cb630 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f639c0098f0 tx=0x7f639c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d0645bf79827d76 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.869+0000 7f63aaffd700 1 -- 172.21.15.61:0/3853785380 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c011830 con 0x7f63ac0b62d0 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.869+0000 7f63aaffd700 1 -- 172.21.15.61:0/3853785380 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c011990 con 0x7f63ac0b62d0 2023-11-28T13:00:03.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.869+0000 7f63aaffd700 1 -- 172.21.15.61:0/3853785380 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c011d20 con 0x7f63ac0b62d0 2023-11-28T13:00:03.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.870+0000 7f63b298f700 1 -- 172.21.15.61:0/3853785380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b62d0 msgr2=0x7f63ac0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.870+0000 7f63b298f700 1 --2- 172.21.15.61:0/3853785380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b62d0 0x7f63ac0cb630 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f639c0098f0 tx=0x7f639c009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:03.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.870+0000 7f63b298f700 1 -- 172.21.15.61:0/3853785380 shutdown_connections 2023-11-28T13:00:03.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.870+0000 7f63b298f700 1 --2- 172.21.15.61:0/3853785380 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b62d0 0x7f63ac0cb630 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.870+0000 7f63b298f700 1 --2- 172.21.15.61:0/3853785380 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac06f8b0 0x7f63ac0b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.870+0000 7f63b298f700 1 -- 172.21.15.61:0/3853785380 >> 172.21.15.61:0/3853785380 conn(0x7f63ac0cd680 msgr2=0x7f63ac0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:03.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.870+0000 7f63b298f700 1 -- 172.21.15.61:0/3853785380 shutdown_connections 2023-11-28T13:00:03.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.870+0000 7f63b298f700 1 -- 172.21.15.61:0/3853785380 wait complete. 2023-11-28T13:00:03.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.872+0000 7f63b298f700 1 Processor -- start 2023-11-28T13:00:03.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.872+0000 7f63b298f700 1 -- start start 2023-11-28T13:00:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b298f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac06f8b0 0x7f63ac0c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b298f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b2bc0 0x7f63ac14a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b298f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac0b62d0 0x7f63ac1dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b298f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0c7090 con 0x7f63ac06f8b0 2023-11-28T13:00:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b298f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac14a650 con 0x7f63ac0b2bc0 2023-11-28T13:00:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b298f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0c6130 con 0x7f63ac0b62d0 2023-11-28T13:00:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b2bc0 0x7f63ac14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b2bc0 0x7f63ac14a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57030/0 (socket says 172.21.15.61:57030) 2023-11-28T13:00:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b0f2c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac0b62d0 0x7f63ac1dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63ab7fe700 1 -- 172.21.15.61:0/888521397 learned_addr learned my addr 172.21.15.61:0/888521397 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63abfff700 1 --2- 172.21.15.61:0/888521397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac06f8b0 0x7f63ac0c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b0f2c700 1 -- 172.21.15.61:0/888521397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b2bc0 msgr2=0x7f63ac14a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b0f2c700 1 --2- 172.21.15.61:0/888521397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b2bc0 0x7f63ac14a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b0f2c700 1 -- 172.21.15.61:0/888521397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac06f8b0 msgr2=0x7f63ac0c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b0f2c700 1 --2- 172.21.15.61:0/888521397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac06f8b0 0x7f63ac0c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b0f2c700 1 -- 172.21.15.61:0/888521397 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a00094a0 con 0x7f63ac0b62d0 2023-11-28T13:00:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63ab7fe700 1 --2- 172.21.15.61:0/888521397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b2bc0 0x7f63ac14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.873+0000 7f63b0f2c700 1 --2- 172.21.15.61:0/888521397 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac0b62d0 0x7f63ac1dffb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f63a400cb90 tx=0x7f63a400cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.874+0000 7f63a8ff9700 1 -- 172.21.15.61:0/888521397 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4003c00 con 0x7f63ac0b62d0 2023-11-28T13:00:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.874+0000 7f63a8ff9700 1 -- 172.21.15.61:0/888521397 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a4003d60 con 0x7f63ac0b62d0 2023-11-28T13:00:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.874+0000 7f63b298f700 1 -- 172.21.15.61:0/888521397 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c009550 con 0x7f63ac0b62d0 2023-11-28T13:00:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.874+0000 7f63a8ff9700 1 -- 172.21.15.61:0/888521397 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a400aa80 con 0x7f63ac0b62d0 2023-11-28T13:00:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.874+0000 7f63b298f700 1 -- 172.21.15.61:0/888521397 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac0b66a0 con 0x7f63ac0b62d0 2023-11-28T13:00:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.874+0000 7f63a8ff9700 1 -- 172.21.15.61:0/888521397 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f63a4010050 con 0x7f63ac0b62d0 2023-11-28T13:00:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.874+0000 7f63a8ff9700 1 -- 172.21.15.61:0/888521397 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f63a4012760 con 0x7f63ac0b62d0 2023-11-28T13:00:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.875+0000 7f63b298f700 1 -- 172.21.15.61:0/888521397 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac0cffb0 con 0x7f63ac0b62d0 2023-11-28T13:00:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.877+0000 7f63a8ff9700 1 -- 172.21.15.61:0/888521397 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f63a40050d0 con 0x7f63ac0b62d0 2023-11-28T13:00:03.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.963+0000 7f63b298f700 1 -- 172.21.15.61:0/888521397 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9217a667-026f-41e4-9f6a-4a9bf892353e", "id": 0} v 0) v1 -- 0x7f63ac06fd00 con 0x7f63ac0b62d0 2023-11-28T13:00:03.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.966+0000 7f63a8ff9700 1 -- 172.21.15.61:0/888521397 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9217a667-026f-41e4-9f6a-4a9bf892353e", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f63a4005270 con 0x7f63ac0b62d0 2023-11-28T13:00:03.967 INFO:teuthology.orchestra.run.smithi061.stdout:0 2023-11-28T13:00:03.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.967+0000 7f63b298f700 1 -- 172.21.15.61:0/888521397 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac0b62d0 msgr2=0x7f63ac1dffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:03.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.967+0000 7f63b298f700 1 --2- 172.21.15.61:0/888521397 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac0b62d0 0x7f63ac1dffb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f63a400cb90 tx=0x7f63a400cea0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.967+0000 7f63b298f700 1 -- 172.21.15.61:0/888521397 shutdown_connections 2023-11-28T13:00:03.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.967+0000 7f63b298f700 1 --2- 172.21.15.61:0/888521397 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63ac0b62d0 0x7f63ac1dffb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.967+0000 7f63b298f700 1 --2- 172.21.15.61:0/888521397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63ac06f8b0 0x7f63ac0c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.967+0000 7f63b298f700 1 --2- 172.21.15.61:0/888521397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63ac0b2bc0 0x7f63ac14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:03.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.967+0000 7f63b298f700 1 -- 172.21.15.61:0/888521397 >> 172.21.15.61:0/888521397 conn(0x7f63ac0cd680 msgr2=0x7f63ac0b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:03.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.967+0000 7f63b298f700 1 -- 172.21.15.61:0/888521397 shutdown_connections 2023-11-28T13:00:03.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:03.967+0000 7f63b298f700 1 -- 172.21.15.61:0/888521397 wait complete. 2023-11-28T13:00:03.980 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 614e9b07-0417-4fd0-b01b-b4ff14d65052 1 2023-11-28T13:00:04.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.102+0000 7f866c4de700 1 Processor -- start 2023-11-28T13:00:04.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.102+0000 7f866c4de700 1 -- start start 2023-11-28T13:00:04.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f866c4de700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f86640cd140 0x7f866406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f866c4de700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f866406d140 0x7f866406b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f866c4de700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86640ae730 con 0x7f866415a640 2023-11-28T13:00:04.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f866c4de700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86640cb210 con 0x7f866406d140 2023-11-28T13:00:04.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f866c4de700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86640c3e10 con 0x7f86640cd140 2023-11-28T13:00:04.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f8669a79700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f86640cd140 0x7f866406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f8669a79700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f86640cd140 0x7f866406b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57036/0 (socket says 172.21.15.61:57036) 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f8669a79700 1 -- 172.21.15.61:0/179089736 learned_addr learned my addr 172.21.15.61:0/179089736 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f866aa7b700 1 --2- 172.21.15.61:0/179089736 >> v2:172.21.15.6:3300/0 conn(0x7f866406d140 0x7f866406b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f8669a79700 1 -- 172.21.15.61:0/179089736 >> v2:172.21.15.6:3300/0 conn(0x7f866406d140 msgr2=0x7f866406b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f8669a79700 1 --2- 172.21.15.61:0/179089736 >> v2:172.21.15.6:3300/0 conn(0x7f866406d140 0x7f866406b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f8669a79700 1 -- 172.21.15.61:0/179089736 >> v1:172.21.15.61:6789/0 conn(0x7f866415a640 legacy=0x7f86640c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f8669a79700 1 -- 172.21.15.61:0/179089736 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866406e9e0 con 0x7f86640cd140 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.104+0000 7f8669a79700 1 --2- 172.21.15.61:0/179089736 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86640cd140 0x7f866406b1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f86640d0c50 tx=0x7f8654009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66d8e34c370349b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f8669278700 1 -- 172.21.15.61:0/179089736 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654011830 con 0x7f86640cd140 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f8669278700 1 -- 172.21.15.61:0/179089736 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654011990 con 0x7f86640cd140 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f8669278700 1 -- 172.21.15.61:0/179089736 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654011d20 con 0x7f86640cd140 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f866c4de700 1 -- 172.21.15.61:0/179089736 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86640cd140 msgr2=0x7f866406b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f866c4de700 1 --2- 172.21.15.61:0/179089736 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86640cd140 0x7f866406b1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f86640d0c50 tx=0x7f8654009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f866c4de700 1 -- 172.21.15.61:0/179089736 shutdown_connections 2023-11-28T13:00:04.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f866c4de700 1 --2- 172.21.15.61:0/179089736 >> v2:172.21.15.6:3300/0 conn(0x7f866406d140 0x7f866406b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f866c4de700 1 --2- 172.21.15.61:0/179089736 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86640cd140 0x7f866406b1f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f866c4de700 1 -- 172.21.15.61:0/179089736 >> 172.21.15.61:0/179089736 conn(0x7f86640cc630 msgr2=0x7f86640cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:04.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f866c4de700 1 -- 172.21.15.61:0/179089736 shutdown_connections 2023-11-28T13:00:04.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.105+0000 7f866c4de700 1 -- 172.21.15.61:0/179089736 wait complete. 2023-11-28T13:00:04.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.107+0000 7f866c4de700 1 Processor -- start 2023-11-28T13:00:04.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.107+0000 7f866c4de700 1 -- start start 2023-11-28T13:00:04.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f866c4de700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f866406d140 0x7f866414d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f866c4de700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86640cd140 0x7f866415cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f866c4de700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f866415a640 0x7f866415d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f866c4de700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86641f6ab0 con 0x7f866415a640 2023-11-28T13:00:04.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f866c4de700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866414d5d0 con 0x7f866406d140 2023-11-28T13:00:04.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f866c4de700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86641f5ec0 con 0x7f86640cd140 2023-11-28T13:00:04.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f8669a79700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86640cd140 0x7f866415cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f866a27a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f866406d140 0x7f866414d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f8669a79700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86640cd140 0x7f866415cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:59718/0 (socket says 172.21.15.61:59718) 2023-11-28T13:00:04.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f8669a79700 1 -- 172.21.15.61:0/1871866031 learned_addr learned my addr 172.21.15.61:0/1871866031 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:04.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f866aa7b700 1 --2- 172.21.15.61:0/1871866031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f866415a640 0x7f866415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f8669a79700 1 -- 172.21.15.61:0/1871866031 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f866406d140 msgr2=0x7f866414d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f8669a79700 1 --2- 172.21.15.61:0/1871866031 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f866406d140 0x7f866414d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f8669a79700 1 -- 172.21.15.61:0/1871866031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f866415a640 msgr2=0x7f866415d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f8669a79700 1 --2- 172.21.15.61:0/1871866031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f866415a640 0x7f866415d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f8669a79700 1 -- 172.21.15.61:0/1871866031 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8654009550 con 0x7f86640cd140 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f866aa7b700 1 --2- 172.21.15.61:0/1871866031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f866415a640 0x7f866415d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f8669a79700 1 --2- 172.21.15.61:0/1871866031 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86640cd140 0x7f866415cf10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8654009c10 tx=0x7f8654004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f865affd700 1 -- 172.21.15.61:0/1871866031 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654004380 con 0x7f86640cd140 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f865affd700 1 -- 172.21.15.61:0/1871866031 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865401f070 con 0x7f86640cd140 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f866c4de700 1 -- 172.21.15.61:0/1871866031 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86640aec60 con 0x7f86640cd140 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.109+0000 7f865affd700 1 -- 172.21.15.61:0/1871866031 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540036a0 con 0x7f86640cd140 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.110+0000 7f866c4de700 1 -- 172.21.15.61:0/1871866031 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86640ae980 con 0x7f86640cd140 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.110+0000 7f865affd700 1 -- 172.21.15.61:0/1871866031 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f865402a020 con 0x7f86640cd140 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.110+0000 7f865affd700 1 -- 172.21.15.61:0/1871866031 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f865401a660 con 0x7f86640cd140 2023-11-28T13:00:04.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.110+0000 7f866c4de700 1 -- 172.21.15.61:0/1871866031 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86640aef40 con 0x7f86640cd140 2023-11-28T13:00:04.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.113+0000 7f865affd700 1 -- 172.21.15.61:0/1871866031 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f8654015020 con 0x7f86640cd140 2023-11-28T13:00:04.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.197+0000 7f866c4de700 1 -- 172.21.15.61:0/1871866031 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd new", "uuid": "614e9b07-0417-4fd0-b01b-b4ff14d65052", "id": 1} v 0) v1 -- 0x7f86641f6c60 con 0x7f86640cd140 2023-11-28T13:00:04.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.206+0000 7f865affd700 1 -- 172.21.15.61:0/1871866031 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "614e9b07-0417-4fd0-b01b-b4ff14d65052", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f86641f6c60 con 0x7f86640cd140 2023-11-28T13:00:04.207 INFO:teuthology.orchestra.run.smithi061.stdout:1 2023-11-28T13:00:04.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.206+0000 7f866c4de700 1 -- 172.21.15.61:0/1871866031 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86640cd140 msgr2=0x7f866415cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.206+0000 7f866c4de700 1 --2- 172.21.15.61:0/1871866031 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86640cd140 0x7f866415cf10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8654009c10 tx=0x7f8654004750 comp rx=0 tx=0).stop 2023-11-28T13:00:04.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.206+0000 7f866c4de700 1 -- 172.21.15.61:0/1871866031 shutdown_connections 2023-11-28T13:00:04.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.206+0000 7f866c4de700 1 --2- 172.21.15.61:0/1871866031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f866415a640 0x7f866415d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.206+0000 7f866c4de700 1 --2- 172.21.15.61:0/1871866031 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f866406d140 0x7f866414d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.206+0000 7f866c4de700 1 --2- 172.21.15.61:0/1871866031 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86640cd140 0x7f866415cf10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.206+0000 7f866c4de700 1 -- 172.21.15.61:0/1871866031 >> 172.21.15.61:0/1871866031 conn(0x7f86640cc630 msgr2=0x7f86640c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:04.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.206+0000 7f866c4de700 1 -- 172.21.15.61:0/1871866031 shutdown_connections 2023-11-28T13:00:04.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.206+0000 7f866c4de700 1 -- 172.21.15.61:0/1871866031 wait complete. 2023-11-28T13:00:04.221 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3 2 2023-11-28T13:00:04.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.347+0000 7f52d8a9b700 1 Processor -- start 2023-11-28T13:00:04.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.347+0000 7f52d8a9b700 1 -- start start 2023-11-28T13:00:04.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d8a9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d8a9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f52d406afc0 0x7f52d406b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d8a9b700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40ce550 con 0x7f52d40b1a00 2023-11-28T13:00:04.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d8a9b700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d40ce830 con 0x7f52d406afc0 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d8a9b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40ceb10 con 0x7f52d40b1630 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d259c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d259c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:59722/0 (socket says 172.21.15.61:59722) 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d259c700 1 -- 172.21.15.61:0/2728971942 learned_addr learned my addr 172.21.15.61:0/2728971942 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d2d9d700 1 --2- 172.21.15.61:0/2728971942 >> v2:172.21.15.6:3300/0 conn(0x7f52d406afc0 0x7f52d406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d259c700 1 -- 172.21.15.61:0/2728971942 >> v2:172.21.15.6:3300/0 conn(0x7f52d406afc0 msgr2=0x7f52d406b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d259c700 1 --2- 172.21.15.61:0/2728971942 >> v2:172.21.15.6:3300/0 conn(0x7f52d406afc0 0x7f52d406b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d259c700 1 -- 172.21.15.61:0/2728971942 >> v1:172.21.15.6:6789/0 conn(0x7f52d40b1a00 legacy=0x7f52d40caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d259c700 1 -- 172.21.15.61:0/2728971942 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d40cedf0 con 0x7f52d40b1630 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.348+0000 7f52d259c700 1 --2- 172.21.15.61:0/2728971942 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40cf5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f52c40098f0 tx=0x7f52c4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d91bdb0f03296e86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d159a700 1 -- 172.21.15.61:0/2728971942 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4011830 con 0x7f52d40b1630 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d159a700 1 -- 172.21.15.61:0/2728971942 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c4011990 con 0x7f52d40b1630 2023-11-28T13:00:04.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d159a700 1 -- 172.21.15.61:0/2728971942 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4011d20 con 0x7f52d40b1630 2023-11-28T13:00:04.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d8a9b700 1 -- 172.21.15.61:0/2728971942 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 msgr2=0x7f52d40cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d8a9b700 1 --2- 172.21.15.61:0/2728971942 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40cf5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f52c40098f0 tx=0x7f52c4009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:04.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d8a9b700 1 -- 172.21.15.61:0/2728971942 shutdown_connections 2023-11-28T13:00:04.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d8a9b700 1 --2- 172.21.15.61:0/2728971942 >> v2:172.21.15.6:3300/0 conn(0x7f52d406afc0 0x7f52d406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d8a9b700 1 --2- 172.21.15.61:0/2728971942 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40cf5f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d8a9b700 1 -- 172.21.15.61:0/2728971942 >> 172.21.15.61:0/2728971942 conn(0x7f52d40cc630 msgr2=0x7f52d40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:04.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d8a9b700 1 -- 172.21.15.61:0/2728971942 shutdown_connections 2023-11-28T13:00:04.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.349+0000 7f52d8a9b700 1 -- 172.21.15.61:0/2728971942 wait complete. 2023-11-28T13:00:04.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.351+0000 7f52d8a9b700 1 Processor -- start 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.351+0000 7f52d8a9b700 1 -- start start 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d8a9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52d406afc0 0x7f52d40c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d8a9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d8a9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52d40b1a00 0x7f52d41e6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d8a9b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40c3cc0 con 0x7f52d406afc0 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d8a9b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40c3fa0 con 0x7f52d40b1a00 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d8a9b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40c4280 con 0x7f52d40b1630 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d1d9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d2d9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52d40b1a00 0x7f52d41e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d1d9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:59730/0 (socket says 172.21.15.61:59730) 2023-11-28T13:00:04.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d1d9b700 1 -- 172.21.15.61:0/492417073 learned_addr learned my addr 172.21.15.61:0/492417073 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d259c700 1 --2- 172.21.15.61:0/492417073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52d406afc0 0x7f52d40c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d259c700 1 -- 172.21.15.61:0/492417073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 msgr2=0x7f52d40b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d259c700 1 --2- 172.21.15.61:0/492417073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d259c700 1 -- 172.21.15.61:0/492417073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52d40b1a00 msgr2=0x7f52d41e6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d259c700 1 --2- 172.21.15.61:0/492417073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52d40b1a00 0x7f52d41e6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d259c700 1 -- 172.21.15.61:0/492417073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009550 con 0x7f52d406afc0 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d1d9b700 1 --2- 172.21.15.61:0/492417073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d2d9d700 1 --2- 172.21.15.61:0/492417073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52d40b1a00 0x7f52d41e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52d259c700 1 --2- 172.21.15.61:0/492417073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52d406afc0 0x7f52d40c36e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f52c4016000 tx=0x7f52c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52c2ffd700 1 -- 172.21.15.61:0/492417073 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4004400 con 0x7f52d406afc0 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52c2ffd700 1 -- 172.21.15.61:0/492417073 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c401f070 con 0x7f52d406afc0 2023-11-28T13:00:04.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.352+0000 7f52c2ffd700 1 -- 172.21.15.61:0/492417073 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40038e0 con 0x7f52d406afc0 2023-11-28T13:00:04.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.353+0000 7f52d8a9b700 1 -- 172.21.15.61:0/492417073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d40b1e70 con 0x7f52d406afc0 2023-11-28T13:00:04.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.353+0000 7f52d8a9b700 1 -- 172.21.15.61:0/492417073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d40cf0d0 con 0x7f52d406afc0 2023-11-28T13:00:04.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.353+0000 7f52c2ffd700 1 -- 172.21.15.61:0/492417073 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f52c402a020 con 0x7f52d406afc0 2023-11-28T13:00:04.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.353+0000 7f52c2ffd700 1 -- 172.21.15.61:0/492417073 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f52c401a590 con 0x7f52d406afc0 2023-11-28T13:00:04.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.354+0000 7f52d8a9b700 1 -- 172.21.15.61:0/492417073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d40cb5b0 con 0x7f52d406afc0 2023-11-28T13:00:04.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.356+0000 7f52c2ffd700 1 -- 172.21.15.61:0/492417073 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f52c4015020 con 0x7f52d406afc0 2023-11-28T13:00:04.401 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:04.400+0000 7f53b3660200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T13:00:04.418 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:04.417+0000 7f1d19520200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T13:00:04.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.442+0000 7f52d8a9b700 1 -- 172.21.15.61:0/492417073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3", "id": 2} v 0) v1 -- 0x7f52d40739f0 con 0x7f52d406afc0 2023-11-28T13:00:04.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.449+0000 7f52c2ffd700 1 -- 172.21.15.61:0/492417073 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f52c4003a40 con 0x7f52d406afc0 2023-11-28T13:00:04.451 INFO:teuthology.orchestra.run.smithi061.stdout:2 2023-11-28T13:00:04.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.450+0000 7f52d8a9b700 1 -- 172.21.15.61:0/492417073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52d406afc0 msgr2=0x7f52d40c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.450+0000 7f52d8a9b700 1 --2- 172.21.15.61:0/492417073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52d406afc0 0x7f52d40c36e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f52c4016000 tx=0x7f52c4004750 comp rx=0 tx=0).stop 2023-11-28T13:00:04.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.450+0000 7f52d8a9b700 1 -- 172.21.15.61:0/492417073 shutdown_connections 2023-11-28T13:00:04.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.450+0000 7f52d8a9b700 1 --2- 172.21.15.61:0/492417073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52d40b1a00 0x7f52d41e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.450+0000 7f52d8a9b700 1 --2- 172.21.15.61:0/492417073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52d406afc0 0x7f52d40c36e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.450+0000 7f52d8a9b700 1 --2- 172.21.15.61:0/492417073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52d40b1630 0x7f52d40b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.450+0000 7f52d8a9b700 1 -- 172.21.15.61:0/492417073 >> 172.21.15.61:0/492417073 conn(0x7f52d40cc630 msgr2=0x7f52d40cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:04.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.451+0000 7f52d8a9b700 1 -- 172.21.15.61:0/492417073 shutdown_connections 2023-11-28T13:00:04.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.451+0000 7f52d8a9b700 1 -- 172.21.15.61:0/492417073 wait complete. 2023-11-28T13:00:04.464 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 154eaf7a-f54d-4e18-8e19-3a319d24dfc6 3 2023-11-28T13:00:04.480 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:04.479+0000 7f53b3660200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T13:00:04.503 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:04.501+0000 7f1d19520200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T13:00:04.569 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:04.568+0000 7f53b3660200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T13:00:04.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.587+0000 7fe042bf7700 1 Processor -- start 2023-11-28T13:00:04.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.587+0000 7fe042bf7700 1 -- start start 2023-11-28T13:00:04.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.587+0000 7fe042bf7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.587+0000 7fe042bf7700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c0cd2a0 con 0x7fe03c0cb630 2023-11-28T13:00:04.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.587+0000 7fe042bf7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c0d0400 con 0x7fe03c0b62d0 2023-11-28T13:00:04.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.587+0000 7fe042bf7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0d06e0 con 0x7fe03c0b2d90 2023-11-28T13:00:04.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe041194700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe040993700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fe03c0b62d0 0x7fe03c06f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.61:51564/0 (socket says 172.21.15.61:51564) 2023-11-28T13:00:04.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe040993700 1 -- 172.21.15.61:0/3788345005 learned_addr learned my addr 172.21.15.61:0/3788345005 (peer_addr_for_me v1:172.21.15.61:0/0) 2023-11-28T13:00:04.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe03b7fe700 1 -- 172.21.15.61:0/3788345005 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650452753 0 0) 0x7fe03c0d0400 con 0x7fe03c0b62d0 2023-11-28T13:00:04.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe03b7fe700 1 -- 172.21.15.61:0/3788345005 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0280035b0 con 0x7fe03c0b62d0 2023-11-28T13:00:04.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe041194700 1 -- 172.21.15.61:0/3788345005 >> v1:172.21.15.61:6789/0 conn(0x7fe03c0b62d0 legacy=0x7fe03c06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe03b7fe700 1 -- 172.21.15.61:0/3788345005 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756365663 0 0) 0x7fe03c0cd2a0 con 0x7fe03c0cb630 2023-11-28T13:00:04.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe041194700 1 -- 172.21.15.61:0/3788345005 >> v1:172.21.15.6:6789/0 conn(0x7fe03c0cb630 legacy=0x7fe03c0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe041194700 1 -- 172.21.15.61:0/3788345005 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c0cffb0 con 0x7fe03c0b2d90 2023-11-28T13:00:04.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe041194700 1 --2- 172.21.15.61:0/3788345005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c0ce7d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009870 tx=0x7fe02c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63aac6ee332af946 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:04.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe03b7fe700 1 -- 172.21.15.61:0/3788345005 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c011830 con 0x7fe03c0b2d90 2023-11-28T13:00:04.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe03b7fe700 1 -- 172.21.15.61:0/3788345005 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c011990 con 0x7fe03c0b2d90 2023-11-28T13:00:04.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.588+0000 7fe03b7fe700 1 -- 172.21.15.61:0/3788345005 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c011d20 con 0x7fe03c0b2d90 2023-11-28T13:00:04.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.589+0000 7fe042bf7700 1 -- 172.21.15.61:0/3788345005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 msgr2=0x7fe03c0ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.589+0000 7fe042bf7700 1 --2- 172.21.15.61:0/3788345005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c0ce7d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009870 tx=0x7fe02c009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:04.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.589+0000 7fe042bf7700 1 -- 172.21.15.61:0/3788345005 shutdown_connections 2023-11-28T13:00:04.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.589+0000 7fe042bf7700 1 --2- 172.21.15.61:0/3788345005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c0ce7d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.589+0000 7fe042bf7700 1 -- 172.21.15.61:0/3788345005 >> 172.21.15.61:0/3788345005 conn(0x7fe03c0cd680 msgr2=0x7fe03c0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:04.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.589+0000 7fe042bf7700 1 -- 172.21.15.61:0/3788345005 shutdown_connections 2023-11-28T13:00:04.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.589+0000 7fe042bf7700 1 -- 172.21.15.61:0/3788345005 wait complete. 2023-11-28T13:00:04.593 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:04.590+0000 7f1d19520200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T13:00:04.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.590+0000 7fe042bf7700 1 Processor -- start 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.590+0000 7fe042bf7700 1 -- start start 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe042bf7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe042bf7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe03c0b62d0 0x7fe03c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe042bf7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe03c0cb630 0x7fe03c1f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe042bf7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c15ba80 con 0x7fe03c0cb630 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe042bf7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c06bb80 con 0x7fe03c0b62d0 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe042bf7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c15afb0 con 0x7fe03c0b2d90 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe03bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe03c0b62d0 0x7fe03c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe040993700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe03bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe03c0b62d0 0x7fe03c15aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57066/0 (socket says 172.21.15.61:57066) 2023-11-28T13:00:04.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe040993700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c06d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:59740/0 (socket says 172.21.15.61:59740) 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe03bfff700 1 -- 172.21.15.61:0/3913045486 learned_addr learned my addr 172.21.15.61:0/3913045486 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.591+0000 7fe041194700 1 --2- 172.21.15.61:0/3913045486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe03c0cb630 0x7fe03c1f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe03bfff700 1 -- 172.21.15.61:0/3913045486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 msgr2=0x7fe03c06d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe03bfff700 1 --2- 172.21.15.61:0/3913045486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c06d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe03bfff700 1 -- 172.21.15.61:0/3913045486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe03c0cb630 msgr2=0x7fe03c1f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe03bfff700 1 --2- 172.21.15.61:0/3913045486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe03c0cb630 0x7fe03c1f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe03bfff700 1 -- 172.21.15.61:0/3913045486 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c0094a0 con 0x7fe03c0b62d0 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe040993700 1 --2- 172.21.15.61:0/3913045486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c06d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe03bfff700 1 --2- 172.21.15.61:0/3913045486 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe03c0b62d0 0x7fe03c15aa70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe02400c550 tx=0x7fe02400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe0397fa700 1 -- 172.21.15.61:0/3913045486 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0240054e0 con 0x7fe03c0b62d0 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe0397fa700 1 -- 172.21.15.61:0/3913045486 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe024003680 con 0x7fe03c0b62d0 2023-11-28T13:00:04.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe042bf7700 1 -- 172.21.15.61:0/3913045486 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c0cce50 con 0x7fe03c0b62d0 2023-11-28T13:00:04.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe0397fa700 1 -- 172.21.15.61:0/3913045486 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024003a30 con 0x7fe03c0b62d0 2023-11-28T13:00:04.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe042bf7700 1 -- 172.21.15.61:0/3913045486 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c0b66a0 con 0x7fe03c0b62d0 2023-11-28T13:00:04.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe0397fa700 1 -- 172.21.15.61:0/3913045486 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fe024019030 con 0x7fe03c0b62d0 2023-11-28T13:00:04.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.592+0000 7fe0397fa700 1 -- 172.21.15.61:0/3913045486 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe02401b840 con 0x7fe03c0b62d0 2023-11-28T13:00:04.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.593+0000 7fe042bf7700 1 -- 172.21.15.61:0/3913045486 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c0ccfe0 con 0x7fe03c0b62d0 2023-11-28T13:00:04.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.595+0000 7fe0397fa700 1 -- 172.21.15.61:0/3913045486 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fe024017070 con 0x7fe03c0b62d0 2023-11-28T13:00:04.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.681+0000 7fe042bf7700 1 -- 172.21.15.61:0/3913045486 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd new", "uuid": "154eaf7a-f54d-4e18-8e19-3a319d24dfc6", "id": 3} v 0) v1 -- 0x7fe03c0739f0 con 0x7fe03c0b62d0 2023-11-28T13:00:04.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.688+0000 7fe0397fa700 1 -- 172.21.15.61:0/3913045486 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "154eaf7a-f54d-4e18-8e19-3a319d24dfc6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe024003be0 con 0x7fe03c0b62d0 2023-11-28T13:00:04.690 INFO:teuthology.orchestra.run.smithi061.stdout:3 2023-11-28T13:00:04.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.689+0000 7fe042bf7700 1 -- 172.21.15.61:0/3913045486 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe03c0b62d0 msgr2=0x7fe03c15aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.689+0000 7fe042bf7700 1 --2- 172.21.15.61:0/3913045486 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe03c0b62d0 0x7fe03c15aa70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe02400c550 tx=0x7fe02400c860 comp rx=0 tx=0).stop 2023-11-28T13:00:04.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.689+0000 7fe042bf7700 1 -- 172.21.15.61:0/3913045486 shutdown_connections 2023-11-28T13:00:04.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.689+0000 7fe042bf7700 1 --2- 172.21.15.61:0/3913045486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe03c0cb630 0x7fe03c1f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.689+0000 7fe042bf7700 1 --2- 172.21.15.61:0/3913045486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe03c0b2d90 0x7fe03c06d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.689+0000 7fe042bf7700 1 --2- 172.21.15.61:0/3913045486 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe03c0b62d0 0x7fe03c15aa70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.689+0000 7fe042bf7700 1 -- 172.21.15.61:0/3913045486 >> 172.21.15.61:0/3913045486 conn(0x7fe03c0cd680 msgr2=0x7fe03c0cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:04.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.689+0000 7fe042bf7700 1 -- 172.21.15.61:0/3913045486 shutdown_connections 2023-11-28T13:00:04.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.689+0000 7fe042bf7700 1 -- 172.21.15.61:0/3913045486 wait complete. 2023-11-28T13:00:04.703 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 9fb332d4-e6e3-49d9-9172-f1d3884c2d5d 4 2023-11-28T13:00:04.710 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:04.709+0000 7f53b3660200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T13:00:04.736 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:04.736+0000 7f1d19520200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T13:00:04.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.859+0000 7fb5fdb19700 1 Processor -- start 2023-11-28T13:00:04.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.860+0000 7fb5fdb19700 1 -- start start 2023-11-28T13:00:04.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.860+0000 7fb5fdb19700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb5f8159500 0x7fb5f80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.860+0000 7fb5fdb19700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb5f80b1d40 0x7fb5f80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.860+0000 7fb5fdb19700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f815e040 con 0x7fb5f815dbf0 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.860+0000 7fb5fdb19700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f80cede0 con 0x7fb5f8159500 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.860+0000 7fb5fdb19700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f80ceff0 con 0x7fb5f80b1d40 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f7fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb5f80b1d40 0x7fb5f80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f7fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb5f80b1d40 0x7fb5f80ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57074/0 (socket says 172.21.15.61:57074) 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f77fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb5f8159500 0x7fb5f80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f7fff700 1 -- 172.21.15.61:0/2099216457 learned_addr learned my addr 172.21.15.61:0/2099216457 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f6ffd700 1 --1- 172.21.15.61:0/2099216457 >> v1:172.21.15.6:6789/0 conn(0x7fb5f815dbf0 0x7fb5f80b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.61:42438/0 (socket says 172.21.15.61:42438) 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f7fff700 1 -- 172.21.15.61:0/2099216457 >> v2:172.21.15.6:3300/0 conn(0x7fb5f8159500 msgr2=0x7fb5f80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f7fff700 1 --2- 172.21.15.61:0/2099216457 >> v2:172.21.15.6:3300/0 conn(0x7fb5f8159500 0x7fb5f80cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f7fff700 1 -- 172.21.15.61:0/2099216457 >> v1:172.21.15.6:6789/0 conn(0x7fb5f815dbf0 legacy=0x7fb5f80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f7fff700 1 -- 172.21.15.61:0/2099216457 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f80cf2d0 con 0x7fb5f80b1d40 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f77fe700 1 --2- 172.21.15.61:0/2099216457 >> v2:172.21.15.6:3300/0 conn(0x7fb5f8159500 0x7fb5f80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f7fff700 1 --2- 172.21.15.61:0/2099216457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 0x7fb5f80ce8a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80cfb50 tx=0x7fb5f000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab738401abfb19d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f67fc700 1 -- 172.21.15.61:0/2099216457 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000e070 con 0x7fb5f80b1d40 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f67fc700 1 -- 172.21.15.61:0/2099216457 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f000b910 con 0x7fb5f80b1d40 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.861+0000 7fb5f67fc700 1 -- 172.21.15.61:0/2099216457 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000bcc0 con 0x7fb5f80b1d40 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.862+0000 7fb5fdb19700 1 -- 172.21.15.61:0/2099216457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 msgr2=0x7fb5f80ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.862+0000 7fb5fdb19700 1 --2- 172.21.15.61:0/2099216457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 0x7fb5f80ce8a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80cfb50 tx=0x7fb5f000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.862+0000 7fb5fdb19700 1 -- 172.21.15.61:0/2099216457 shutdown_connections 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.862+0000 7fb5fdb19700 1 --2- 172.21.15.61:0/2099216457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 0x7fb5f80ce8a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.862+0000 7fb5fdb19700 1 --2- 172.21.15.61:0/2099216457 >> v2:172.21.15.6:3300/0 conn(0x7fb5f8159500 0x7fb5f80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.862+0000 7fb5fdb19700 1 -- 172.21.15.61:0/2099216457 >> 172.21.15.61:0/2099216457 conn(0x7fb5f80cc630 msgr2=0x7fb5f80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.862+0000 7fb5fdb19700 1 -- 172.21.15.61:0/2099216457 shutdown_connections 2023-11-28T13:00:04.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.862+0000 7fb5fdb19700 1 -- 172.21.15.61:0/2099216457 wait complete. 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.864+0000 7fb5fdb19700 1 Processor -- start 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.864+0000 7fb5fdb19700 1 -- start start 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5fdb19700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 0x7fb5f80b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5fdb19700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5f8159500 0x7fb5f80ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5fdb19700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f815dbf0 0x7fb5f8152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5fdb19700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80b5020 con 0x7fb5f8159500 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5fdb19700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80b4d50 con 0x7fb5f80b1d40 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5fdb19700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80b3520 con 0x7fb5f815dbf0 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f77fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 0x7fb5f80b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f77fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 0x7fb5f80b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57076/0 (socket says 172.21.15.61:57076) 2023-11-28T13:00:04.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f815dbf0 0x7fb5f8152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f77fe700 1 -- 172.21.15.61:0/3849467288 learned_addr learned my addr 172.21.15.61:0/3849467288 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f6ffd700 1 --2- 172.21.15.61:0/3849467288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5f8159500 0x7fb5f80ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f77fe700 1 -- 172.21.15.61:0/3849467288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f815dbf0 msgr2=0x7fb5f8152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f77fe700 1 --2- 172.21.15.61:0/3849467288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f815dbf0 0x7fb5f8152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f77fe700 1 -- 172.21.15.61:0/3849467288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5f8159500 msgr2=0x7fb5f80ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f77fe700 1 --2- 172.21.15.61:0/3849467288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5f8159500 0x7fb5f80ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f77fe700 1 -- 172.21.15.61:0/3849467288 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e800b740 con 0x7fb5f80b1d40 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f6ffd700 1 --2- 172.21.15.61:0/3849467288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5f8159500 0x7fb5f80ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f7fff700 1 --2- 172.21.15.61:0/3849467288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f815dbf0 0x7fb5f8152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.865+0000 7fb5f77fe700 1 --2- 172.21.15.61:0/3849467288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 0x7fb5f80b4810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800ed40 tx=0x7fb5e800d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.866+0000 7fb5dffff700 1 -- 172.21.15.61:0/3849467288 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800dd90 con 0x7fb5f80b1d40 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.866+0000 7fb5dffff700 1 -- 172.21.15.61:0/3849467288 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e8010440 con 0x7fb5f80b1d40 2023-11-28T13:00:04.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.866+0000 7fb5fdb19700 1 -- 172.21.15.61:0/3849467288 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f0009b30 con 0x7fb5f80b1d40 2023-11-28T13:00:04.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.866+0000 7fb5dffff700 1 -- 172.21.15.61:0/3849467288 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80107b0 con 0x7fb5f80b1d40 2023-11-28T13:00:04.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.866+0000 7fb5fdb19700 1 -- 172.21.15.61:0/3849467288 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f815d3f0 con 0x7fb5f80b1d40 2023-11-28T13:00:04.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.866+0000 7fb5dffff700 1 -- 172.21.15.61:0/3849467288 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fb5e8005d10 con 0x7fb5f80b1d40 2023-11-28T13:00:04.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.866+0000 7fb5dffff700 1 -- 172.21.15.61:0/3849467288 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fb5e80067f0 con 0x7fb5f80b1d40 2023-11-28T13:00:04.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.867+0000 7fb5fdb19700 1 -- 172.21.15.61:0/3849467288 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f815d110 con 0x7fb5f80b1d40 2023-11-28T13:00:04.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.869+0000 7fb5dffff700 1 -- 172.21.15.61:0/3849467288 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb5e8036b00 con 0x7fb5f80b1d40 2023-11-28T13:00:04.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.954+0000 7fb5fdb19700 1 -- 172.21.15.61:0/3849467288 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9fb332d4-e6e3-49d9-9172-f1d3884c2d5d", "id": 4} v 0) v1 -- 0x7fb5f815d6d0 con 0x7fb5f80b1d40 2023-11-28T13:00:04.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.962+0000 7fb5dffff700 1 -- 172.21.15.61:0/3849467288 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9fb332d4-e6e3-49d9-9172-f1d3884c2d5d", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb5e8025070 con 0x7fb5f80b1d40 2023-11-28T13:00:04.962 INFO:teuthology.orchestra.run.smithi061.stdout:4 2023-11-28T13:00:04.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.963+0000 7fb5fdb19700 1 -- 172.21.15.61:0/3849467288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 msgr2=0x7fb5f80b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:04.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.963+0000 7fb5fdb19700 1 --2- 172.21.15.61:0/3849467288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 0x7fb5f80b4810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800ed40 tx=0x7fb5e800d630 comp rx=0 tx=0).stop 2023-11-28T13:00:04.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.963+0000 7fb5fdb19700 1 -- 172.21.15.61:0/3849467288 shutdown_connections 2023-11-28T13:00:04.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.963+0000 7fb5fdb19700 1 --2- 172.21.15.61:0/3849467288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5f815dbf0 0x7fb5f8152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.963+0000 7fb5fdb19700 1 --2- 172.21.15.61:0/3849467288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5f80b1d40 0x7fb5f80b4810 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.963+0000 7fb5fdb19700 1 --2- 172.21.15.61:0/3849467288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5f8159500 0x7fb5f80ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:04.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.963+0000 7fb5fdb19700 1 -- 172.21.15.61:0/3849467288 >> 172.21.15.61:0/3849467288 conn(0x7fb5f80cc630 msgr2=0x7fb5f8157050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:04.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.963+0000 7fb5fdb19700 1 -- 172.21.15.61:0/3849467288 shutdown_connections 2023-11-28T13:00:04.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:04.963+0000 7fb5fdb19700 1 -- 172.21.15.61:0/3849467288 wait complete. 2023-11-28T13:00:04.975 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new bc0d4e9c-c75d-4959-b71c-66a04ca98515 5 2023-11-28T13:00:05.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.101+0000 7f64602bd700 1 Processor -- start 2023-11-28T13:00:05.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.101+0000 7f64602bd700 1 -- start start 2023-11-28T13:00:05.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f64602bd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64580b1d40 0x7f64580cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f64602bd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f645815dbf0 0x7f64580ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f64602bd700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64580ceff0 con 0x7f6458159500 2023-11-28T13:00:05.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f64602bd700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645815e090 con 0x7f645815dbf0 2023-11-28T13:00:05.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f64602bd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64580cede0 con 0x7f64580b1d40 2023-11-28T13:00:05.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f645d858700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64580b1d40 0x7f64580cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f645e85a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f645815dbf0 0x7f64580ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f645d858700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64580b1d40 0x7f64580cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:59760/0 (socket says 172.21.15.61:59760) 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f645d858700 1 -- 172.21.15.61:0/3412999119 learned_addr learned my addr 172.21.15.61:0/3412999119 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f645e85a700 1 -- 172.21.15.61:0/3412999119 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64580b1d40 msgr2=0x7f64580cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f645e85a700 1 --2- 172.21.15.61:0/3412999119 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64580b1d40 0x7f64580cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f645e85a700 1 -- 172.21.15.61:0/3412999119 >> v1:172.21.15.6:6789/0 conn(0x7f6458159500 legacy=0x7f64580b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.102+0000 7f645e85a700 1 -- 172.21.15.61:0/3412999119 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580cf2d0 con 0x7f645815dbf0 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.103+0000 7f645e85a700 1 --2- 172.21.15.61:0/3412999119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f645815dbf0 0x7f64580ce8a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f645000bfa0 tx=0x7f645000d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=504d7d2c5d754d55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.103+0000 7f645d057700 1 -- 172.21.15.61:0/3412999119 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450010070 con 0x7f645815dbf0 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.103+0000 7f645d057700 1 -- 172.21.15.61:0/3412999119 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645000b2e0 con 0x7f645815dbf0 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.103+0000 7f645d057700 1 -- 172.21.15.61:0/3412999119 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645000dc40 con 0x7f645815dbf0 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.103+0000 7f64602bd700 1 -- 172.21.15.61:0/3412999119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f645815dbf0 msgr2=0x7f64580ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.103+0000 7f64602bd700 1 --2- 172.21.15.61:0/3412999119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f645815dbf0 0x7f64580ce8a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f645000bfa0 tx=0x7f645000d330 comp rx=0 tx=0).stop 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.103+0000 7f64602bd700 1 -- 172.21.15.61:0/3412999119 shutdown_connections 2023-11-28T13:00:05.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.103+0000 7f64602bd700 1 --2- 172.21.15.61:0/3412999119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f645815dbf0 0x7f64580ce8a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.103+0000 7f64602bd700 1 --2- 172.21.15.61:0/3412999119 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f64580b1d40 0x7f64580cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.104+0000 7f64602bd700 1 -- 172.21.15.61:0/3412999119 >> 172.21.15.61:0/3412999119 conn(0x7f64580cc630 msgr2=0x7f64580cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:05.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.104+0000 7f64602bd700 1 -- 172.21.15.61:0/3412999119 shutdown_connections 2023-11-28T13:00:05.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.104+0000 7f64602bd700 1 -- 172.21.15.61:0/3412999119 wait complete. 2023-11-28T13:00:05.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.105+0000 7f64602bd700 1 Processor -- start 2023-11-28T13:00:05.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.105+0000 7f64602bd700 1 -- start start 2023-11-28T13:00:05.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f64602bd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64580b1d40 0x7f6458154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f64602bd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6458159500 0x7f64580c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f64602bd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645815dbf0 0x7f6458143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f64602bd700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64580c8810 con 0x7f645815dbf0 2023-11-28T13:00:05.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f64602bd700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6458146c40 con 0x7f64580b1d40 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f64602bd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64580c7e70 con 0x7f6458159500 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645d858700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6458159500 0x7f64580c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645e059700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64580b1d40 0x7f6458154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645d858700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6458159500 0x7f64580c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:59766/0 (socket says 172.21.15.61:59766) 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645d858700 1 -- 172.21.15.61:0/3022167331 learned_addr learned my addr 172.21.15.61:0/3022167331 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645e85a700 1 --2- 172.21.15.61:0/3022167331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645815dbf0 0x7f6458143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645e059700 1 -- 172.21.15.61:0/3022167331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6458159500 msgr2=0x7f64580c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645e059700 1 --2- 172.21.15.61:0/3022167331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6458159500 0x7f64580c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645e059700 1 -- 172.21.15.61:0/3022167331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645815dbf0 msgr2=0x7f6458143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645e059700 1 --2- 172.21.15.61:0/3022167331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645815dbf0 0x7f6458143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645e059700 1 -- 172.21.15.61:0/3022167331 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645000bc80 con 0x7f64580b1d40 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645d858700 1 --2- 172.21.15.61:0/3022167331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6458159500 0x7f64580c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:05.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f645e059700 1 --2- 172.21.15.61:0/3022167331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64580b1d40 0x7f6458154cf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6454000f60 tx=0x7f645400c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:05.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f644effd700 1 -- 172.21.15.61:0/3022167331 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64540052d0 con 0x7f64580b1d40 2023-11-28T13:00:05.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f644effd700 1 -- 172.21.15.61:0/3022167331 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6454005430 con 0x7f64580b1d40 2023-11-28T13:00:05.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f64602bd700 1 -- 172.21.15.61:0/3022167331 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645815d3f0 con 0x7f64580b1d40 2023-11-28T13:00:05.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f644effd700 1 -- 172.21.15.61:0/3022167331 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64540038c0 con 0x7f64580b1d40 2023-11-28T13:00:05.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.106+0000 7f64602bd700 1 -- 172.21.15.61:0/3022167331 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645815d110 con 0x7f64580b1d40 2023-11-28T13:00:05.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.107+0000 7f644effd700 1 -- 172.21.15.61:0/3022167331 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f6454019030 con 0x7f64580b1d40 2023-11-28T13:00:05.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.107+0000 7f644effd700 1 -- 172.21.15.61:0/3022167331 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f6454011ca0 con 0x7f64580b1d40 2023-11-28T13:00:05.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.107+0000 7f64602bd700 1 -- 172.21.15.61:0/3022167331 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645815d6d0 con 0x7f64580b1d40 2023-11-28T13:00:05.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.109+0000 7f644effd700 1 -- 172.21.15.61:0/3022167331 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f6454017070 con 0x7f64580b1d40 2023-11-28T13:00:05.117 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:05.117+0000 7f53b3660200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T13:00:05.158 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:05.157+0000 7f1d19520200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T13:00:05.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.195+0000 7f64602bd700 1 -- 172.21.15.61:0/3022167331 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bc0d4e9c-c75d-4959-b71c-66a04ca98515", "id": 5} v 0) v1 -- 0x7f64580739f0 con 0x7f64580b1d40 2023-11-28T13:00:05.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.204+0000 7f644effd700 1 -- 172.21.15.61:0/3022167331 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bc0d4e9c-c75d-4959-b71c-66a04ca98515", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f645400ce30 con 0x7f64580b1d40 2023-11-28T13:00:05.205 INFO:teuthology.orchestra.run.smithi061.stdout:5 2023-11-28T13:00:05.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.205+0000 7f64602bd700 1 -- 172.21.15.61:0/3022167331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64580b1d40 msgr2=0x7f6458154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.205+0000 7f64602bd700 1 --2- 172.21.15.61:0/3022167331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64580b1d40 0x7f6458154cf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6454000f60 tx=0x7f645400c730 comp rx=0 tx=0).stop 2023-11-28T13:00:05.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.205+0000 7f64602bd700 1 -- 172.21.15.61:0/3022167331 shutdown_connections 2023-11-28T13:00:05.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.205+0000 7f64602bd700 1 --2- 172.21.15.61:0/3022167331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f645815dbf0 0x7f6458143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.205+0000 7f64602bd700 1 --2- 172.21.15.61:0/3022167331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64580b1d40 0x7f6458154cf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.205+0000 7f64602bd700 1 --2- 172.21.15.61:0/3022167331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6458159500 0x7f64580c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.205+0000 7f64602bd700 1 -- 172.21.15.61:0/3022167331 >> 172.21.15.61:0/3022167331 conn(0x7f64580cc630 msgr2=0x7f64580b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:05.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.205+0000 7f64602bd700 1 -- 172.21.15.61:0/3022167331 shutdown_connections 2023-11-28T13:00:05.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.206+0000 7f64602bd700 1 -- 172.21.15.61:0/3022167331 wait complete. 2023-11-28T13:00:05.221 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 264f6e4c-db13-4cb0-a749-d507ab7c900d 6 2023-11-28T13:00:05.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.343+0000 7f8c93e4a700 1 Processor -- start 2023-11-28T13:00:05.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.344+0000 7f8c93e4a700 1 -- start start 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.344+0000 7f8c93e4a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8c8c0b1630 0x7f8c8c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c93e4a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c06afc0 0x7f8c8c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c93e4a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8c0ce550 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c913e5700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f8c8c0b1a00 0x7f8c8c0caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.61:51602/0 (socket says 172.21.15.61:51602) 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c93e4a700 1 -- 172.21.15.61:0/4289288968 --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8c0ce830 con 0x7f8c8c0b1630 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c913e5700 1 -- 172.21.15.61:0/4289288968 learned_addr learned my addr 172.21.15.61:0/4289288968 (peer_addr_for_me v1:172.21.15.61:0/0) 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c93e4a700 1 -- 172.21.15.61:0/4289288968 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0ceb10 con 0x7f8c8c06afc0 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c91be6700 1 --2- 172.21.15.61:0/4289288968 >> v2:172.21.15.6:3300/0 conn(0x7f8c8c0b1630 0x7f8c8c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c923e7700 1 --2- 172.21.15.61:0/4289288968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c06afc0 0x7f8c8c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c90be4700 1 -- 172.21.15.61:0/4289288968 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545170593 0 0) 0x7f8c8c0ce550 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c90be4700 1 -- 172.21.15.61:0/4289288968 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c780035b0 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c91be6700 1 -- 172.21.15.61:0/4289288968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c06afc0 msgr2=0x7f8c8c06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c91be6700 1 --2- 172.21.15.61:0/4289288968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c06afc0 0x7f8c8c06b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c91be6700 1 -- 172.21.15.61:0/4289288968 >> v1:172.21.15.61:6789/0 conn(0x7f8c8c0b1a00 legacy=0x7f8c8c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c91be6700 1 -- 172.21.15.61:0/4289288968 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c0cedf0 con 0x7f8c8c0b1630 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c923e7700 1 --2- 172.21.15.61:0/4289288968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c06afc0 0x7f8c8c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c91be6700 1 --2- 172.21.15.61:0/4289288968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c8c0b1630 0x7f8c8c0cf5f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009870 tx=0x7f8c88009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da99f77dfa2a7524 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c90be4700 1 -- 172.21.15.61:0/4289288968 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88011830 con 0x7f8c8c0b1630 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c90be4700 1 -- 172.21.15.61:0/4289288968 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c88011990 con 0x7f8c8c0b1630 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.345+0000 7f8c90be4700 1 -- 172.21.15.61:0/4289288968 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88011d20 con 0x7f8c8c0b1630 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.346+0000 7f8c93e4a700 1 -- 172.21.15.61:0/4289288968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c8c0b1630 msgr2=0x7f8c8c0cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.346+0000 7f8c93e4a700 1 --2- 172.21.15.61:0/4289288968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c8c0b1630 0x7f8c8c0cf5f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009870 tx=0x7f8c88009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:05.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.346+0000 7f8c93e4a700 1 -- 172.21.15.61:0/4289288968 shutdown_connections 2023-11-28T13:00:05.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.346+0000 7f8c93e4a700 1 --2- 172.21.15.61:0/4289288968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c06afc0 0x7f8c8c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.346+0000 7f8c93e4a700 1 --2- 172.21.15.61:0/4289288968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c8c0b1630 0x7f8c8c0cf5f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.346+0000 7f8c93e4a700 1 -- 172.21.15.61:0/4289288968 >> 172.21.15.61:0/4289288968 conn(0x7f8c8c0cc630 msgr2=0x7f8c8c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:05.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.346+0000 7f8c93e4a700 1 -- 172.21.15.61:0/4289288968 shutdown_connections 2023-11-28T13:00:05.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.346+0000 7f8c93e4a700 1 -- 172.21.15.61:0/4289288968 wait complete. 2023-11-28T13:00:05.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.347+0000 7f8c93e4a700 1 Processor -- start 2023-11-28T13:00:05.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.347+0000 7f8c93e4a700 1 -- start start 2023-11-28T13:00:05.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c93e4a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c8c06afc0 0x7f8c8c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c93e4a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c8c0b1630 0x7f8c8c149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c93e4a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c0b1a00 0x7f8c8c14a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c93e4a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0c9fd0 con 0x7f8c8c06afc0 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c93e4a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0ac060 con 0x7f8c8c0b1630 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c93e4a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c15aa30 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c923e7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c0b1a00 0x7f8c8c14a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c913e5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c8c0b1630 0x7f8c8c149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c91be6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c8c06afc0 0x7f8c8c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c923e7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c0b1a00 0x7f8c8c14a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:59780/0 (socket says 172.21.15.61:59780) 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.348+0000 7f8c923e7700 1 -- 172.21.15.61:0/685394604 learned_addr learned my addr 172.21.15.61:0/685394604 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c923e7700 1 -- 172.21.15.61:0/685394604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c8c0b1630 msgr2=0x7f8c8c149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c923e7700 1 --2- 172.21.15.61:0/685394604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c8c0b1630 0x7f8c8c149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c923e7700 1 -- 172.21.15.61:0/685394604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c8c06afc0 msgr2=0x7f8c8c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c923e7700 1 --2- 172.21.15.61:0/685394604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c8c06afc0 0x7f8c8c157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c923e7700 1 -- 172.21.15.61:0/685394604 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c84009a70 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c923e7700 1 --2- 172.21.15.61:0/685394604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c0b1a00 0x7f8c8c14a300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400efc0 tx=0x7f8c8400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c827fc700 1 -- 172.21.15.61:0/685394604 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8400ca90 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c827fc700 1 -- 172.21.15.61:0/685394604 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8400f040 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c93e4a700 1 -- 172.21.15.61:0/685394604 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c880094a0 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c827fc700 1 -- 172.21.15.61:0/685394604 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8400ce10 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c93e4a700 1 -- 172.21.15.61:0/685394604 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c0b1ea0 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c827fc700 1 -- 172.21.15.61:0/685394604 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8c84003a20 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.349+0000 7f8c827fc700 1 -- 172.21.15.61:0/685394604 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f8c84004750 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.350+0000 7f8c93e4a700 1 -- 172.21.15.61:0/685394604 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8c0cf0d0 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.352+0000 7f8c827fc700 1 -- 172.21.15.61:0/685394604 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f8c84018030 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.438+0000 7f8c93e4a700 1 -- 172.21.15.61:0/685394604 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd new", "uuid": "264f6e4c-db13-4cb0-a749-d507ab7c900d", "id": 6} v 0) v1 -- 0x7f8c8c0cb5b0 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.445+0000 7f8c827fc700 1 -- 172.21.15.61:0/685394604 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "264f6e4c-db13-4cb0-a749-d507ab7c900d", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8c84004020 con 0x7f8c8c0b1a00 2023-11-28T13:00:05.446 INFO:teuthology.orchestra.run.smithi061.stdout:6 2023-11-28T13:00:05.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.446+0000 7f8c93e4a700 1 -- 172.21.15.61:0/685394604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c0b1a00 msgr2=0x7f8c8c14a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.446+0000 7f8c93e4a700 1 --2- 172.21.15.61:0/685394604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c0b1a00 0x7f8c8c14a300 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400efc0 tx=0x7f8c8400c3a0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.446+0000 7f8c93e4a700 1 -- 172.21.15.61:0/685394604 shutdown_connections 2023-11-28T13:00:05.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.446+0000 7f8c93e4a700 1 --2- 172.21.15.61:0/685394604 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c8c0b1a00 0x7f8c8c14a300 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.446+0000 7f8c93e4a700 1 --2- 172.21.15.61:0/685394604 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c8c06afc0 0x7f8c8c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.446+0000 7f8c93e4a700 1 --2- 172.21.15.61:0/685394604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c8c0b1630 0x7f8c8c149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.446+0000 7f8c93e4a700 1 -- 172.21.15.61:0/685394604 >> 172.21.15.61:0/685394604 conn(0x7f8c8c0cc630 msgr2=0x7f8c8c0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:05.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.447+0000 7f8c93e4a700 1 -- 172.21.15.61:0/685394604 shutdown_connections 2023-11-28T13:00:05.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.447+0000 7f8c93e4a700 1 -- 172.21.15.61:0/685394604 wait complete. 2023-11-28T13:00:05.459 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd new 5030167b-9ac0-434e-b096-61c9eaac0940 7 2023-11-28T13:00:05.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.597+0000 7f2e8dea5700 1 Processor -- start 2023-11-28T13:00:05.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.597+0000 7f2e8dea5700 1 -- start start 2023-11-28T13:00:05.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e8dea5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b62d0 0x7f2e880cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e8dea5700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2e8806f8b0 0x7f2e880b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e8dea5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2e880b2bc0 0x7f2e880ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e8dea5700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e880b3020 con 0x7f2e8806f8b0 2023-11-28T13:00:05.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e8dea5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e880cd2a0 con 0x7f2e880b2bc0 2023-11-28T13:00:05.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e8dea5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880d0400 con 0x7f2e880b62d0 2023-11-28T13:00:05.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e86ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2e8806f8b0 0x7f2e880b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e86ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2e8806f8b0 0x7f2e880b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.61:49670/0 (socket says 172.21.15.61:49670) 2023-11-28T13:00:05.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e86ffd700 1 -- 172.21.15.61:0/470538559 learned_addr learned my addr 172.21.15.61:0/470538559 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:05.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e86ffd700 1 -- 172.21.15.61:0/470538559 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b62d0 msgr2=0x7f2e880cb630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:00:05.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e86ffd700 1 --2- 172.21.15.61:0/470538559 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b62d0 0x7f2e880cb630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e86ffd700 1 -- 172.21.15.61:0/470538559 >> v2:172.21.15.61:3300/0 conn(0x7f2e880b2bc0 msgr2=0x7f2e880ce7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:00:05.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e86ffd700 1 --2- 172.21.15.61:0/470538559 >> v2:172.21.15.61:3300/0 conn(0x7f2e880b2bc0 0x7f2e880ce7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.599+0000 7f2e86ffd700 1 -- 172.21.15.61:0/470538559 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e880d06e0 con 0x7f2e8806f8b0 2023-11-28T13:00:05.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.600+0000 7f2e86ffd700 1 --2- 172.21.15.61:0/470538559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e8806f8b0 0x7f2e880b2680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e880d1cc0 tx=0x7f2e7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e212ba616ecd3d11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:05.602 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.600+0000 7f2e867fc700 1 -- 172.21.15.61:0/470538559 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7800e070 con 0x7f2e8806f8b0 2023-11-28T13:00:05.602 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.600+0000 7f2e867fc700 1 -- 172.21.15.61:0/470538559 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7800b9c0 con 0x7f2e8806f8b0 2023-11-28T13:00:05.602 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.601+0000 7f2e8dea5700 1 -- 172.21.15.61:0/470538559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e8806f8b0 msgr2=0x7f2e880b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.601+0000 7f2e8dea5700 1 --2- 172.21.15.61:0/470538559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e8806f8b0 0x7f2e880b2680 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e880d1cc0 tx=0x7f2e7800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:05.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.601+0000 7f2e8dea5700 1 -- 172.21.15.61:0/470538559 shutdown_connections 2023-11-28T13:00:05.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.601+0000 7f2e8dea5700 1 --2- 172.21.15.61:0/470538559 >> v2:172.21.15.61:3300/0 conn(0x7f2e880b2bc0 0x7f2e880ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.601+0000 7f2e8dea5700 1 --2- 172.21.15.61:0/470538559 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b62d0 0x7f2e880cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.601+0000 7f2e8dea5700 1 --2- 172.21.15.61:0/470538559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e8806f8b0 0x7f2e880b2680 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.601+0000 7f2e8dea5700 1 -- 172.21.15.61:0/470538559 >> 172.21.15.61:0/470538559 conn(0x7f2e880cd680 msgr2=0x7f2e880cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:05.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.601+0000 7f2e8dea5700 1 -- 172.21.15.61:0/470538559 shutdown_connections 2023-11-28T13:00:05.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.601+0000 7f2e8dea5700 1 -- 172.21.15.61:0/470538559 wait complete. 2023-11-28T13:00:05.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.603+0000 7f2e8dea5700 1 Processor -- start 2023-11-28T13:00:05.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.604+0000 7f2e8dea5700 1 -- start start 2023-11-28T13:00:05.605 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.605+0000 7f2e8dea5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e8806f8b0 0x7f2e881e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.605+0000 7f2e8dea5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b2bc0 0x7f2e881e4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.605+0000 7f2e8dea5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e880b62d0 0x7f2e88158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.605+0000 7f2e8dea5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880cab60 con 0x7f2e880b62d0 2023-11-28T13:00:05.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.605+0000 7f2e8dea5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e881e3180 con 0x7f2e8806f8b0 2023-11-28T13:00:05.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.605+0000 7f2e8dea5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e881e33b0 con 0x7f2e880b2bc0 2023-11-28T13:00:05.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.606+0000 7f2e86ffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b2bc0 0x7f2e881e4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.606+0000 7f2e86ffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b2bc0 0x7f2e881e4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.61:59786/0 (socket says 172.21.15.61:59786) 2023-11-28T13:00:05.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.606+0000 7f2e86ffd700 1 -- 172.21.15.61:0/1552150359 learned_addr learned my addr 172.21.15.61:0/1552150359 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-11-28T13:00:05.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.606+0000 7f2e877fe700 1 --2- 172.21.15.61:0/1552150359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e8806f8b0 0x7f2e881e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.606+0000 7f2e87fff700 1 --2- 172.21.15.61:0/1552150359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e880b62d0 0x7f2e88158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.606+0000 7f2e86ffd700 1 -- 172.21.15.61:0/1552150359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e8806f8b0 msgr2=0x7f2e881e2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.606+0000 7f2e86ffd700 1 --2- 172.21.15.61:0/1552150359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e8806f8b0 0x7f2e881e2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.606+0000 7f2e86ffd700 1 -- 172.21.15.61:0/1552150359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e880b62d0 msgr2=0x7f2e88158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.606+0000 7f2e86ffd700 1 --2- 172.21.15.61:0/1552150359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e880b62d0 0x7f2e88158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.606+0000 7f2e86ffd700 1 -- 172.21.15.61:0/1552150359 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e78009b30 con 0x7f2e880b2bc0 2023-11-28T13:00:05.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.607+0000 7f2e86ffd700 1 --2- 172.21.15.61:0/1552150359 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b2bc0 0x7f2e881e4c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2e78000f80 tx=0x7f2e78004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:05.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.607+0000 7f2e7ffff700 1 -- 172.21.15.61:0/1552150359 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7801c850 con 0x7f2e880b2bc0 2023-11-28T13:00:05.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.607+0000 7f2e7ffff700 1 -- 172.21.15.61:0/1552150359 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7801c9d0 con 0x7f2e880b2bc0 2023-11-28T13:00:05.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.607+0000 7f2e8dea5700 1 -- 172.21.15.61:0/1552150359 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e880b66a0 con 0x7f2e880b2bc0 2023-11-28T13:00:05.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.607+0000 7f2e7ffff700 1 -- 172.21.15.61:0/1552150359 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7801cd80 con 0x7f2e880b2bc0 2023-11-28T13:00:05.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.607+0000 7f2e8dea5700 1 -- 172.21.15.61:0/1552150359 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e880cffb0 con 0x7f2e880b2bc0 2023-11-28T13:00:05.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.607+0000 7f2e7ffff700 1 -- 172.21.15.61:0/1552150359 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f2e78024030 con 0x7f2e880b2bc0 2023-11-28T13:00:05.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.608+0000 7f2e7ffff700 1 -- 172.21.15.61:0/1552150359 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f2e7800bb30 con 0x7f2e880b2bc0 2023-11-28T13:00:05.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.609+0000 7f2e8dea5700 1 -- 172.21.15.61:0/1552150359 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e8806fd20 con 0x7f2e880b2bc0 2023-11-28T13:00:05.614 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.614+0000 7f2e7ffff700 1 -- 172.21.15.61:0/1552150359 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f2e78022070 con 0x7f2e880b2bc0 2023-11-28T13:00:05.711 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:05.709+0000 7f53b3660200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T13:00:05.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.722+0000 7f2e8dea5700 1 -- 172.21.15.61:0/1552150359 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5030167b-9ac0-434e-b096-61c9eaac0940", "id": 7} v 0) v1 -- 0x7f2e880739f0 con 0x7f2e880b2bc0 2023-11-28T13:00:05.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.730+0000 7f2e7ffff700 1 -- 172.21.15.61:0/1552150359 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5030167b-9ac0-434e-b096-61c9eaac0940", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2e78027450 con 0x7f2e880b2bc0 2023-11-28T13:00:05.730 INFO:teuthology.orchestra.run.smithi061.stdout:7 2023-11-28T13:00:05.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.731+0000 7f2e8dea5700 1 -- 172.21.15.61:0/1552150359 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b2bc0 msgr2=0x7f2e881e4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.731+0000 7f2e8dea5700 1 --2- 172.21.15.61:0/1552150359 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b2bc0 0x7f2e881e4c10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2e78000f80 tx=0x7f2e78004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.731+0000 7f2e8dea5700 1 -- 172.21.15.61:0/1552150359 shutdown_connections 2023-11-28T13:00:05.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.731+0000 7f2e8dea5700 1 --2- 172.21.15.61:0/1552150359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e880b62d0 0x7f2e88158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.731+0000 7f2e8dea5700 1 --2- 172.21.15.61:0/1552150359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e8806f8b0 0x7f2e881e2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.731+0000 7f2e8dea5700 1 --2- 172.21.15.61:0/1552150359 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2e880b2bc0 0x7f2e881e4c10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.731+0000 7f2e8dea5700 1 -- 172.21.15.61:0/1552150359 >> 172.21.15.61:0/1552150359 conn(0x7f2e880cd680 msgr2=0x7f2e8815ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:05.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.731+0000 7f2e8dea5700 1 -- 172.21.15.61:0/1552150359 shutdown_connections 2023-11-28T13:00:05.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-11-28T13:00:05.731+0000 7f2e8dea5700 1 -- 172.21.15.61:0/1552150359 wait complete. 2023-11-28T13:00:05.743 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T13:00:05.743 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T13:00:05.745 INFO:tasks.ceph.osd.0:Started 2023-11-28T13:00:05.746 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T13:00:05.746 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T13:00:05.748 INFO:tasks.ceph.osd.1:Started 2023-11-28T13:00:05.748 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T13:00:05.748 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T13:00:05.750 INFO:tasks.ceph.osd.2:Started 2023-11-28T13:00:05.750 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T13:00:05.750 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T13:00:05.752 INFO:tasks.ceph.osd.3:Started 2023-11-28T13:00:05.752 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T13:00:05.752 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T13:00:05.754 INFO:tasks.ceph.osd.4:Started 2023-11-28T13:00:05.754 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T13:00:05.754 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T13:00:05.756 INFO:tasks.ceph.osd.5:Started 2023-11-28T13:00:05.756 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T13:00:05.756 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T13:00:05.758 INFO:tasks.ceph.osd.6:Started 2023-11-28T13:00:05.758 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T13:00:05.758 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T13:00:05.761 INFO:tasks.ceph.osd.7:Started 2023-11-28T13:00:05.762 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:05.771 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:05.771+0000 7f1d19520200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T13:00:05.802 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:05.800+0000 7f53b3660200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T13:00:05.861 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:05.861+0000 7f1d19520200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T13:00:05.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.928+0000 7f603bdaf700 1 Processor -- start 2023-11-28T13:00:05.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.928+0000 7f603bdaf700 1 -- start start 2023-11-28T13:00:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.929+0000 7f603bdaf700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f603415c990 0x7f60340cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.929+0000 7f603bdaf700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f60340cc310 0x7f60340cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.929+0000 7f603bdaf700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60340ce340 con 0x7f6034161090 2023-11-28T13:00:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.929+0000 7f603bdaf700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60340cebe0 con 0x7f603415c990 2023-11-28T13:00:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.929+0000 7f603bdaf700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60340cccb0 con 0x7f60340cc310 2023-11-28T13:00:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.929+0000 7f6039b4b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f603415c990 0x7f60340cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.929+0000 7f603934a700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6034161090 0x7f60340ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44242/0 (socket says 172.21.15.6:44242) 2023-11-28T13:00:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.929+0000 7f603934a700 1 -- 172.21.15.6:0/292867936 learned_addr learned my addr 172.21.15.6:0/292867936 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:05.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.929+0000 7f603a34c700 1 --2- 172.21.15.6:0/292867936 >> v2:172.21.15.61:3300/0 conn(0x7f60340cc310 0x7f60340cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.930+0000 7f6039b4b700 1 -- 172.21.15.6:0/292867936 >> v2:172.21.15.61:3300/0 conn(0x7f60340cc310 msgr2=0x7f60340cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.930+0000 7f6039b4b700 1 --2- 172.21.15.6:0/292867936 >> v2:172.21.15.61:3300/0 conn(0x7f60340cc310 0x7f60340cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.930+0000 7f6039b4b700 1 -- 172.21.15.6:0/292867936 >> v1:172.21.15.6:6789/0 conn(0x7f6034161090 legacy=0x7f60340ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.930+0000 7f6039b4b700 1 -- 172.21.15.6:0/292867936 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60340cb3f0 con 0x7f603415c990 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.930+0000 7f6039b4b700 1 --2- 172.21.15.6:0/292867936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f603415c990 0x7f60340cf070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6034071a50 tx=0x7f6030009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f70e19d1a49a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.930+0000 7f6038b49700 1 -- 172.21.15.6:0/292867936 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030011830 con 0x7f603415c990 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.930+0000 7f6038b49700 1 -- 172.21.15.6:0/292867936 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6030011990 con 0x7f603415c990 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.930+0000 7f6038b49700 1 -- 172.21.15.6:0/292867936 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030011d20 con 0x7f603415c990 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.931+0000 7f603bdaf700 1 -- 172.21.15.6:0/292867936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f603415c990 msgr2=0x7f60340cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.931+0000 7f603bdaf700 1 --2- 172.21.15.6:0/292867936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f603415c990 0x7f60340cf070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6034071a50 tx=0x7f6030009b20 comp rx=0 tx=0).stop 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.931+0000 7f603bdaf700 1 -- 172.21.15.6:0/292867936 shutdown_connections 2023-11-28T13:00:05.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.931+0000 7f603bdaf700 1 --2- 172.21.15.6:0/292867936 >> v2:172.21.15.61:3300/0 conn(0x7f60340cc310 0x7f60340cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.931+0000 7f603bdaf700 1 --2- 172.21.15.6:0/292867936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f603415c990 0x7f60340cf070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.931+0000 7f603bdaf700 1 -- 172.21.15.6:0/292867936 >> 172.21.15.6:0/292867936 conn(0x7f60340b1650 msgr2=0x7f60340b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:05.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.931+0000 7f603bdaf700 1 -- 172.21.15.6:0/292867936 shutdown_connections 2023-11-28T13:00:05.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.931+0000 7f603bdaf700 1 -- 172.21.15.6:0/292867936 wait complete. 2023-11-28T13:00:05.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.932+0000 7f603bdaf700 1 Processor -- start 2023-11-28T13:00:05.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.932+0000 7f603bdaf700 1 -- start start 2023-11-28T13:00:05.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f603bdaf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60340cc310 0x7f6034156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f603bdaf700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f603415c990 0x7f6034147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f603bdaf700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6034161090 0x7f6034147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:05.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f603bdaf700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340c3a70 con 0x7f60340cc310 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f603bdaf700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6034145740 con 0x7f603415c990 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f603bdaf700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60340b3a40 con 0x7f6034161090 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f6039b4b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60340cc310 0x7f6034156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f6039b4b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60340cc310 0x7f6034156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56928/0 (socket says 172.21.15.6:56928) 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f6039b4b700 1 -- 172.21.15.6:0/2832542065 learned_addr learned my addr 172.21.15.6:0/2832542065 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f603934a700 1 --2- 172.21.15.6:0/2832542065 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f603415c990 0x7f6034147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f603a34c700 1 --2- 172.21.15.6:0/2832542065 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6034161090 0x7f6034147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f6039b4b700 1 -- 172.21.15.6:0/2832542065 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6034161090 msgr2=0x7f6034147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f6039b4b700 1 --2- 172.21.15.6:0/2832542065 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6034161090 0x7f6034147fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f6039b4b700 1 -- 172.21.15.6:0/2832542065 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f603415c990 msgr2=0x7f6034147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f6039b4b700 1 --2- 172.21.15.6:0/2832542065 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f603415c990 0x7f6034147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f6039b4b700 1 -- 172.21.15.6:0/2832542065 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300094a0 con 0x7f60340cc310 2023-11-28T13:00:05.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f6039b4b700 1 --2- 172.21.15.6:0/2832542065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60340cc310 0x7f6034156750 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6030009b60 tx=0x7f6030004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.933+0000 7f602a7fc700 1 -- 172.21.15.6:0/2832542065 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300036a0 con 0x7f60340cc310 2023-11-28T13:00:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.934+0000 7f602a7fc700 1 -- 172.21.15.6:0/2832542065 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603001f070 con 0x7f60340cc310 2023-11-28T13:00:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.934+0000 7f603bdaf700 1 -- 172.21.15.6:0/2832542065 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60340ce870 con 0x7f60340cc310 2023-11-28T13:00:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.934+0000 7f602a7fc700 1 -- 172.21.15.6:0/2832542065 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030003a60 con 0x7f60340cc310 2023-11-28T13:00:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.934+0000 7f603bdaf700 1 -- 172.21.15.6:0/2832542065 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60340ce590 con 0x7f60340cc310 2023-11-28T13:00:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.934+0000 7f602a7fc700 1 -- 172.21.15.6:0/2832542065 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f603000f050 con 0x7f60340cc310 2023-11-28T13:00:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.934+0000 7f602a7fc700 1 -- 172.21.15.6:0/2832542065 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f6030021070 con 0x7f60340cc310 2023-11-28T13:00:05.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.934+0000 7f603bdaf700 1 -- 172.21.15.6:0/2832542065 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60340cb080 con 0x7f60340cc310 2023-11-28T13:00:05.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:05.937+0000 7f602a7fc700 1 -- 172.21.15.6:0/2832542065 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f6030015020 con 0x7f60340cc310 2023-11-28T13:00:06.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.008+0000 7f603bdaf700 1 -- 172.21.15.6:0/2832542065 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60340b3c00 con 0x7f60340cc310 2023-11-28T13:00:06.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.009+0000 7f602a7fc700 1 -- 172.21.15.6:0/2832542065 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f6030003bc0 con 0x7f60340cc310 2023-11-28T13:00:06.010 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:06.010 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":10,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:05.723768+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:06.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.010+0000 7f603bdaf700 1 -- 172.21.15.6:0/2832542065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60340cc310 msgr2=0x7f6034156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:06.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.010+0000 7f603bdaf700 1 --2- 172.21.15.6:0/2832542065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60340cc310 0x7f6034156750 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6030009b60 tx=0x7f6030004750 comp rx=0 tx=0).stop 2023-11-28T13:00:06.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.010+0000 7f603bdaf700 1 -- 172.21.15.6:0/2832542065 shutdown_connections 2023-11-28T13:00:06.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.010+0000 7f603bdaf700 1 --2- 172.21.15.6:0/2832542065 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6034161090 0x7f6034147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:06.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.010+0000 7f603bdaf700 1 --2- 172.21.15.6:0/2832542065 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f60340cc310 0x7f6034156750 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:06.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.010+0000 7f603bdaf700 1 --2- 172.21.15.6:0/2832542065 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f603415c990 0x7f6034147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:06.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.010+0000 7f603bdaf700 1 -- 172.21.15.6:0/2832542065 >> 172.21.15.6:0/2832542065 conn(0x7f60340b1650 msgr2=0x7f60340c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:06.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.010+0000 7f603bdaf700 1 -- 172.21.15.6:0/2832542065 shutdown_connections 2023-11-28T13:00:06.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:06.010+0000 7f603bdaf700 1 -- 172.21.15.6:0/2832542065 wait complete. 2023-11-28T13:00:06.024 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T13:00:06.024 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T13:00:06.115 INFO:tasks.ceph.osd.6.smithi061.stderr:2023-11-28T13:00:06.114+0000 7fe559735640 -1 Falling back to public interface 2023-11-28T13:00:06.368 INFO:tasks.ceph.osd.5.smithi061.stderr:2023-11-28T13:00:06.367+0000 7f2b18ae9640 -1 Falling back to public interface 2023-11-28T13:00:06.370 INFO:tasks.ceph.osd.1.smithi006.stderr:2023-11-28T13:00:06.369+0000 7f3be2ad5640 -1 Falling back to public interface 2023-11-28T13:00:06.383 INFO:tasks.ceph.osd.4.smithi061.stderr:2023-11-28T13:00:06.383+0000 7f314fe3a640 -1 Falling back to public interface 2023-11-28T13:00:06.390 INFO:tasks.ceph.osd.7.smithi061.stderr:2023-11-28T13:00:06.389+0000 7f00940bc640 -1 Falling back to public interface 2023-11-28T13:00:06.391 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-11-28T13:00:06.390+0000 7f31ca395640 -1 Falling back to public interface 2023-11-28T13:00:06.391 INFO:tasks.ceph.osd.0.smithi006.stderr:2023-11-28T13:00:06.390+0000 7fd68fe91640 -1 Falling back to public interface 2023-11-28T13:00:06.400 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-11-28T13:00:06.399+0000 7fc8ffbc3640 -1 Falling back to public interface 2023-11-28T13:00:06.529 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:06.527+0000 7f53b3660200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T13:00:06.577 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:06.577+0000 7f1d19520200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T13:00:06.610 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:06.609+0000 7f53b3660200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T13:00:06.659 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:06.658+0000 7f1d19520200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T13:00:06.755 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:06.754+0000 7f53b3660200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T13:00:06.805 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:06.804+0000 7f1d19520200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T13:00:06.830 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:06.949 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.948+0000 7f53a61ef700 1 Processor -- start 2023-11-28T13:00:06.949 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.948+0000 7f53a61ef700 1 -- start start 2023-11-28T13:00:06.951 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f53a61ef700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f53a00d0630 0x7f53a00b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:06.951 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f53a61ef700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a006f3e0 con 0x7f53a015a7a0 2023-11-28T13:00:06.951 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f53a61ef700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a006f6c0 con 0x7f53a00b4ed0 2023-11-28T13:00:06.951 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f53a61ef700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a00b5370 con 0x7f53a00d0630 2023-11-28T13:00:06.951 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f539f7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f53a015a7a0 0x7f53a006eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44254/0 (socket says 172.21.15.6:44254) 2023-11-28T13:00:06.951 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f539f7fe700 1 -- 172.21.15.6:0/2176459256 learned_addr learned my addr 172.21.15.6:0/2176459256 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f539effd700 1 --2- 172.21.15.6:0/2176459256 >> v2:172.21.15.61:3300/0 conn(0x7f53a00d0630 0x7f53a00b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f539e7fc700 1 -- 172.21.15.6:0/2176459256 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023453308 0 0) 0x7f53a006f3e0 con 0x7f53a015a7a0 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f539e7fc700 1 -- 172.21.15.6:0/2176459256 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53880035b0 con 0x7f53a015a7a0 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f539effd700 1 -- 172.21.15.6:0/2176459256 >> v1:172.21.15.61:6789/0 conn(0x7f53a00b4ed0 legacy=0x7f53a00acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f539effd700 1 -- 172.21.15.6:0/2176459256 >> v1:172.21.15.6:6789/0 conn(0x7f53a015a7a0 legacy=0x7f53a006eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.949+0000 7f539e7fc700 1 -- 172.21.15.6:0/2176459256 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403878853 0 0) 0x7f53a006f6c0 con 0x7f53a00b4ed0 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.950+0000 7f539effd700 1 -- 172.21.15.6:0/2176459256 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a00caa60 con 0x7f53a00d0630 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.950+0000 7f539effd700 1 --2- 172.21.15.6:0/2176459256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53a00d0630 0x7f53a00b4990 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5394009840 tx=0x7f5394009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7673c469fafac663 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.950+0000 7f539e7fc700 1 -- 172.21.15.6:0/2176459256 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394011830 con 0x7f53a00d0630 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.950+0000 7f539e7fc700 1 -- 172.21.15.6:0/2176459256 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5394011990 con 0x7f53a00d0630 2023-11-28T13:00:06.952 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.950+0000 7f539e7fc700 1 -- 172.21.15.6:0/2176459256 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394011d20 con 0x7f53a00d0630 2023-11-28T13:00:06.953 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.951+0000 7f53a61ef700 1 -- 172.21.15.6:0/2176459256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53a00d0630 msgr2=0x7f53a00b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:06.953 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.951+0000 7f53a61ef700 1 --2- 172.21.15.6:0/2176459256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53a00d0630 0x7f53a00b4990 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5394009840 tx=0x7f5394009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:06.953 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.951+0000 7f53a61ef700 1 -- 172.21.15.6:0/2176459256 shutdown_connections 2023-11-28T13:00:06.953 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.951+0000 7f53a61ef700 1 --2- 172.21.15.6:0/2176459256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53a00d0630 0x7f53a00b4990 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:06.953 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.951+0000 7f53a61ef700 1 -- 172.21.15.6:0/2176459256 >> 172.21.15.6:0/2176459256 conn(0x7f53a00b3500 msgr2=0x7f53a00cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:06.954 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.951+0000 7f53a61ef700 1 -- 172.21.15.6:0/2176459256 shutdown_connections 2023-11-28T13:00:06.954 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.951+0000 7f53a61ef700 1 -- 172.21.15.6:0/2176459256 wait complete. 2023-11-28T13:00:06.954 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.952+0000 7f53a61ef700 1 Processor -- start 2023-11-28T13:00:06.954 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.952+0000 7f53a61ef700 1 -- start start 2023-11-28T13:00:06.955 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f53a61ef700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53a00b4ed0 0x7f53a014bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:06.955 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f53a61ef700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00d0630 0x7f53a00b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f53a61ef700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53a015a7a0 0x7f53a0156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f53a61ef700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00ced80 con 0x7f53a00d0630 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f53a61ef700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00af5d0 con 0x7f53a015a7a0 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f53a61ef700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00af8b0 con 0x7f53a00b4ed0 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f539effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00d0630 0x7f53a00b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f539effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00d0630 0x7f53a00b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:56942/0 (socket says 172.21.15.6:56942) 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f539effd700 1 -- 172.21.15.6:0/1805197032 learned_addr learned my addr 172.21.15.6:0/1805197032 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f539ffff700 1 --2- 172.21.15.6:0/1805197032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53a015a7a0 0x7f53a0156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.953+0000 7f539f7fe700 1 --2- 172.21.15.6:0/1805197032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53a00b4ed0 0x7f53a014bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f539effd700 1 -- 172.21.15.6:0/1805197032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53a00b4ed0 msgr2=0x7f53a014bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f539effd700 1 --2- 172.21.15.6:0/1805197032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53a00b4ed0 0x7f53a014bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f539effd700 1 -- 172.21.15.6:0/1805197032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53a015a7a0 msgr2=0x7f53a0156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:06.956 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f539effd700 1 --2- 172.21.15.6:0/1805197032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53a015a7a0 0x7f53a0156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:06.957 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f539effd700 1 -- 172.21.15.6:0/1805197032 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940094a0 con 0x7f53a00d0630 2023-11-28T13:00:06.957 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f539effd700 1 --2- 172.21.15.6:0/1805197032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00d0630 0x7f53a00b00b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f53940090b0 tx=0x7f5394004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:06.957 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f5387fff700 1 -- 172.21.15.6:0/1805197032 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539401a7e0 con 0x7f53a00d0630 2023-11-28T13:00:06.957 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f5387fff700 1 -- 172.21.15.6:0/1805197032 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539401a940 con 0x7f53a00d0630 2023-11-28T13:00:06.957 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f5387fff700 1 -- 172.21.15.6:0/1805197032 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539401acd0 con 0x7f53a00d0630 2023-11-28T13:00:06.957 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f53a61ef700 1 -- 172.21.15.6:0/1805197032 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a00cb020 con 0x7f53a00d0630 2023-11-28T13:00:06.958 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.954+0000 7f53a61ef700 1 -- 172.21.15.6:0/1805197032 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a00cad40 con 0x7f53a00d0630 2023-11-28T13:00:06.958 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.955+0000 7f5387fff700 1 -- 172.21.15.6:0/1805197032 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f539400f050 con 0x7f53a00d0630 2023-11-28T13:00:06.958 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.955+0000 7f53a61ef700 1 -- 172.21.15.6:0/1805197032 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a00cb540 con 0x7f53a00d0630 2023-11-28T13:00:06.958 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.955+0000 7f5387fff700 1 -- 172.21.15.6:0/1805197032 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5394019a80 con 0x7f53a00d0630 2023-11-28T13:00:06.958 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:06.957+0000 7f5387fff700 1 -- 172.21.15.6:0/1805197032 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f5394029070 con 0x7f53a00d0630 2023-11-28T13:00:07.030 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.029+0000 7f53a61ef700 1 -- 172.21.15.6:0/1805197032 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53a00af430 con 0x7f53a00d0630 2023-11-28T13:00:07.030 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.029+0000 7f5387fff700 1 -- 172.21.15.6:0/1805197032 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f53a00af430 con 0x7f53a00d0630 2023-11-28T13:00:07.031 INFO:teuthology.misc.health.smithi006.stdout: 2023-11-28T13:00:07.031 INFO:teuthology.misc.health.smithi006.stdout:{"epoch":10,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:05.723768+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:07.032 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.030+0000 7f53a61ef700 1 -- 172.21.15.6:0/1805197032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00d0630 msgr2=0x7f53a00b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:07.032 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.030+0000 7f53a61ef700 1 --2- 172.21.15.6:0/1805197032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00d0630 0x7f53a00b00b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f53940090b0 tx=0x7f5394004750 comp rx=0 tx=0).stop 2023-11-28T13:00:07.032 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.030+0000 7f53a61ef700 1 -- 172.21.15.6:0/1805197032 shutdown_connections 2023-11-28T13:00:07.032 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.030+0000 7f53a61ef700 1 --2- 172.21.15.6:0/1805197032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53a015a7a0 0x7f53a0156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:07.032 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.030+0000 7f53a61ef700 1 --2- 172.21.15.6:0/1805197032 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53a00b4ed0 0x7f53a014bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:07.032 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.030+0000 7f53a61ef700 1 --2- 172.21.15.6:0/1805197032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53a00d0630 0x7f53a00b00b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:07.032 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.030+0000 7f53a61ef700 1 -- 172.21.15.6:0/1805197032 >> 172.21.15.6:0/1805197032 conn(0x7f53a00b3500 msgr2=0x7f53a0158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:07.032 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.030+0000 7f53a61ef700 1 -- 172.21.15.6:0/1805197032 shutdown_connections 2023-11-28T13:00:07.032 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:07.030+0000 7f53a61ef700 1 -- 172.21.15.6:0/1805197032 wait complete. 2023-11-28T13:00:07.042 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T13:00:07.071 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:07.070+0000 7f53b3660200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T13:00:07.116 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:07.116+0000 7f1d19520200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T13:00:07.210 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:07.209+0000 7f53b3660200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T13:00:07.257 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:07.257+0000 7f1d19520200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T13:00:07.437 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:07.435+0000 7f53b3660200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T13:00:07.483 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:07.483+0000 7f1d19520200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T13:00:07.600 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:07.599+0000 7f53b3660200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T13:00:07.645 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:07.645+0000 7f1d19520200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T13:00:07.721 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:07.720+0000 7f53b3660200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T13:00:07.767 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:07.766+0000 7f1d19520200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T13:00:08.078 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:08.077+0000 7f53b3660200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T13:00:08.101 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:08.100+0000 7f1d19520200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T13:00:08.181 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:08.180+0000 7f53b3660200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T13:00:08.204 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:08.204+0000 7f1d19520200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T13:00:08.372 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:08.371+0000 7f53b3660200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T13:00:08.396 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:08.395+0000 7f1d19520200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T13:00:08.924 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:08.922+0000 7f53b3660200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T13:00:08.976 INFO:tasks.ceph.mgr.x.smithi061.stderr:2023-11-28T13:00:08.975+0000 7f1d19520200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T13:00:09.627 INFO:tasks.ceph.osd.6.smithi061.stderr:2023-11-28T13:00:09.627+0000 7fe559735640 -1 osd.6 0 log_to_monitors true 2023-11-28T13:00:09.863 INFO:tasks.ceph.osd.1.smithi006.stderr:2023-11-28T13:00:09.862+0000 7f3be2ad5640 -1 osd.1 0 log_to_monitors true 2023-11-28T13:00:09.863 INFO:tasks.ceph.osd.5.smithi061.stderr:2023-11-28T13:00:09.863+0000 7f2b18ae9640 -1 osd.5 0 log_to_monitors true 2023-11-28T13:00:09.915 INFO:tasks.ceph.osd.4.smithi061.stderr:2023-11-28T13:00:09.915+0000 7f314fe3a640 -1 osd.4 0 log_to_monitors true 2023-11-28T13:00:09.916 INFO:tasks.ceph.osd.7.smithi061.stderr:2023-11-28T13:00:09.915+0000 7f00940bc640 -1 osd.7 0 log_to_monitors true 2023-11-28T13:00:09.925 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-11-28T13:00:09.924+0000 7fc8ffbc3640 -1 osd.2 0 log_to_monitors true 2023-11-28T13:00:09.937 INFO:tasks.ceph.osd.0.smithi006.stderr:2023-11-28T13:00:09.937+0000 7fd68fe91640 -1 osd.0 0 log_to_monitors true 2023-11-28T13:00:09.947 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:09.946+0000 7f536a88e700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T13:00:09.949 INFO:tasks.ceph.mgr.y.smithi006.stderr:2023-11-28T13:00:09.948+0000 7f536a88e700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T13:00:09.969 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-11-28T13:00:09.968+0000 7f31ca395640 -1 osd.3 0 log_to_monitors true 2023-11-28T13:00:12.918 INFO:tasks.ceph.osd.1.smithi006.stderr:2023-11-28T13:00:12.915+0000 7f3bd6e8d700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T13:00:12.937 INFO:tasks.ceph.osd.1.smithi006.stderr:2023-11-28T13:00:12.936+0000 7f3bcea67700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:12.979 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-11-28T13:00:12.978+0000 7f31be74d700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T13:00:12.990 INFO:tasks.ceph.osd.3.smithi006.stderr:2023-11-28T13:00:12.989+0000 7f31b6327700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:12.993 INFO:tasks.ceph.osd.7.smithi061.stderr:2023-11-28T13:00:12.987+0000 7f0088474700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T13:00:13.002 INFO:tasks.ceph.osd.0.smithi006.stderr:2023-11-28T13:00:13.001+0000 7fd684249700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T13:00:13.004 INFO:tasks.ceph.osd.7.smithi061.stderr:2023-11-28T13:00:13.004+0000 7f008004e700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:13.009 INFO:tasks.ceph.osd.5.smithi061.stderr:2023-11-28T13:00:13.009+0000 7f2b0cea1700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T13:00:13.015 INFO:tasks.ceph.osd.0.smithi006.stderr:2023-11-28T13:00:13.014+0000 7fd67be23700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:13.019 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-11-28T13:00:13.018+0000 7fc8f3f7b700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T13:00:13.024 INFO:tasks.ceph.osd.5.smithi061.stderr:2023-11-28T13:00:13.024+0000 7f2b04a7b700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:13.036 INFO:tasks.ceph.osd.2.smithi006.stderr:2023-11-28T13:00:13.035+0000 7fc8ebb55700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:13.062 INFO:tasks.ceph.osd.6.smithi061.stderr:2023-11-28T13:00:13.062+0000 7fe54daed700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T13:00:13.079 INFO:tasks.ceph.osd.6.smithi061.stderr:2023-11-28T13:00:13.079+0000 7fe5456c7700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:13.132 INFO:tasks.ceph.osd.4.smithi061.stderr:2023-11-28T13:00:13.132+0000 7f31441f2700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T13:00:13.147 INFO:tasks.ceph.osd.4.smithi061.stderr:2023-11-28T13:00:13.147+0000 7f313bdcc700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T13:00:13.848 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:13.981 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.980+0000 7fe070f8f700 1 Processor -- start 2023-11-28T13:00:13.981 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.980+0000 7fe070f8f700 1 -- start start 2023-11-28T13:00:13.983 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe070f8f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe06c0b4c00 0x7fe06c0ae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.983 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe070f8f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe06c0b4fd0 0x7fe06c0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.983 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe070f8f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06c0ce950 con 0x7fe06c0cdae0 2023-11-28T13:00:13.983 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe070f8f700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06c0d0310 con 0x7fe06c0b4c00 2023-11-28T13:00:13.983 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe070f8f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06c06f530 con 0x7fe06c0b4fd0 2023-11-28T13:00:13.983 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe06a59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe06c0b4c00 0x7fe06c0ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.983 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe06a59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe06c0b4c00 0x7fe06c0ae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57158/0 (socket says 172.21.15.6:57158) 2023-11-28T13:00:13.983 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe06a59c700 1 -- 172.21.15.6:0/3726942705 learned_addr learned my addr 172.21.15.6:0/3726942705 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:13.984 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe069d9b700 1 --2- 172.21.15.6:0/3726942705 >> v2:172.21.15.61:3300/0 conn(0x7fe06c0b4fd0 0x7fe06c0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.984 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe06a59c700 1 -- 172.21.15.6:0/3726942705 >> v2:172.21.15.61:3300/0 conn(0x7fe06c0b4fd0 msgr2=0x7fe06c0cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:13.984 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe06a59c700 1 --2- 172.21.15.6:0/3726942705 >> v2:172.21.15.61:3300/0 conn(0x7fe06c0b4fd0 0x7fe06c0cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.984 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe06a59c700 1 -- 172.21.15.6:0/3726942705 >> v1:172.21.15.61:6789/0 conn(0x7fe06c0cdae0 legacy=0x7fe06c0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.984 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe06a59c700 1 -- 172.21.15.6:0/3726942705 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c0ceba0 con 0x7fe06c0b4c00 2023-11-28T13:00:13.984 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe06a59c700 1 --2- 172.21.15.6:0/3726942705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0b4c00 0x7fe06c0ae960 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe06c070310 tx=0x7fe05c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54231dff4a81c742 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.984 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.981+0000 7fe06959a700 1 -- 172.21.15.6:0/3726942705 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c011830 con 0x7fe06c0b4c00 2023-11-28T13:00:13.984 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.982+0000 7fe06959a700 1 -- 172.21.15.6:0/3726942705 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c011990 con 0x7fe06c0b4c00 2023-11-28T13:00:13.984 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.982+0000 7fe06959a700 1 -- 172.21.15.6:0/3726942705 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c011d20 con 0x7fe06c0b4c00 2023-11-28T13:00:13.985 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.982+0000 7fe070f8f700 1 -- 172.21.15.6:0/3726942705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0b4c00 msgr2=0x7fe06c0ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.985 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.982+0000 7fe070f8f700 1 --2- 172.21.15.6:0/3726942705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0b4c00 0x7fe06c0ae960 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe06c070310 tx=0x7fe05c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.985 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.982+0000 7fe070f8f700 1 -- 172.21.15.6:0/3726942705 shutdown_connections 2023-11-28T13:00:13.985 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.982+0000 7fe070f8f700 1 --2- 172.21.15.6:0/3726942705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0b4c00 0x7fe06c0ae960 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.986 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.982+0000 7fe070f8f700 1 --2- 172.21.15.6:0/3726942705 >> v2:172.21.15.61:3300/0 conn(0x7fe06c0b4fd0 0x7fe06c0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.986 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.982+0000 7fe070f8f700 1 -- 172.21.15.6:0/3726942705 >> 172.21.15.6:0/3726942705 conn(0x7fe06c0b3500 msgr2=0x7fe06c0cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:13.986 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.982+0000 7fe070f8f700 1 -- 172.21.15.6:0/3726942705 shutdown_connections 2023-11-28T13:00:13.986 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.982+0000 7fe070f8f700 1 -- 172.21.15.6:0/3726942705 wait complete. 2023-11-28T13:00:13.986 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.984+0000 7fe070f8f700 1 Processor -- start 2023-11-28T13:00:13.986 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.984+0000 7fe070f8f700 1 -- start start 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe070f8f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe06c0b4c00 0x7fe06c1ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe070f8f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe06c0b4fd0 0x7fe06c0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe070f8f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0cdae0 0x7fe06c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe070f8f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c1cf660 con 0x7fe06c0cdae0 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe070f8f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c1cf940 con 0x7fe06c0b4c00 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe070f8f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c1cfc20 con 0x7fe06c0b4fd0 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe06ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0cdae0 0x7fe06c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe06ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0cdae0 0x7fe06c06c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57168/0 (socket says 172.21.15.6:57168) 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe06ad9d700 1 -- 172.21.15.6:0/2463459482 learned_addr learned my addr 172.21.15.6:0/2463459482 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe06a59c700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe06c0b4c00 0x7fe06c1ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.989 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe069d9b700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe06c0b4fd0 0x7fe06c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe06ad9d700 1 -- 172.21.15.6:0/2463459482 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe06c0b4fd0 msgr2=0x7fe06c0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe06ad9d700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe06c0b4fd0 0x7fe06c0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe06ad9d700 1 -- 172.21.15.6:0/2463459482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe06c0b4c00 msgr2=0x7fe06c1ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe06ad9d700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe06c0b4c00 0x7fe06c1ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe06ad9d700 1 -- 172.21.15.6:0/2463459482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c009550 con 0x7fe06c0cdae0 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe06ad9d700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0cdae0 0x7fe06c06c290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe06400edf0 tx=0x7fe06400c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe05affd700 1 -- 172.21.15.6:0/2463459482 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064003a00 con 0x7fe06c0cdae0 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe05affd700 1 -- 172.21.15.6:0/2463459482 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe064003b60 con 0x7fe06c0cdae0 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe070f8f700 1 -- 172.21.15.6:0/2463459482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06c0af170 con 0x7fe06c0cdae0 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe05affd700 1 -- 172.21.15.6:0/2463459482 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0640050d0 con 0x7fe06c0cdae0 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.985+0000 7fe070f8f700 1 -- 172.21.15.6:0/2463459482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c0d0560 con 0x7fe06c0cdae0 2023-11-28T13:00:13.990 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.987+0000 7fe070f8f700 1 -- 172.21.15.6:0/2463459482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06c0b5440 con 0x7fe06c0cdae0 2023-11-28T13:00:13.991 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.987+0000 7fe05affd700 1 -- 172.21.15.6:0/2463459482 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fe064021070 con 0x7fe06c0cdae0 2023-11-28T13:00:13.991 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.987+0000 7fe05affd700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe04c0682a0 0x7fe04c06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:13.991 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.987+0000 7fe05affd700 1 -- 172.21.15.6:0/2463459482 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fe064087dd0 con 0x7fe06c0cdae0 2023-11-28T13:00:13.991 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.987+0000 7fe06a59c700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe04c0682a0 0x7fe04c06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:13.991 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.988+0000 7fe06a59c700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe04c0682a0 0x7fe04c06a750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009c10 tx=0x7fe05c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:13.993 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:13.992+0000 7fe05affd700 1 -- 172.21.15.6:0/2463459482 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe064054c20 con 0x7fe06c0cdae0 2023-11-28T13:00:14.189 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.188+0000 7fe070f8f700 1 -- 172.21.15.6:0/2463459482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe06c0675a0 con 0x7fe06c0cdae0 2023-11-28T13:00:14.190 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.189+0000 7fe05affd700 1 -- 172.21.15.6:0/2463459482 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10440 (secure 0 0 0) 0x7fe06401f070 con 0x7fe06c0cdae0 2023-11-28T13:00:14.190 INFO:teuthology.misc.health.smithi006.stdout: 2023-11-28T13:00:14.190 INFO:teuthology.misc.health.smithi006.stdout:{"epoch":13,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:13.931581+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:14.192 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.190+0000 7fe070f8f700 1 -- 172.21.15.6:0/2463459482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe04c0682a0 msgr2=0x7fe04c06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.192 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe04c0682a0 0x7fe04c06a750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009c10 tx=0x7fe05c008030 comp rx=0 tx=0).stop 2023-11-28T13:00:14.192 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 -- 172.21.15.6:0/2463459482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0cdae0 msgr2=0x7fe06c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.192 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0cdae0 0x7fe06c06c290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe06400edf0 tx=0x7fe06400c550 comp rx=0 tx=0).stop 2023-11-28T13:00:14.193 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 -- 172.21.15.6:0/2463459482 shutdown_connections 2023-11-28T13:00:14.193 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe06c0cdae0 0x7fe06c06c290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.193 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe06c0b4c00 0x7fe06c1ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.193 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe04c0682a0 0x7fe04c06a750 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.193 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 --2- 172.21.15.6:0/2463459482 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe06c0b4fd0 0x7fe06c0c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.193 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 -- 172.21.15.6:0/2463459482 >> 172.21.15.6:0/2463459482 conn(0x7fe06c0b3500 msgr2=0x7fe06c0ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:14.193 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 -- 172.21.15.6:0/2463459482 shutdown_connections 2023-11-28T13:00:14.193 INFO:teuthology.misc.health.smithi006.stderr:2023-11-28T13:00:14.191+0000 7fe070f8f700 1 -- 172.21.15.6:0/2463459482 wait complete. 2023-11-28T13:00:14.203 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T13:00:14.203 INFO:tasks.ceph:Creating RBD pool 2023-11-28T13:00:14.203 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T13:00:14.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.325+0000 7fe7e120e700 1 Processor -- start 2023-11-28T13:00:14.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.325+0000 7fe7e120e700 1 -- start start 2023-11-28T13:00:14.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.326+0000 7fe7e120e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe7dc0b62d0 0x7fe7dc0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.326+0000 7fe7e120e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe7dc06f8b0 0x7fe7dc0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.326+0000 7fe7e120e700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7dc0b2fe0 con 0x7fe7dc0b2bc0 2023-11-28T13:00:14.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.326+0000 7fe7e120e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7dc0cd2a0 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.326+0000 7fe7e120e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7dc0d0400 con 0x7fe7dc0b62d0 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.326+0000 7fe7da59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe7dc06f8b0 0x7fe7dc0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.326+0000 7fe7db59e700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe7dc0b2bc0 0x7fe7dc0ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44526/0 (socket says 172.21.15.6:44526) 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.326+0000 7fe7db59e700 1 -- 172.21.15.6:0/2028423222 learned_addr learned my addr 172.21.15.6:0/2028423222 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7dad9d700 1 --2- 172.21.15.6:0/2028423222 >> v2:172.21.15.61:3300/0 conn(0x7fe7dc0b62d0 0x7fe7dc0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7da59c700 1 -- 172.21.15.6:0/2028423222 >> v2:172.21.15.61:3300/0 conn(0x7fe7dc0b62d0 msgr2=0x7fe7dc0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7da59c700 1 --2- 172.21.15.6:0/2028423222 >> v2:172.21.15.61:3300/0 conn(0x7fe7dc0b62d0 0x7fe7dc0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7da59c700 1 -- 172.21.15.6:0/2028423222 >> v1:172.21.15.6:6789/0 conn(0x7fe7dc0b2bc0 legacy=0x7fe7dc0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7d9d9b700 1 -- 172.21.15.6:0/2028423222 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029577861 0 0) 0x7fe7dc0b2fe0 con 0x7fe7dc0b2bc0 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7da59c700 1 -- 172.21.15.6:0/2028423222 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc0d06e0 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7dad9d700 1 --2- 172.21.15.6:0/2028423222 >> v2:172.21.15.61:3300/0 conn(0x7fe7dc0b62d0 0x7fe7dc0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:14.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7da59c700 1 --2- 172.21.15.6:0/2028423222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 0x7fe7dc0b2680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0720d0 tx=0x7fe7c4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55f5e20babedfc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7d9d9b700 1 -- 172.21.15.6:0/2028423222 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4011830 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7d9d9b700 1 -- 172.21.15.6:0/2028423222 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c4011990 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7d9d9b700 1 -- 172.21.15.6:0/2028423222 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4011d20 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7e120e700 1 -- 172.21.15.6:0/2028423222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 msgr2=0x7fe7dc0b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7e120e700 1 --2- 172.21.15.6:0/2028423222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 0x7fe7dc0b2680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0720d0 tx=0x7fe7c4009b20 comp rx=0 tx=0).stop 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7e120e700 1 -- 172.21.15.6:0/2028423222 shutdown_connections 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7e120e700 1 --2- 172.21.15.6:0/2028423222 >> v2:172.21.15.61:3300/0 conn(0x7fe7dc0b62d0 0x7fe7dc0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7e120e700 1 --2- 172.21.15.6:0/2028423222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 0x7fe7dc0b2680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.327+0000 7fe7e120e700 1 -- 172.21.15.6:0/2028423222 >> 172.21.15.6:0/2028423222 conn(0x7fe7dc0cd680 msgr2=0x7fe7dc0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.328+0000 7fe7e120e700 1 -- 172.21.15.6:0/2028423222 shutdown_connections 2023-11-28T13:00:14.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.328+0000 7fe7e120e700 1 -- 172.21.15.6:0/2028423222 wait complete. 2023-11-28T13:00:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.329+0000 7fe7e120e700 1 Processor -- start 2023-11-28T13:00:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.329+0000 7fe7e120e700 1 -- start start 2023-11-28T13:00:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7e120e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 0x7fe7dc1f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7e120e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7dc0b2bc0 0x7fe7dc0c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7e120e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7dc0b62d0 0x7fe7dc0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7e120e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc146c80 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7e120e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc0cab60 con 0x7fe7dc0b2bc0 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7e120e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc0cae40 con 0x7fe7dc0b62d0 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7dad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 0x7fe7dc1f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7dad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 0x7fe7dc1f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57208/0 (socket says 172.21.15.6:57208) 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7dad9d700 1 -- 172.21.15.6:0/3593954499 learned_addr learned my addr 172.21.15.6:0/3593954499 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7da59c700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7dc0b2bc0 0x7fe7dc0c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7db59e700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7dc0b62d0 0x7fe7dc0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7dad9d700 1 -- 172.21.15.6:0/3593954499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7dc0b62d0 msgr2=0x7fe7dc0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7dad9d700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7dc0b62d0 0x7fe7dc0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7dad9d700 1 -- 172.21.15.6:0/3593954499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7dc0b2bc0 msgr2=0x7fe7dc0c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7dad9d700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7dc0b2bc0 0x7fe7dc0c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7dad9d700 1 -- 172.21.15.6:0/3593954499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c40094a0 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7da59c700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7dc0b2bc0 0x7fe7dc0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7dad9d700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 0x7fe7dc1f3bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c7f0 tx=0x7fe7cc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.330+0000 7fe7d37fe700 1 -- 172.21.15.6:0/3593954499 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc011800 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.331+0000 7fe7d37fe700 1 -- 172.21.15.6:0/3593954499 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7cc011960 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.331+0000 7fe7e120e700 1 -- 172.21.15.6:0/3593954499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc06fd30 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.331+0000 7fe7d37fe700 1 -- 172.21.15.6:0/3593954499 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc011d10 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.331+0000 7fe7e120e700 1 -- 172.21.15.6:0/3593954499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc0cffb0 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.331+0000 7fe7e120e700 1 -- 172.21.15.6:0/3593954499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7dc0b66a0 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.332+0000 7fe7d37fe700 1 -- 172.21.15.6:0/3593954499 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fe7cc021070 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.333+0000 7fe7d37fe700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe7bc0682a0 0x7fe7bc06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.333+0000 7fe7da59c700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe7bc0682a0 0x7fe7bc06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:14.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.333+0000 7fe7d37fe700 1 -- 172.21.15.6:0/3593954499 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fe7cc085940 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.333+0000 7fe7da59c700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe7bc0682a0 0x7fe7bc06a750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40090b0 tx=0x7fe7c4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:14.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.336+0000 7fe7d37fe700 1 -- 172.21.15.6:0/3593954499 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe7cc0527b0 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.530+0000 7fe7e120e700 1 -- 172.21.15.6:0/3593954499 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fe7dc0739f0 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.941+0000 7fe7d37fe700 1 -- 172.21.15.6:0/3593954499 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7fe7cc01f0a0 con 0x7fe7dc06f8b0 2023-11-28T13:00:14.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 -- 172.21.15.6:0/3593954499 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe7bc0682a0 msgr2=0x7fe7bc06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe7bc0682a0 0x7fe7bc06a750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40090b0 tx=0x7fe7c4008030 comp rx=0 tx=0).stop 2023-11-28T13:00:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 -- 172.21.15.6:0/3593954499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 msgr2=0x7fe7dc1f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 0x7fe7dc1f3bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c7f0 tx=0x7fe7cc00cbb0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 -- 172.21.15.6:0/3593954499 shutdown_connections 2023-11-28T13:00:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe7bc0682a0 0x7fe7bc06a750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7dc0b62d0 0x7fe7dc0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7dc06f8b0 0x7fe7dc1f3bc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 --2- 172.21.15.6:0/3593954499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7dc0b2bc0 0x7fe7dc0c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 -- 172.21.15.6:0/3593954499 >> 172.21.15.6:0/3593954499 conn(0x7fe7dc0cd680 msgr2=0x7fe7dc0b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:14.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 -- 172.21.15.6:0/3593954499 shutdown_connections 2023-11-28T13:00:14.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:14.944+0000 7fe7e120e700 1 -- 172.21.15.6:0/3593954499 wait complete. 2023-11-28T13:00:14.947 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'rbd' created 2023-11-28T13:00:14.959 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T13:00:15.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.089+0000 7f86e0b5a700 1 Processor -- start 2023-11-28T13:00:15.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.089+0000 7f86e0b5a700 1 -- start start 2023-11-28T13:00:15.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.090+0000 7f86e0b5a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f86dc0cc310 0x7f86dc0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.090+0000 7f86e0b5a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc0ce340 con 0x7f86dc15c990 2023-11-28T13:00:15.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.090+0000 7f86e0b5a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86dc0cebe0 con 0x7f86dc161090 2023-11-28T13:00:15.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.090+0000 7f86e0b5a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86dc0cf780 con 0x7f86dc0cc310 2023-11-28T13:00:15.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.090+0000 7f86dad9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f86dc0cc310 0x7f86dc0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.090+0000 7f86da59c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f86dc15c990 0x7f86dc0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44560/0 (socket says 172.21.15.6:44560) 2023-11-28T13:00:15.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.090+0000 7f86da59c700 1 -- 172.21.15.6:0/3209755059 learned_addr learned my addr 172.21.15.6:0/3209755059 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:15.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.091+0000 7f86d959a700 1 -- 172.21.15.6:0/3209755059 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732295622 0 0) 0x7f86dc0ce340 con 0x7f86dc15c990 2023-11-28T13:00:15.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.091+0000 7f86d959a700 1 -- 172.21.15.6:0/3209755059 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c40035b0 con 0x7f86dc15c990 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.091+0000 7f86dad9d700 1 -- 172.21.15.6:0/3209755059 >> v1:172.21.15.61:6789/0 conn(0x7f86dc161090 legacy=0x7f86dc0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.091+0000 7f86dad9d700 1 -- 172.21.15.6:0/3209755059 >> v1:172.21.15.6:6789/0 conn(0x7f86dc15c990 legacy=0x7f86dc0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.091+0000 7f86dad9d700 1 -- 172.21.15.6:0/3209755059 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0cccb0 con 0x7f86dc0cc310 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.091+0000 7f86dad9d700 1 --2- 172.21.15.6:0/3209755059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc0cc310 0x7f86dc0cc770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009d90 tx=0x7f86d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539bcf43e61b1204 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.091+0000 7f86d959a700 1 -- 172.21.15.6:0/3209755059 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400e070 con 0x7f86dc0cc310 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.091+0000 7f86d959a700 1 -- 172.21.15.6:0/3209755059 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d400bae0 con 0x7f86dc0cc310 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.091+0000 7f86d959a700 1 -- 172.21.15.6:0/3209755059 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400be90 con 0x7f86dc0cc310 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.092+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3209755059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc0cc310 msgr2=0x7f86dc0cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.092+0000 7f86e0b5a700 1 --2- 172.21.15.6:0/3209755059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc0cc310 0x7f86dc0cc770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009d90 tx=0x7f86d400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.092+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3209755059 shutdown_connections 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.092+0000 7f86e0b5a700 1 --2- 172.21.15.6:0/3209755059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc0cc310 0x7f86dc0cc770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.092+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3209755059 >> 172.21.15.6:0/3209755059 conn(0x7f86dc0b1650 msgr2=0x7f86dc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:15.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.092+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3209755059 shutdown_connections 2023-11-28T13:00:15.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.092+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3209755059 wait complete. 2023-11-28T13:00:15.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.093+0000 7f86e0b5a700 1 Processor -- start 2023-11-28T13:00:15.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.093+0000 7f86e0b5a700 1 -- start start 2023-11-28T13:00:15.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86e0b5a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86dc0cc310 0x7f86dc158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86e0b5a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc15c990 0x7f86dc0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86e0b5a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86dc161090 0x7f86dc0c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86e0b5a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc158690 con 0x7f86dc15c990 2023-11-28T13:00:15.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86e0b5a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0ad1e0 con 0x7f86dc0cc310 2023-11-28T13:00:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86e0b5a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc0ad4c0 con 0x7f86dc161090 2023-11-28T13:00:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86d9d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc15c990 0x7f86dc0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86d9d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc15c990 0x7f86dc0b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57248/0 (socket says 172.21.15.6:57248) 2023-11-28T13:00:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86d9d9b700 1 -- 172.21.15.6:0/3222109371 learned_addr learned my addr 172.21.15.6:0/3222109371 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86dad9d700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86dc161090 0x7f86dc0c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86da59c700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86dc0cc310 0x7f86dc158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86d9d9b700 1 -- 172.21.15.6:0/3222109371 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86dc161090 msgr2=0x7f86dc0c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86d9d9b700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86dc161090 0x7f86dc0c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86d9d9b700 1 -- 172.21.15.6:0/3222109371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86dc0cc310 msgr2=0x7f86dc158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:15.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86d9d9b700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86dc0cc310 0x7f86dc158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86d9d9b700 1 -- 172.21.15.6:0/3222109371 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d4009a70 con 0x7f86dc15c990 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86da59c700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86dc0cc310 0x7f86dc158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.094+0000 7f86d9d9b700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc15c990 0x7f86dc0b3020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00b4a0 tx=0x7f86cc00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.095+0000 7f86caffd700 1 -- 172.21.15.6:0/3222109371 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc011830 con 0x7f86dc15c990 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.095+0000 7f86caffd700 1 -- 172.21.15.6:0/3222109371 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc011990 con 0x7f86dc15c990 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.095+0000 7f86caffd700 1 -- 172.21.15.6:0/3222109371 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc011d40 con 0x7f86dc15c990 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.095+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3222109371 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc0ce590 con 0x7f86dc15c990 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.095+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3222109371 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc0cb3f0 con 0x7f86dc15c990 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.095+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3222109371 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86dc0ce870 con 0x7f86dc15c990 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.096+0000 7f86caffd700 1 -- 172.21.15.6:0/3222109371 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f86cc00f050 con 0x7f86dc15c990 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.096+0000 7f86caffd700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86bc0682a0 0x7f86bc06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:15.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.097+0000 7f86da59c700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86bc0682a0 0x7f86bc06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:15.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.097+0000 7f86caffd700 1 -- 172.21.15.6:0/3222109371 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4813+0+0 (secure 0 0 0) 0x7f86cc085320 con 0x7f86dc15c990 2023-11-28T13:00:15.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.097+0000 7f86da59c700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86bc0682a0 0x7f86bc06a750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005910 tx=0x7f86d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:15.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.101+0000 7f86caffd700 1 -- 172.21.15.6:0/3222109371 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f86cc052050 con 0x7f86dc15c990 2023-11-28T13:00:15.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.298+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3222109371 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f86dc0739f0 con 0x7f86dc15c990 2023-11-28T13:00:15.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.952+0000 7f86caffd700 1 -- 172.21.15.6:0/3222109371 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f86cc055670 con 0x7f86dc15c990 2023-11-28T13:00:15.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3222109371 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86bc0682a0 msgr2=0x7f86bc06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:15.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86bc0682a0 0x7f86bc06a750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005910 tx=0x7f86d0009040 comp rx=0 tx=0).stop 2023-11-28T13:00:15.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3222109371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc15c990 msgr2=0x7f86dc0b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:15.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc15c990 0x7f86dc0b3020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00b4a0 tx=0x7f86cc00b860 comp rx=0 tx=0).stop 2023-11-28T13:00:15.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3222109371 shutdown_connections 2023-11-28T13:00:15.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86bc0682a0 0x7f86bc06a750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86dc161090 0x7f86dc0c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86dc0cc310 0x7f86dc158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 --2- 172.21.15.6:0/3222109371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f86dc15c990 0x7f86dc0b3020 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:15.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3222109371 >> 172.21.15.6:0/3222109371 conn(0x7f86dc0b1650 msgr2=0x7f86dc06b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:15.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3222109371 shutdown_connections 2023-11-28T13:00:15.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:15.955+0000 7f86e0b5a700 1 -- 172.21.15.6:0/3222109371 wait complete. 2023-11-28T13:00:15.959 INFO:teuthology.orchestra.run.smithi006.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T13:00:15.971 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T13:00:15.972 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:00:15.972 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:00:15.974 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:00:15.975 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:00:15.975 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:00:15.977 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:00:15.977 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:00:15.977 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:00:15.980 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:00:15.980 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:00:15.980 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:00:15.982 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:00:15.982 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T13:00:15.983 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:16.064 INFO:tasks.ceph.mds.a.smithi006.stdout:starting mds.a at 2023-11-28T13:00:16.066 INFO:tasks.ceph.mds.c.smithi006.stdout:starting mds.c at 2023-11-28T13:00:16.070 INFO:tasks.ceph.mds.d.smithi061.stdout:starting mds.d at 2023-11-28T13:00:16.070 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2023-11-28T13:00:16.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.122+0000 7f69d0a26700 1 Processor -- start 2023-11-28T13:00:16.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.122+0000 7f69d0a26700 1 -- start start 2023-11-28T13:00:16.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.123+0000 7f69d0a26700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69cc15c640 0x7f69cc0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.123+0000 7f69d0a26700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f69cc06bf30 0x7f69cc06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.123+0000 7f69d0a26700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69cc06c890 con 0x7f69cc0d2820 2023-11-28T13:00:16.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.123+0000 7f69d0a26700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69cc15ca10 con 0x7f69cc06bf30 2023-11-28T13:00:16.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.123+0000 7f69d0a26700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0b6350 con 0x7f69cc15c640 2023-11-28T13:00:16.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69cad9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f69cc06bf30 0x7f69cc06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69cad9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f69cc06bf30 0x7f69cc06c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57296/0 (socket says 172.21.15.6:57296) 2023-11-28T13:00:16.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69cad9d700 1 -- 172.21.15.6:0/3017489103 learned_addr learned my addr 172.21.15.6:0/3017489103 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69ca59c700 1 --2- 172.21.15.6:0/3017489103 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69cc15c640 0x7f69cc0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69cad9d700 1 -- 172.21.15.6:0/3017489103 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69cc15c640 msgr2=0x7f69cc0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69cad9d700 1 --2- 172.21.15.6:0/3017489103 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69cc15c640 0x7f69cc0cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69cad9d700 1 -- 172.21.15.6:0/3017489103 >> v1:172.21.15.61:6789/0 conn(0x7f69cc0d2820 legacy=0x7f69cc0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69cad9d700 1 -- 172.21.15.6:0/3017489103 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc0d1470 con 0x7f69cc06bf30 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69cad9d700 1 --2- 172.21.15.6:0/3017489103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc06bf30 0x7f69cc06c350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009e50 tx=0x7f69c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca412bfdd05ccfba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69c959a700 1 -- 172.21.15.6:0/3017489103 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c400e070 con 0x7f69cc06bf30 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69c959a700 1 -- 172.21.15.6:0/3017489103 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c4003a20 con 0x7f69cc06bf30 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.124+0000 7f69c959a700 1 -- 172.21.15.6:0/3017489103 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4003dd0 con 0x7f69cc06bf30 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.125+0000 7f69d0a26700 1 -- 172.21.15.6:0/3017489103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc06bf30 msgr2=0x7f69cc06c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.125+0000 7f69d0a26700 1 --2- 172.21.15.6:0/3017489103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc06bf30 0x7f69cc06c350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009e50 tx=0x7f69c400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:16.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.125+0000 7f69d0a26700 1 -- 172.21.15.6:0/3017489103 shutdown_connections 2023-11-28T13:00:16.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.125+0000 7f69d0a26700 1 --2- 172.21.15.6:0/3017489103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc06bf30 0x7f69cc06c350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.125+0000 7f69d0a26700 1 --2- 172.21.15.6:0/3017489103 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69cc15c640 0x7f69cc0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.125+0000 7f69d0a26700 1 -- 172.21.15.6:0/3017489103 >> 172.21.15.6:0/3017489103 conn(0x7f69cc0d2df0 msgr2=0x7f69cc0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:16.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.125+0000 7f69d0a26700 1 -- 172.21.15.6:0/3017489103 shutdown_connections 2023-11-28T13:00:16.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.125+0000 7f69d0a26700 1 -- 172.21.15.6:0/3017489103 wait complete. 2023-11-28T13:00:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.126+0000 7f69d0a26700 1 Processor -- start 2023-11-28T13:00:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.127+0000 7f69d0a26700 1 -- start start 2023-11-28T13:00:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.127+0000 7f69d0a26700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69cc06bf30 0x7f69cc157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.127+0000 7f69d0a26700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69cc0d2820 0x7f69cc0c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69d0a26700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc15c640 0x7f69cc0c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69d0a26700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc158160 con 0x7f69cc15c640 2023-11-28T13:00:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69d0a26700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc1e8ba0 con 0x7f69cc0d2820 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69d0a26700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc0cb5c0 con 0x7f69cc06bf30 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69cad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc15c640 0x7f69cc0c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69cad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc15c640 0x7f69cc0c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57302/0 (socket says 172.21.15.6:57302) 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69cad9d700 1 -- 172.21.15.6:0/1295103478 learned_addr learned my addr 172.21.15.6:0/1295103478 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69ca59c700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69cc06bf30 0x7f69cc157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69c9d9b700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69cc0d2820 0x7f69cc0c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69cad9d700 1 -- 172.21.15.6:0/1295103478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69cc06bf30 msgr2=0x7f69cc157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69cad9d700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69cc06bf30 0x7f69cc157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69cad9d700 1 -- 172.21.15.6:0/1295103478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69cc0d2820 msgr2=0x7f69cc0c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69cad9d700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69cc0d2820 0x7f69cc0c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69cad9d700 1 -- 172.21.15.6:0/1295103478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c4009b30 con 0x7f69cc15c640 2023-11-28T13:00:16.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69cad9d700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc15c640 0x7f69cc0c7e80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f69c4015040 tx=0x7f69c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69baffd700 1 -- 172.21.15.6:0/1295103478 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4004440 con 0x7f69cc15c640 2023-11-28T13:00:16.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69baffd700 1 -- 172.21.15.6:0/1295103478 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c40045c0 con 0x7f69cc15c640 2023-11-28T13:00:16.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69d0a26700 1 -- 172.21.15.6:0/1295103478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc0cd7c0 con 0x7f69cc15c640 2023-11-28T13:00:16.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69baffd700 1 -- 172.21.15.6:0/1295103478 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c4004970 con 0x7f69cc15c640 2023-11-28T13:00:16.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.128+0000 7f69d0a26700 1 -- 172.21.15.6:0/1295103478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc0cd4e0 con 0x7f69cc15c640 2023-11-28T13:00:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.129+0000 7f69d0a26700 1 -- 172.21.15.6:0/1295103478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69cc0d1710 con 0x7f69cc15c640 2023-11-28T13:00:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.130+0000 7f69baffd700 1 -- 172.21.15.6:0/1295103478 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f69c4022030 con 0x7f69cc15c640 2023-11-28T13:00:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.130+0000 7f69baffd700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f69ac0682a0 0x7f69ac06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.130+0000 7f69baffd700 1 -- 172.21.15.6:0/1295103478 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f69c4086b20 con 0x7f69cc15c640 2023-11-28T13:00:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.130+0000 7f69ca59c700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f69ac0682a0 0x7f69ac06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.131+0000 7f69ca59c700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f69ac0682a0 0x7f69ac06a750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f69bc006b20 tx=0x7f69bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.135+0000 7f69baffd700 1 -- 172.21.15.6:0/1295103478 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f69c40535b0 con 0x7f69cc15c640 2023-11-28T13:00:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.332+0000 7f69d0a26700 1 -- 172.21.15.6:0/1295103478 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69cc0c9c70 con 0x7f69cc15c640 2023-11-28T13:00:16.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.333+0000 7f69baffd700 1 -- 172.21.15.6:0/1295103478 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14129 (secure 0 0 0) 0x7f69c4017020 con 0x7f69cc15c640 2023-11-28T13:00:16.335 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:16.335 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":15,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:15.941055+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:16.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 -- 172.21.15.6:0/1295103478 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f69ac0682a0 msgr2=0x7f69ac06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f69ac0682a0 0x7f69ac06a750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f69bc006b20 tx=0x7f69bc008040 comp rx=0 tx=0).stop 2023-11-28T13:00:16.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 -- 172.21.15.6:0/1295103478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc15c640 msgr2=0x7f69cc0c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc15c640 0x7f69cc0c7e80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f69c4015040 tx=0x7f69c4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 -- 172.21.15.6:0/1295103478 shutdown_connections 2023-11-28T13:00:16.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f69cc15c640 0x7f69cc0c7e80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f69ac0682a0 0x7f69ac06a750 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f69cc06bf30 0x7f69cc157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 --2- 172.21.15.6:0/1295103478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69cc0d2820 0x7f69cc0c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 -- 172.21.15.6:0/1295103478 >> 172.21.15.6:0/1295103478 conn(0x7f69cc0d2df0 msgr2=0x7f69cc15af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:16.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 -- 172.21.15.6:0/1295103478 shutdown_connections 2023-11-28T13:00:16.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.335+0000 7f69d0a26700 1 -- 172.21.15.6:0/1295103478 wait complete. 2023-11-28T13:00:16.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:16.349 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.475+0000 7f014aae9700 1 Processor -- start 2023-11-28T13:00:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.475+0000 7f014aae9700 1 -- start start 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f014aae9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0144159500 0x7f01440cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f014aae9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f014415dbf0 0x7f01440ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f014aae9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014415e040 con 0x7f01440b1630 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f014aae9700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01440cede0 con 0x7f0144159500 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f014aae9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01440ceff0 con 0x7f014415dbf0 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f0148885700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0144159500 0x7f01440cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f0148885700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0144159500 0x7f01440cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57306/0 (socket says 172.21.15.6:57306) 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f0148885700 1 -- 172.21.15.6:0/3119896270 learned_addr learned my addr 172.21.15.6:0/3119896270 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f0143fff700 1 --2- 172.21.15.6:0/3119896270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f014415dbf0 0x7f01440ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f0148885700 1 -- 172.21.15.6:0/3119896270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f014415dbf0 msgr2=0x7f01440ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f0148885700 1 --2- 172.21.15.6:0/3119896270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f014415dbf0 0x7f01440ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f0148885700 1 -- 172.21.15.6:0/3119896270 >> v1:172.21.15.61:6789/0 conn(0x7f01440b1630 legacy=0x7f014406bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f0148885700 1 -- 172.21.15.6:0/3119896270 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440cf2d0 con 0x7f0144159500 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.476+0000 7f0148885700 1 --2- 172.21.15.6:0/3119896270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0144159500 0x7f01440cd140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0138009920 tx=0x7f0138009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3796cfe0bdf8f65c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f01437fe700 1 -- 172.21.15.6:0/3119896270 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138011830 con 0x7f0144159500 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f01437fe700 1 -- 172.21.15.6:0/3119896270 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0138011990 con 0x7f0144159500 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f01437fe700 1 -- 172.21.15.6:0/3119896270 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138011d20 con 0x7f0144159500 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f014aae9700 1 -- 172.21.15.6:0/3119896270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0144159500 msgr2=0x7f01440cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f014aae9700 1 --2- 172.21.15.6:0/3119896270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0144159500 0x7f01440cd140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0138009920 tx=0x7f0138009c30 comp rx=0 tx=0).stop 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f014aae9700 1 -- 172.21.15.6:0/3119896270 shutdown_connections 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f014aae9700 1 --2- 172.21.15.6:0/3119896270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0144159500 0x7f01440cd140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f014aae9700 1 --2- 172.21.15.6:0/3119896270 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f014415dbf0 0x7f01440ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f014aae9700 1 -- 172.21.15.6:0/3119896270 >> 172.21.15.6:0/3119896270 conn(0x7f01440cc630 msgr2=0x7f01440cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:16.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f014aae9700 1 -- 172.21.15.6:0/3119896270 shutdown_connections 2023-11-28T13:00:16.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.477+0000 7f014aae9700 1 -- 172.21.15.6:0/3119896270 wait complete. 2023-11-28T13:00:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.479+0000 7f014aae9700 1 Processor -- start 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.479+0000 7f014aae9700 1 -- start start 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f014aae9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01440b1630 0x7f0144154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f014aae9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0144159500 0x7f01441579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f014aae9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f014415dbf0 0x7f01441f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f014aae9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0144155460 con 0x7f014415dbf0 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f014aae9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0144155700 con 0x7f0144159500 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f014aae9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01441559e0 con 0x7f01440b1630 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0149086700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f014415dbf0 0x7f01441f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0149086700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f014415dbf0 0x7f01441f4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57316/0 (socket says 172.21.15.6:57316) 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0149086700 1 -- 172.21.15.6:0/2188704874 learned_addr learned my addr 172.21.15.6:0/2188704874 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0148885700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01440b1630 0x7f0144154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0143fff700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0144159500 0x7f01441579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0149086700 1 -- 172.21.15.6:0/2188704874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01440b1630 msgr2=0x7f0144154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0149086700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01440b1630 0x7f0144154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0149086700 1 -- 172.21.15.6:0/2188704874 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0144159500 msgr2=0x7f01441579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0149086700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0144159500 0x7f01441579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0149086700 1 -- 172.21.15.6:0/2188704874 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0138009550 con 0x7f014415dbf0 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.480+0000 7f0148885700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01440b1630 0x7f0144154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.481+0000 7f0149086700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f014415dbf0 0x7f01441f4d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f013c00cb50 tx=0x7f013c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.481+0000 7f01417fa700 1 -- 172.21.15.6:0/2188704874 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c003c00 con 0x7f014415dbf0 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.481+0000 7f01417fa700 1 -- 172.21.15.6:0/2188704874 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c003d60 con 0x7f014415dbf0 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.481+0000 7f014aae9700 1 -- 172.21.15.6:0/2188704874 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014415d3f0 con 0x7f014415dbf0 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.481+0000 7f01417fa700 1 -- 172.21.15.6:0/2188704874 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00ab70 con 0x7f014415dbf0 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.481+0000 7f014aae9700 1 -- 172.21.15.6:0/2188704874 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014415d110 con 0x7f014415dbf0 2023-11-28T13:00:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.482+0000 7f014aae9700 1 -- 172.21.15.6:0/2188704874 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014415d6d0 con 0x7f014415dbf0 2023-11-28T13:00:16.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.483+0000 7f01417fa700 1 -- 172.21.15.6:0/2188704874 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f013c010050 con 0x7f014415dbf0 2023-11-28T13:00:16.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.483+0000 7f01417fa700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01280682a0 0x7f012806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.483+0000 7f0148885700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01280682a0 0x7f012806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.483+0000 7f01417fa700 1 -- 172.21.15.6:0/2188704874 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f013c0857b0 con 0x7f014415dbf0 2023-11-28T13:00:16.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.483+0000 7f0148885700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01280682a0 0x7f012806a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0138014040 tx=0x7f0138008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.487+0000 7f01417fa700 1 -- 172.21.15.6:0/2188704874 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f013c0523c0 con 0x7f014415dbf0 2023-11-28T13:00:16.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.679+0000 7f014aae9700 1 -- 172.21.15.6:0/2188704874 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f01440742e0 con 0x7f014415dbf0 2023-11-28T13:00:16.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.679+0000 7f01417fa700 1 -- 172.21.15.6:0/2188704874 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f013c0559e0 con 0x7f014415dbf0 2023-11-28T13:00:16.681 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:00:16.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 -- 172.21.15.6:0/2188704874 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01280682a0 msgr2=0x7f012806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01280682a0 0x7f012806a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0138014040 tx=0x7f0138008030 comp rx=0 tx=0).stop 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 -- 172.21.15.6:0/2188704874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f014415dbf0 msgr2=0x7f01441f4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f014415dbf0 0x7f01441f4d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f013c00cb50 tx=0x7f013c00cf10 comp rx=0 tx=0).stop 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 -- 172.21.15.6:0/2188704874 shutdown_connections 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f014415dbf0 0x7f01441f4d60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01280682a0 0x7f012806a750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01440b1630 0x7f0144154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 --2- 172.21.15.6:0/2188704874 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0144159500 0x7f01441579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 -- 172.21.15.6:0/2188704874 >> 172.21.15.6:0/2188704874 conn(0x7f01440cc630 msgr2=0x7f014406c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 -- 172.21.15.6:0/2188704874 shutdown_connections 2023-11-28T13:00:16.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.681+0000 7f014aae9700 1 -- 172.21.15.6:0/2188704874 wait complete. 2023-11-28T13:00:16.694 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:16.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.820+0000 7fb6274d6700 1 Processor -- start 2023-11-28T13:00:16.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.820+0000 7fb6274d6700 1 -- start start 2023-11-28T13:00:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb6274d6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb620159500 0x7fb6200cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb6274d6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb6200b1d40 0x7fb6200ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb6274d6700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62015e040 con 0x7fb62015dbf0 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb6274d6700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6200cede0 con 0x7fb6200b1d40 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb6274d6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6200ceff0 con 0x7fb620159500 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb625a73700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb6200b1d40 0x7fb6200ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb625a73700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb6200b1d40 0x7fb6200ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57324/0 (socket says 172.21.15.6:57324) 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb625a73700 1 -- 172.21.15.6:0/98479632 learned_addr learned my addr 172.21.15.6:0/98479632 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb624a71700 1 --1- 172.21.15.6:0/98479632 >> v1:172.21.15.6:6789/0 conn(0x7fb62015dbf0 0x7fb6200b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44640/0 (socket says 172.21.15.6:44640) 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb625272700 1 --2- 172.21.15.6:0/98479632 >> v2:172.21.15.61:3300/0 conn(0x7fb620159500 0x7fb6200cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb625a73700 1 -- 172.21.15.6:0/98479632 >> v2:172.21.15.61:3300/0 conn(0x7fb620159500 msgr2=0x7fb6200cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb625a73700 1 --2- 172.21.15.6:0/98479632 >> v2:172.21.15.61:3300/0 conn(0x7fb620159500 0x7fb6200cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb625a73700 1 -- 172.21.15.6:0/98479632 >> v1:172.21.15.6:6789/0 conn(0x7fb62015dbf0 legacy=0x7fb6200b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb617fff700 1 -- 172.21.15.6:0/98479632 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364831076 0 0) 0x7fb62015e040 con 0x7fb62015dbf0 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.821+0000 7fb625a73700 1 -- 172.21.15.6:0/98479632 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6200cf2d0 con 0x7fb6200b1d40 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb625a73700 1 --2- 172.21.15.6:0/98479632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6200b1d40 0x7fb6200ce8a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb6200d0bc0 tx=0x7fb61800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2538d3f8d748d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb617fff700 1 -- 172.21.15.6:0/98479632 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61800e070 con 0x7fb6200b1d40 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb617fff700 1 -- 172.21.15.6:0/98479632 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61800b910 con 0x7fb6200b1d40 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb617fff700 1 -- 172.21.15.6:0/98479632 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61800bcc0 con 0x7fb6200b1d40 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb6274d6700 1 -- 172.21.15.6:0/98479632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6200b1d40 msgr2=0x7fb6200ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb6274d6700 1 --2- 172.21.15.6:0/98479632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6200b1d40 0x7fb6200ce8a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb6200d0bc0 tx=0x7fb61800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb6274d6700 1 -- 172.21.15.6:0/98479632 shutdown_connections 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb6274d6700 1 --2- 172.21.15.6:0/98479632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6200b1d40 0x7fb6200ce8a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb6274d6700 1 --2- 172.21.15.6:0/98479632 >> v2:172.21.15.61:3300/0 conn(0x7fb620159500 0x7fb6200cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb6274d6700 1 -- 172.21.15.6:0/98479632 >> 172.21.15.6:0/98479632 conn(0x7fb6200cc630 msgr2=0x7fb6200cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:16.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb6274d6700 1 -- 172.21.15.6:0/98479632 shutdown_connections 2023-11-28T13:00:16.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.822+0000 7fb6274d6700 1 -- 172.21.15.6:0/98479632 wait complete. 2023-11-28T13:00:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.824+0000 7fb6274d6700 1 Processor -- start 2023-11-28T13:00:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.824+0000 7fb6274d6700 1 -- start start 2023-11-28T13:00:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb6274d6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6200b1d40 0x7fb620152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb6274d6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb620159500 0x7fb6200c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb6274d6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb62015dbf0 0x7fb6200c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb6274d6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200c9460 con 0x7fb62015dbf0 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb6274d6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200c8850 con 0x7fb620159500 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb6274d6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200c8ac0 con 0x7fb6200b1d40 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb625a73700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb62015dbf0 0x7fb6200c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb625a73700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb62015dbf0 0x7fb6200c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57330/0 (socket says 172.21.15.6:57330) 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb625a73700 1 -- 172.21.15.6:0/1145417037 learned_addr learned my addr 172.21.15.6:0/1145417037 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb625272700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6200b1d40 0x7fb620152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb624a71700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb620159500 0x7fb6200c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb625a73700 1 -- 172.21.15.6:0/1145417037 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6200b1d40 msgr2=0x7fb620152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb625a73700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6200b1d40 0x7fb620152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb625a73700 1 -- 172.21.15.6:0/1145417037 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb620159500 msgr2=0x7fb6200c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb625a73700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb620159500 0x7fb6200c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb625a73700 1 -- 172.21.15.6:0/1145417037 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb618009b30 con 0x7fb62015dbf0 2023-11-28T13:00:16.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.825+0000 7fb624a71700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb620159500 0x7fb6200c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:16.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.826+0000 7fb625a73700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb62015dbf0 0x7fb6200c9880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb620066a20 tx=0x7fb618004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.826+0000 7fb615ffb700 1 -- 172.21.15.6:0/1145417037 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618004960 con 0x7fb62015dbf0 2023-11-28T13:00:16.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.826+0000 7fb615ffb700 1 -- 172.21.15.6:0/1145417037 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6180099b0 con 0x7fb62015dbf0 2023-11-28T13:00:16.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.826+0000 7fb6274d6700 1 -- 172.21.15.6:0/1145417037 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62015d3f0 con 0x7fb62015dbf0 2023-11-28T13:00:16.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.826+0000 7fb615ffb700 1 -- 172.21.15.6:0/1145417037 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61801c660 con 0x7fb62015dbf0 2023-11-28T13:00:16.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.826+0000 7fb6274d6700 1 -- 172.21.15.6:0/1145417037 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62015d110 con 0x7fb62015dbf0 2023-11-28T13:00:16.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.827+0000 7fb6274d6700 1 -- 172.21.15.6:0/1145417037 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62015d6d0 con 0x7fb62015dbf0 2023-11-28T13:00:16.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.827+0000 7fb615ffb700 1 -- 172.21.15.6:0/1145417037 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fb618022030 con 0x7fb62015dbf0 2023-11-28T13:00:16.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.828+0000 7fb615ffb700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6040682a0 0x7fb60406a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:16.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.828+0000 7fb615ffb700 1 -- 172.21.15.6:0/1145417037 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5199+0+0 (secure 0 0 0) 0x7fb618087220 con 0x7fb62015dbf0 2023-11-28T13:00:16.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.828+0000 7fb625272700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6040682a0 0x7fb60406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:16.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.828+0000 7fb625272700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6040682a0 0x7fb60406a750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb62015cfd0 tx=0x7fb610008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:16.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:16.832+0000 7fb615ffb700 1 -- 172.21.15.6:0/1145417037 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb618053d40 con 0x7fb62015dbf0 2023-11-28T13:00:17.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.030+0000 7fb6274d6700 1 -- 172.21.15.6:0/1145417037 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6200c95a0 con 0x7fb62015dbf0 2023-11-28T13:00:17.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.030+0000 7fb615ffb700 1 -- 172.21.15.6:0/1145417037 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6200c95a0 con 0x7fb62015dbf0 2023-11-28T13:00:17.031 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:00:17.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 -- 172.21.15.6:0/1145417037 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6040682a0 msgr2=0x7fb60406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6040682a0 0x7fb60406a750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb62015cfd0 tx=0x7fb610008040 comp rx=0 tx=0).stop 2023-11-28T13:00:17.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 -- 172.21.15.6:0/1145417037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb62015dbf0 msgr2=0x7fb6200c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb62015dbf0 0x7fb6200c9880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb620066a20 tx=0x7fb618004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 -- 172.21.15.6:0/1145417037 shutdown_connections 2023-11-28T13:00:17.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb62015dbf0 0x7fb6200c9880 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6040682a0 0x7fb60406a750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6200b1d40 0x7fb620152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 --2- 172.21.15.6:0/1145417037 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb620159500 0x7fb6200c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 -- 172.21.15.6:0/1145417037 >> 172.21.15.6:0/1145417037 conn(0x7fb6200cc630 msgr2=0x7fb620156af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 -- 172.21.15.6:0/1145417037 shutdown_connections 2023-11-28T13:00:17.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.032+0000 7fb6274d6700 1 -- 172.21.15.6:0/1145417037 wait complete. 2023-11-28T13:00:17.046 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:00:17.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.176+0000 7f6f581ee700 1 Processor -- start 2023-11-28T13:00:17.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.176+0000 7f6f581ee700 1 -- start start 2023-11-28T13:00:17.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.177+0000 7f6f581ee700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f500cb0a0 0x7f6f500ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.177+0000 7f6f581ee700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6f5006bb80 0x7f6f5014fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.177+0000 7f6f581ee700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5006c140 con 0x7f6f500cb470 2023-11-28T13:00:17.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.177+0000 7f6f581ee700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f500caad0 con 0x7f6f5006bb80 2023-11-28T13:00:17.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.177+0000 7f6f581ee700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500c9ab0 con 0x7f6f500cb0a0 2023-11-28T13:00:17.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.177+0000 7f6f5678b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6f5006bb80 0x7f6f5014fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.177+0000 7f6f55789700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6f500cb470 0x7f6f5015a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44678/0 (socket says 172.21.15.6:44678) 2023-11-28T13:00:17.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.177+0000 7f6f5678b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6f5006bb80 0x7f6f5014fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57360/0 (socket says 172.21.15.6:57360) 2023-11-28T13:00:17.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.177+0000 7f6f55789700 1 -- 172.21.15.6:0/3476439995 learned_addr learned my addr 172.21.15.6:0/3476439995 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:17.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.178+0000 7f6f55f8a700 1 --2- 172.21.15.6:0/3476439995 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f500cb0a0 0x7f6f500ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.178+0000 7f6f5678b700 1 -- 172.21.15.6:0/3476439995 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f500cb0a0 msgr2=0x7f6f500ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:17.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.178+0000 7f6f5678b700 1 --2- 172.21.15.6:0/3476439995 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f500cb0a0 0x7f6f500ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.178+0000 7f6f5678b700 1 -- 172.21.15.6:0/3476439995 >> v1:172.21.15.6:6789/0 conn(0x7f6f500cb470 legacy=0x7f6f5015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.178+0000 7f6f5678b700 1 -- 172.21.15.6:0/3476439995 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f500c9d90 con 0x7f6f5006bb80 2023-11-28T13:00:17.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.178+0000 7f6f54f88700 1 -- 172.21.15.6:0/3476439995 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999827679 0 0) 0x7f6f5006c140 con 0x7f6f500cb470 2023-11-28T13:00:17.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.178+0000 7f6f5678b700 1 --2- 172.21.15.6:0/3476439995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f5006bb80 0x7f6f5014fea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800bfa0 tx=0x7f6f4800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba67067d1a1ce52c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.178+0000 7f6f54f88700 1 -- 172.21.15.6:0/3476439995 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48010070 con 0x7f6f5006bb80 2023-11-28T13:00:17.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.178+0000 7f6f54f88700 1 -- 172.21.15.6:0/3476439995 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4800b2e0 con 0x7f6f5006bb80 2023-11-28T13:00:17.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.178+0000 7f6f54f88700 1 -- 172.21.15.6:0/3476439995 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800dcf0 con 0x7f6f5006bb80 2023-11-28T13:00:17.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.179+0000 7f6f581ee700 1 -- 172.21.15.6:0/3476439995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f5006bb80 msgr2=0x7f6f5014fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.179+0000 7f6f581ee700 1 --2- 172.21.15.6:0/3476439995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f5006bb80 0x7f6f5014fea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800bfa0 tx=0x7f6f4800d330 comp rx=0 tx=0).stop 2023-11-28T13:00:17.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.179+0000 7f6f581ee700 1 -- 172.21.15.6:0/3476439995 shutdown_connections 2023-11-28T13:00:17.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.179+0000 7f6f581ee700 1 --2- 172.21.15.6:0/3476439995 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f5006bb80 0x7f6f5014fea0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.179+0000 7f6f581ee700 1 --2- 172.21.15.6:0/3476439995 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f500cb0a0 0x7f6f500ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.179+0000 7f6f581ee700 1 -- 172.21.15.6:0/3476439995 >> 172.21.15.6:0/3476439995 conn(0x7f6f500b0dd0 msgr2=0x7f6f500b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.179+0000 7f6f581ee700 1 -- 172.21.15.6:0/3476439995 shutdown_connections 2023-11-28T13:00:17.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.179+0000 7f6f581ee700 1 -- 172.21.15.6:0/3476439995 wait complete. 2023-11-28T13:00:17.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.181+0000 7f6f581ee700 1 Processor -- start 2023-11-28T13:00:17.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.181+0000 7f6f581ee700 1 -- start start 2023-11-28T13:00:17.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.182+0000 7f6f581ee700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5006bb80 0x7f6f501ed160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.182+0000 7f6f581ee700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f500cb0a0 0x7f6f5015d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.182+0000 7f6f581ee700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500cb470 0x7f6f5015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.182+0000 7f6f581ee700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500b2110 con 0x7f6f500cb470 2023-11-28T13:00:17.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.182+0000 7f6f581ee700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5014e240 con 0x7f6f500cb0a0 2023-11-28T13:00:17.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.182+0000 7f6f581ee700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f501e3890 con 0x7f6f5006bb80 2023-11-28T13:00:17.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.182+0000 7f6f5678b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500cb470 0x7f6f5015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.182+0000 7f6f5678b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500cb470 0x7f6f5015a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57366/0 (socket says 172.21.15.6:57366) 2023-11-28T13:00:17.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.182+0000 7f6f5678b700 1 -- 172.21.15.6:0/3202442319 learned_addr learned my addr 172.21.15.6:0/3202442319 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:17.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f55f8a700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5006bb80 0x7f6f501ed160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f55789700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f500cb0a0 0x7f6f5015d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f5678b700 1 -- 172.21.15.6:0/3202442319 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5006bb80 msgr2=0x7f6f501ed160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f5678b700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5006bb80 0x7f6f501ed160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f5678b700 1 -- 172.21.15.6:0/3202442319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f500cb0a0 msgr2=0x7f6f5015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f5678b700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f500cb0a0 0x7f6f5015d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f5678b700 1 -- 172.21.15.6:0/3202442319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4800bc80 con 0x7f6f500cb470 2023-11-28T13:00:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f5678b700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500cb470 0x7f6f5015a170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6f5006d2f0 tx=0x7f6f48004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f467fc700 1 -- 172.21.15.6:0/3202442319 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800dea0 con 0x7f6f500cb470 2023-11-28T13:00:17.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f467fc700 1 -- 172.21.15.6:0/3202442319 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4800b450 con 0x7f6f500cb470 2023-11-28T13:00:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f581ee700 1 -- 172.21.15.6:0/3202442319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f501503e0 con 0x7f6f500cb470 2023-11-28T13:00:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f467fc700 1 -- 172.21.15.6:0/3202442319 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48009780 con 0x7f6f500cb470 2023-11-28T13:00:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.183+0000 7f6f581ee700 1 -- 172.21.15.6:0/3202442319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f500ca070 con 0x7f6f500cb470 2023-11-28T13:00:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.184+0000 7f6f581ee700 1 -- 172.21.15.6:0/3202442319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f50150570 con 0x7f6f500cb470 2023-11-28T13:00:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.185+0000 7f6f467fc700 1 -- 172.21.15.6:0/3202442319 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f6f4802e070 con 0x7f6f500cb470 2023-11-28T13:00:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.185+0000 7f6f467fc700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f340682a0 0x7f6f3406a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.185+0000 7f6f467fc700 1 -- 172.21.15.6:0/3202442319 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6f48089510 con 0x7f6f500cb470 2023-11-28T13:00:17.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.185+0000 7f6f55f8a700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f340682a0 0x7f6f3406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.185+0000 7f6f55f8a700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f340682a0 0x7f6f3406a750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009950 tx=0x7f6f4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.189+0000 7f6f467fc700 1 -- 172.21.15.6:0/3202442319 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6f480560b0 con 0x7f6f500cb470 2023-11-28T13:00:17.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.435+0000 7f6f581ee700 1 -- 172.21.15.6:0/3202442319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f501e3a50 con 0x7f6f500cb470 2023-11-28T13:00:17.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.435+0000 7f6f467fc700 1 -- 172.21.15.6:0/3202442319 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3518 (secure 0 0 0) 0x7f6f480596d0 con 0x7f6f500cb470 2023-11-28T13:00:17.437 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:17.437 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6834/1568686201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":1568686201},{"type":"v1","addr":"172.21.15.61:6834","nonce":1568686201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/3333487635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6833","nonce":3333487635},{"type":"v1","addr":"172.21.15.61:6835","nonce":3333487635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3917075730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3917075730},{"type":"v1","addr":"172.21.15.6:6835","nonce":3917075730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/1668558371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1668558371},{"type":"v1","addr":"172.21.15.6:6837","nonce":1668558371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-28T13:00:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 -- 172.21.15.6:0/3202442319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f340682a0 msgr2=0x7f6f3406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f340682a0 0x7f6f3406a750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009950 tx=0x7f6f4c008040 comp rx=0 tx=0).stop 2023-11-28T13:00:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 -- 172.21.15.6:0/3202442319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500cb470 msgr2=0x7f6f5015a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500cb470 0x7f6f5015a170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6f5006d2f0 tx=0x7f6f48004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 -- 172.21.15.6:0/3202442319 shutdown_connections 2023-11-28T13:00:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f500cb470 0x7f6f5015a170 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f340682a0 0x7f6f3406a750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5006bb80 0x7f6f501ed160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 --2- 172.21.15.6:0/3202442319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f500cb0a0 0x7f6f5015d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 -- 172.21.15.6:0/3202442319 >> 172.21.15.6:0/3202442319 conn(0x7f6f500b0dd0 msgr2=0x7f6f5006aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 -- 172.21.15.6:0/3202442319 shutdown_connections 2023-11-28T13:00:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.437+0000 7f6f581ee700 1 -- 172.21.15.6:0/3202442319 wait complete. 2023-11-28T13:00:17.440 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 3 2023-11-28T13:00:17.451 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:17.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.577+0000 7f4fa0269700 1 Processor -- start 2023-11-28T13:00:17.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.577+0000 7f4fa0269700 1 -- start start 2023-11-28T13:00:17.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.578+0000 7f4fa0269700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 0x7f4f9806aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.578+0000 7f4fa0269700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4f980ce150 0x7f4f980ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.578+0000 7f4fa0269700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4f980ceae0 0x7f4f980ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.578+0000 7f4fa0269700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9806ed80 con 0x7f4f980ce150 2023-11-28T13:00:17.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.578+0000 7f4fa0269700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9806f060 con 0x7f4f980ceae0 2023-11-28T13:00:17.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.578+0000 7f4fa0269700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980cd950 con 0x7f4f98158a30 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9e005700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 0x7f4f9806aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9e806700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4f980ceae0 0x7f4f980ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9e005700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 0x7f4f9806aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47082/0 (socket says 172.21.15.6:47082) 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9e005700 1 -- 172.21.15.6:0/461966188 learned_addr learned my addr 172.21.15.6:0/461966188 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9d804700 1 --2- 172.21.15.6:0/461966188 >> v2:172.21.15.6:3300/0 conn(0x7f4f980ce150 0x7f4f980ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9e005700 1 -- 172.21.15.6:0/461966188 >> v2:172.21.15.61:3300/0 conn(0x7f4f980ceae0 msgr2=0x7f4f980ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9e005700 1 --2- 172.21.15.6:0/461966188 >> v2:172.21.15.61:3300/0 conn(0x7f4f980ceae0 0x7f4f980ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9e005700 1 -- 172.21.15.6:0/461966188 >> v2:172.21.15.6:3300/0 conn(0x7f4f980ce150 msgr2=0x7f4f980ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9e005700 1 --2- 172.21.15.6:0/461966188 >> v2:172.21.15.6:3300/0 conn(0x7f4f980ce150 0x7f4f980ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9e005700 1 -- 172.21.15.6:0/461966188 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980cdc30 con 0x7f4f98158a30 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9d804700 1 --2- 172.21.15.6:0/461966188 >> v2:172.21.15.6:3300/0 conn(0x7f4f980ce150 0x7f4f980ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:17.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9e005700 1 --2- 172.21.15.6:0/461966188 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 0x7f4f9806aa70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4f980c6330 tx=0x7f4f8c00be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ed0c534dd76587 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9d003700 1 -- 172.21.15.6:0/461966188 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00db60 con 0x7f4f98158a30 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9d003700 1 -- 172.21.15.6:0/461966188 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c00dcc0 con 0x7f4f98158a30 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.579+0000 7f4f9d003700 1 -- 172.21.15.6:0/461966188 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c017630 con 0x7f4f98158a30 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.580+0000 7f4fa0269700 1 -- 172.21.15.6:0/461966188 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 msgr2=0x7f4f9806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.580+0000 7f4fa0269700 1 --2- 172.21.15.6:0/461966188 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 0x7f4f9806aa70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4f980c6330 tx=0x7f4f8c00be70 comp rx=0 tx=0).stop 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.580+0000 7f4fa0269700 1 -- 172.21.15.6:0/461966188 shutdown_connections 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.580+0000 7f4fa0269700 1 --2- 172.21.15.6:0/461966188 >> v2:172.21.15.61:3300/0 conn(0x7f4f980ceae0 0x7f4f980ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.580+0000 7f4fa0269700 1 --2- 172.21.15.6:0/461966188 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 0x7f4f9806aa70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.580+0000 7f4fa0269700 1 --2- 172.21.15.6:0/461966188 >> v2:172.21.15.6:3300/0 conn(0x7f4f980ce150 0x7f4f980ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.580+0000 7f4fa0269700 1 -- 172.21.15.6:0/461966188 >> 172.21.15.6:0/461966188 conn(0x7f4f980b0dd0 msgr2=0x7f4f980b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.580+0000 7f4fa0269700 1 -- 172.21.15.6:0/461966188 shutdown_connections 2023-11-28T13:00:17.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.580+0000 7f4fa0269700 1 -- 172.21.15.6:0/461966188 wait complete. 2023-11-28T13:00:17.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.581+0000 7f4fa0269700 1 Processor -- start 2023-11-28T13:00:17.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.581+0000 7f4fa0269700 1 -- start start 2023-11-28T13:00:17.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4fa0269700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f980ce150 0x7f4f981edc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4fa0269700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f980ceae0 0x7f4f98154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4fa0269700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 0x7f4f981e2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4fa0269700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98155b50 con 0x7f4f980ce150 2023-11-28T13:00:17.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4fa0269700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98159580 con 0x7f4f980ceae0 2023-11-28T13:00:17.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4fa0269700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980b23f0 con 0x7f4f98158a30 2023-11-28T13:00:17.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4f9e005700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f980ce150 0x7f4f981edc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4f9e005700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f980ce150 0x7f4f981edc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57384/0 (socket says 172.21.15.6:57384) 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4f9e005700 1 -- 172.21.15.6:0/4076248648 learned_addr learned my addr 172.21.15.6:0/4076248648 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4f9e806700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 0x7f4f981e2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4f9d804700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f980ceae0 0x7f4f98154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4f9e005700 1 -- 172.21.15.6:0/4076248648 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 msgr2=0x7f4f981e2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4f9e005700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 0x7f4f981e2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4f9e005700 1 -- 172.21.15.6:0/4076248648 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f980ceae0 msgr2=0x7f4f98154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4f9e005700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f980ceae0 0x7f4f98154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.582+0000 7f4f9e005700 1 -- 172.21.15.6:0/4076248648 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f940094a0 con 0x7f4f980ce150 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.583+0000 7f4f9e005700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f980ce150 0x7f4f981edc00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f980cda90 tx=0x7f4f8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.583+0000 7f4f8affd700 1 -- 172.21.15.6:0/4076248648 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c003ab0 con 0x7f4f980ce150 2023-11-28T13:00:17.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.583+0000 7f4f8affd700 1 -- 172.21.15.6:0/4076248648 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c003c30 con 0x7f4f980ce150 2023-11-28T13:00:17.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.583+0000 7f4fa0269700 1 -- 172.21.15.6:0/4076248648 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f8c00b7f0 con 0x7f4f980ce150 2023-11-28T13:00:17.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.583+0000 7f4f8affd700 1 -- 172.21.15.6:0/4076248648 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c020440 con 0x7f4f980ce150 2023-11-28T13:00:17.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.583+0000 7f4fa0269700 1 -- 172.21.15.6:0/4076248648 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9806e9c0 con 0x7f4f980ce150 2023-11-28T13:00:17.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.584+0000 7f4fa0269700 1 -- 172.21.15.6:0/4076248648 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9806f340 con 0x7f4f980ce150 2023-11-28T13:00:17.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.585+0000 7f4f8affd700 1 -- 172.21.15.6:0/4076248648 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f4f8c0205c0 con 0x7f4f980ce150 2023-11-28T13:00:17.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.585+0000 7f4f8affd700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f7c0682a0 0x7f4f7c06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.585+0000 7f4f8affd700 1 -- 172.21.15.6:0/4076248648 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4f8c089ac0 con 0x7f4f980ce150 2023-11-28T13:00:17.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.585+0000 7f4f9d804700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f7c0682a0 0x7f4f7c06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.586+0000 7f4f9d804700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f7c0682a0 0x7f4f7c06a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f981e2f50 tx=0x7f4f94009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.589+0000 7f4f8affd700 1 -- 172.21.15.6:0/4076248648 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4f8c0566c0 con 0x7f4f980ce150 2023-11-28T13:00:17.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.786+0000 7f4fa0269700 1 -- 172.21.15.6:0/4076248648 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f9806eb50 con 0x7f4f980ce150 2023-11-28T13:00:17.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.787+0000 7f4f8affd700 1 -- 172.21.15.6:0/4076248648 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14124 (secure 0 0 0) 0x7f4f8c025020 con 0x7f4f980ce150 2023-11-28T13:00:17.788 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:17.788 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:16.945787+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:17.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 -- 172.21.15.6:0/4076248648 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f7c0682a0 msgr2=0x7f4f7c06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f7c0682a0 0x7f4f7c06a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f981e2f50 tx=0x7f4f94009210 comp rx=0 tx=0).stop 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 -- 172.21.15.6:0/4076248648 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f980ce150 msgr2=0x7f4f981edc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f980ce150 0x7f4f981edc00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4f980cda90 tx=0x7f4f8c004750 comp rx=0 tx=0).stop 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 -- 172.21.15.6:0/4076248648 shutdown_connections 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f7c0682a0 0x7f4f7c06a750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f98158a30 0x7f4f981e2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f980ce150 0x7f4f981edc00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 --2- 172.21.15.6:0/4076248648 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f980ceae0 0x7f4f98154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 -- 172.21.15.6:0/4076248648 >> 172.21.15.6:0/4076248648 conn(0x7f4f980b0dd0 msgr2=0x7f4f980b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 -- 172.21.15.6:0/4076248648 shutdown_connections 2023-11-28T13:00:17.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.789+0000 7f4fa0269700 1 -- 172.21.15.6:0/4076248648 wait complete. 2023-11-28T13:00:17.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:00:17.803 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:17.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.930+0000 7f99110b9700 1 Processor -- start 2023-11-28T13:00:17.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.930+0000 7f99110b9700 1 -- start start 2023-11-28T13:00:17.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f99110b9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f990c0b1630 0x7f990c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f99110b9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f990c06afc0 0x7f990c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f99110b9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990c0ce550 con 0x7f990c0b1a00 2023-11-28T13:00:17.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f99110b9700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990c0ce830 con 0x7f990c06afc0 2023-11-28T13:00:17.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f99110b9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c0ceb10 con 0x7f990c0b1630 2023-11-28T13:00:17.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f990b59e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f990c06afc0 0x7f990c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f990b59e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f990c06afc0 0x7f990c06b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57402/0 (socket says 172.21.15.6:57402) 2023-11-28T13:00:17.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f990b59e700 1 -- 172.21.15.6:0/278370135 learned_addr learned my addr 172.21.15.6:0/278370135 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:17.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f990ad9d700 1 --2- 172.21.15.6:0/278370135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f990c0b1630 0x7f990c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f990b59e700 1 -- 172.21.15.6:0/278370135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f990c0b1630 msgr2=0x7f990c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:17.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f990b59e700 1 --2- 172.21.15.6:0/278370135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f990c0b1630 0x7f990c0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f990b59e700 1 -- 172.21.15.6:0/278370135 >> v1:172.21.15.61:6789/0 conn(0x7f990c0b1a00 legacy=0x7f990c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f990b59e700 1 -- 172.21.15.6:0/278370135 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c0cedf0 con 0x7f990c06afc0 2023-11-28T13:00:17.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f990b59e700 1 --2- 172.21.15.6:0/278370135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 0x7f990c06b440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9904009e50 tx=0x7f990400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bb9118977a443c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f9909d9b700 1 -- 172.21.15.6:0/278370135 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400e070 con 0x7f990c06afc0 2023-11-28T13:00:17.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f9909d9b700 1 -- 172.21.15.6:0/278370135 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990400b910 con 0x7f990c06afc0 2023-11-28T13:00:17.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.931+0000 7f9909d9b700 1 -- 172.21.15.6:0/278370135 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400bcc0 con 0x7f990c06afc0 2023-11-28T13:00:17.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.932+0000 7f99110b9700 1 -- 172.21.15.6:0/278370135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 msgr2=0x7f990c06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.932+0000 7f99110b9700 1 --2- 172.21.15.6:0/278370135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 0x7f990c06b440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9904009e50 tx=0x7f990400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:17.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.932+0000 7f99110b9700 1 -- 172.21.15.6:0/278370135 shutdown_connections 2023-11-28T13:00:17.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.932+0000 7f99110b9700 1 --2- 172.21.15.6:0/278370135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 0x7f990c06b440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.932+0000 7f99110b9700 1 --2- 172.21.15.6:0/278370135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f990c0b1630 0x7f990c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.932+0000 7f99110b9700 1 -- 172.21.15.6:0/278370135 >> 172.21.15.6:0/278370135 conn(0x7f990c0cc630 msgr2=0x7f990c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:17.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.932+0000 7f99110b9700 1 -- 172.21.15.6:0/278370135 shutdown_connections 2023-11-28T13:00:17.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.932+0000 7f99110b9700 1 -- 172.21.15.6:0/278370135 wait complete. 2023-11-28T13:00:17.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.934+0000 7f99110b9700 1 Processor -- start 2023-11-28T13:00:17.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.934+0000 7f99110b9700 1 -- start start 2023-11-28T13:00:17.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f99110b9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 0x7f990c157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f99110b9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f990c0b1630 0x7f990c0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f99110b9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f990c0b1a00 0x7f990c0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f99110b9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c0c9940 con 0x7f990c06afc0 2023-11-28T13:00:17.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f99110b9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c0c5e90 con 0x7f990c0b1630 2023-11-28T13:00:17.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f99110b9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c0c5280 con 0x7f990c0b1a00 2023-11-28T13:00:17.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 0x7f990c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 0x7f990c157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57404/0 (socket says 172.21.15.6:57404) 2023-11-28T13:00:17.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990ad9d700 1 -- 172.21.15.6:0/4273700078 learned_addr learned my addr 172.21.15.6:0/4273700078 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:17.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990a59c700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f990c0b1630 0x7f990c0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990b59e700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f990c0b1a00 0x7f990c0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990ad9d700 1 -- 172.21.15.6:0/4273700078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f990c0b1a00 msgr2=0x7f990c0c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990ad9d700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f990c0b1a00 0x7f990c0c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990ad9d700 1 -- 172.21.15.6:0/4273700078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f990c0b1630 msgr2=0x7f990c0c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:17.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990ad9d700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f990c0b1630 0x7f990c0c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:17.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990ad9d700 1 -- 172.21.15.6:0/4273700078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9904009b30 con 0x7f990c06afc0 2023-11-28T13:00:17.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f990ad9d700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 0x7f990c157370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00e9b0 tx=0x7f98fc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f98f37fe700 1 -- 172.21.15.6:0/4273700078 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc009710 con 0x7f990c06afc0 2023-11-28T13:00:17.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.935+0000 7f98f37fe700 1 -- 172.21.15.6:0/4273700078 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc00ee50 con 0x7f990c06afc0 2023-11-28T13:00:17.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.936+0000 7f99110b9700 1 -- 172.21.15.6:0/4273700078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c0b1e70 con 0x7f990c06afc0 2023-11-28T13:00:17.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.936+0000 7f98f37fe700 1 -- 172.21.15.6:0/4273700078 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc00ca50 con 0x7f990c06afc0 2023-11-28T13:00:17.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.936+0000 7f99110b9700 1 -- 172.21.15.6:0/4273700078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c0cf0d0 con 0x7f990c06afc0 2023-11-28T13:00:17.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.937+0000 7f99110b9700 1 -- 172.21.15.6:0/4273700078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990c0cb5b0 con 0x7f990c06afc0 2023-11-28T13:00:17.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.937+0000 7f98f37fe700 1 -- 172.21.15.6:0/4273700078 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f98fc019030 con 0x7f990c06afc0 2023-11-28T13:00:17.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.937+0000 7f98f37fe700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f98f80682a0 0x7f98f806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:17.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.937+0000 7f98f37fe700 1 -- 172.21.15.6:0/4273700078 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f98fc085840 con 0x7f990c06afc0 2023-11-28T13:00:17.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.938+0000 7f990a59c700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f98f80682a0 0x7f98f806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:17.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.938+0000 7f990a59c700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f98f80682a0 0x7f98f806a750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f990c0b12c0 tx=0x7f9900009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:17.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:17.942+0000 7f98f37fe700 1 -- 172.21.15.6:0/4273700078 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f98fc052350 con 0x7f990c06afc0 2023-11-28T13:00:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.140+0000 7f99110b9700 1 -- 172.21.15.6:0/4273700078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f990c0c6170 con 0x7f990c06afc0 2023-11-28T13:00:18.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.140+0000 7f98f37fe700 1 -- 172.21.15.6:0/4273700078 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f98fc055970 con 0x7f990c06afc0 2023-11-28T13:00:18.141 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 -- 172.21.15.6:0/4273700078 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f98f80682a0 msgr2=0x7f98f806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f98f80682a0 0x7f98f806a750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f990c0b12c0 tx=0x7f9900009040 comp rx=0 tx=0).stop 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 -- 172.21.15.6:0/4273700078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 msgr2=0x7f990c157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 0x7f990c157370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00e9b0 tx=0x7f98fc00ecc0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 -- 172.21.15.6:0/4273700078 shutdown_connections 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f98f80682a0 0x7f98f806a750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f990c0b1a00 0x7f990c0c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f990c06afc0 0x7f990c157370 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 --2- 172.21.15.6:0/4273700078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f990c0b1630 0x7f990c0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 -- 172.21.15.6:0/4273700078 >> 172.21.15.6:0/4273700078 conn(0x7f990c0cc630 msgr2=0x7f990c15b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 -- 172.21.15.6:0/4273700078 shutdown_connections 2023-11-28T13:00:18.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.142+0000 7f99110b9700 1 -- 172.21.15.6:0/4273700078 wait complete. 2023-11-28T13:00:18.156 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:18.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.282+0000 7fab5c2b9700 1 Processor -- start 2023-11-28T13:00:18.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.282+0000 7fab5c2b9700 1 -- start start 2023-11-28T13:00:18.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab5c2b9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1630 0x7fab540cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab5c2b9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fab5406afc0 0x7fab5406b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab5c2b9700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab540ce550 con 0x7fab540b1a00 2023-11-28T13:00:18.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab5c2b9700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab540ce830 con 0x7fab5406afc0 2023-11-28T13:00:18.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab5c2b9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab540ceb10 con 0x7fab540b1630 2023-11-28T13:00:18.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab5a856700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fab5406afc0 0x7fab5406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab59854700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fab540b1a00 0x7fab540caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44740/0 (socket says 172.21.15.6:44740) 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab5a856700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fab5406afc0 0x7fab5406b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57422/0 (socket says 172.21.15.6:57422) 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab59854700 1 -- 172.21.15.6:0/431216957 learned_addr learned my addr 172.21.15.6:0/431216957 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab5a055700 1 --2- 172.21.15.6:0/431216957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1630 0x7fab540cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab59053700 1 -- 172.21.15.6:0/431216957 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75360724 0 0) 0x7fab540ce550 con 0x7fab540b1a00 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.283+0000 7fab59053700 1 -- 172.21.15.6:0/431216957 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab400035b0 con 0x7fab540b1a00 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.284+0000 7fab5a856700 1 -- 172.21.15.6:0/431216957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1630 msgr2=0x7fab540cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.284+0000 7fab5a856700 1 --2- 172.21.15.6:0/431216957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1630 0x7fab540cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.284+0000 7fab5a856700 1 -- 172.21.15.6:0/431216957 >> v1:172.21.15.6:6789/0 conn(0x7fab540b1a00 legacy=0x7fab540caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.284+0000 7fab5a856700 1 -- 172.21.15.6:0/431216957 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab540cedf0 con 0x7fab5406afc0 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.284+0000 7fab5a055700 1 --2- 172.21.15.6:0/431216957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1630 0x7fab540cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.284+0000 7fab5a856700 1 --2- 172.21.15.6:0/431216957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 0x7fab5406b440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009d90 tx=0x7fab4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0917698d9bf0379 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.284+0000 7fab59053700 1 -- 172.21.15.6:0/431216957 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c00e070 con 0x7fab5406afc0 2023-11-28T13:00:18.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.284+0000 7fab59053700 1 -- 172.21.15.6:0/431216957 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c00bb20 con 0x7fab5406afc0 2023-11-28T13:00:18.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.284+0000 7fab59053700 1 -- 172.21.15.6:0/431216957 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c012430 con 0x7fab5406afc0 2023-11-28T13:00:18.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.285+0000 7fab5c2b9700 1 -- 172.21.15.6:0/431216957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 msgr2=0x7fab5406b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.285+0000 7fab5c2b9700 1 --2- 172.21.15.6:0/431216957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 0x7fab5406b440 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009d90 tx=0x7fab4c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:18.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.285+0000 7fab5c2b9700 1 -- 172.21.15.6:0/431216957 shutdown_connections 2023-11-28T13:00:18.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.285+0000 7fab5c2b9700 1 --2- 172.21.15.6:0/431216957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 0x7fab5406b440 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.285+0000 7fab5c2b9700 1 --2- 172.21.15.6:0/431216957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1630 0x7fab540cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.285+0000 7fab5c2b9700 1 -- 172.21.15.6:0/431216957 >> 172.21.15.6:0/431216957 conn(0x7fab540cc630 msgr2=0x7fab540cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:18.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.285+0000 7fab5c2b9700 1 -- 172.21.15.6:0/431216957 shutdown_connections 2023-11-28T13:00:18.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.285+0000 7fab5c2b9700 1 -- 172.21.15.6:0/431216957 wait complete. 2023-11-28T13:00:18.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.286+0000 7fab5c2b9700 1 Processor -- start 2023-11-28T13:00:18.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.286+0000 7fab5c2b9700 1 -- start start 2023-11-28T13:00:18.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab5c2b9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 0x7fab5415b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab5c2b9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab540b1630 0x7fab541f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab5c2b9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1a00 0x7fab540c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab5c2b9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab541f2d30 con 0x7fab5406afc0 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab5c2b9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab541f3010 con 0x7fab540b1630 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab5c2b9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab541f32f0 con 0x7fab540b1a00 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab5a055700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 0x7fab5415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab5a055700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 0x7fab5415b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57424/0 (socket says 172.21.15.6:57424) 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab5a055700 1 -- 172.21.15.6:0/1947934320 learned_addr learned my addr 172.21.15.6:0/1947934320 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab5a856700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1a00 0x7fab540c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.287+0000 7fab59854700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab540b1630 0x7fab541f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab5a055700 1 -- 172.21.15.6:0/1947934320 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1a00 msgr2=0x7fab540c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab5a055700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1a00 0x7fab540c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab5a055700 1 -- 172.21.15.6:0/1947934320 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab540b1630 msgr2=0x7fab541f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab5a055700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab540b1630 0x7fab541f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab5a055700 1 -- 172.21.15.6:0/1947934320 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440094a0 con 0x7fab5406afc0 2023-11-28T13:00:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab5a055700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 0x7fab5415b190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab4400ebe0 tx=0x7fab4400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab4affd700 1 -- 172.21.15.6:0/1947934320 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4400c850 con 0x7fab5406afc0 2023-11-28T13:00:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab4affd700 1 -- 172.21.15.6:0/1947934320 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4400c9b0 con 0x7fab5406afc0 2023-11-28T13:00:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab5c2b9700 1 -- 172.21.15.6:0/1947934320 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4c009a70 con 0x7fab5406afc0 2023-11-28T13:00:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab4affd700 1 -- 172.21.15.6:0/1947934320 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4400cd40 con 0x7fab5406afc0 2023-11-28T13:00:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.288+0000 7fab5c2b9700 1 -- 172.21.15.6:0/1947934320 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab540b1e70 con 0x7fab5406afc0 2023-11-28T13:00:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.290+0000 7fab5c2b9700 1 -- 172.21.15.6:0/1947934320 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab540cf0d0 con 0x7fab5406afc0 2023-11-28T13:00:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.290+0000 7fab4affd700 1 -- 172.21.15.6:0/1947934320 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fab44019030 con 0x7fab5406afc0 2023-11-28T13:00:18.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.291+0000 7fab4affd700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab380682a0 0x7fab3806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.291+0000 7fab4affd700 1 -- 172.21.15.6:0/1947934320 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fab44085880 con 0x7fab5406afc0 2023-11-28T13:00:18.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.291+0000 7fab59854700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab380682a0 0x7fab3806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.291+0000 7fab59854700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab380682a0 0x7fab3806a750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fab540c9bd0 tx=0x7fab50007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.296+0000 7fab4affd700 1 -- 172.21.15.6:0/1947934320 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fab44052480 con 0x7fab5406afc0 2023-11-28T13:00:18.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.488+0000 7fab5c2b9700 1 -- 172.21.15.6:0/1947934320 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab540b4310 con 0x7fab5406afc0 2023-11-28T13:00:18.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.489+0000 7fab4affd700 1 -- 172.21.15.6:0/1947934320 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab4400cea0 con 0x7fab5406afc0 2023-11-28T13:00:18.490 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:00:18.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 -- 172.21.15.6:0/1947934320 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab380682a0 msgr2=0x7fab3806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab380682a0 0x7fab3806a750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fab540c9bd0 tx=0x7fab50007dc0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 -- 172.21.15.6:0/1947934320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 msgr2=0x7fab5415b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 0x7fab5415b190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab4400ebe0 tx=0x7fab4400eef0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 -- 172.21.15.6:0/1947934320 shutdown_connections 2023-11-28T13:00:18.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab380682a0 0x7fab3806a750 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab540b1a00 0x7fab540c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab5406afc0 0x7fab5415b190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 --2- 172.21.15.6:0/1947934320 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab540b1630 0x7fab541f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 -- 172.21.15.6:0/1947934320 >> 172.21.15.6:0/1947934320 conn(0x7fab540cc630 msgr2=0x7fab540caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 -- 172.21.15.6:0/1947934320 shutdown_connections 2023-11-28T13:00:18.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.491+0000 7fab5c2b9700 1 -- 172.21.15.6:0/1947934320 wait complete. 2023-11-28T13:00:18.505 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:00:18.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.627+0000 7f6ab769a700 1 Processor -- start 2023-11-28T13:00:18.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.628+0000 7f6ab769a700 1 -- start start 2023-11-28T13:00:18.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab769a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6ab015c990 0x7f6ab00cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab769a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ab0161090 0x7f6ab00ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab769a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6ab00caeb0 0x7f6ab00cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab769a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab00ce340 con 0x7f6ab015c990 2023-11-28T13:00:18.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab769a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab00cebe0 con 0x7f6ab00caeb0 2023-11-28T13:00:18.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab769a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00cb3f0 con 0x7f6ab0161090 2023-11-28T13:00:18.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab5436700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6ab015c990 0x7f6ab00cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab5436700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6ab015c990 0x7f6ab00cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57432/0 (socket says 172.21.15.6:57432) 2023-11-28T13:00:18.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab5436700 1 -- 172.21.15.6:0/3937155658 learned_addr learned my addr 172.21.15.6:0/3937155658 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:18.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab5c37700 1 --2- 172.21.15.6:0/3937155658 >> v2:172.21.15.61:3300/0 conn(0x7f6ab00caeb0 0x7f6ab00cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab4c35700 1 --2- 172.21.15.6:0/3937155658 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ab0161090 0x7f6ab00ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab5436700 1 -- 172.21.15.6:0/3937155658 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ab0161090 msgr2=0x7f6ab00ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:18.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab5436700 1 --2- 172.21.15.6:0/3937155658 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ab0161090 0x7f6ab00ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab5436700 1 -- 172.21.15.6:0/3937155658 >> v2:172.21.15.61:3300/0 conn(0x7f6ab00caeb0 msgr2=0x7f6ab00cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:18.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab5436700 1 --2- 172.21.15.6:0/3937155658 >> v2:172.21.15.61:3300/0 conn(0x7f6ab00caeb0 0x7f6ab00cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.629+0000 7f6ab5436700 1 -- 172.21.15.6:0/3937155658 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab00ce590 con 0x7f6ab015c990 2023-11-28T13:00:18.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6ab5436700 1 --2- 172.21.15.6:0/3937155658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab015c990 0x7f6ab00cf070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00ca080 tx=0x7f6aac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3429e112827cd1a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6aa7fff700 1 -- 172.21.15.6:0/3937155658 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac011830 con 0x7f6ab015c990 2023-11-28T13:00:18.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6aa7fff700 1 -- 172.21.15.6:0/3937155658 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aac011990 con 0x7f6ab015c990 2023-11-28T13:00:18.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6aa7fff700 1 -- 172.21.15.6:0/3937155658 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac011d20 con 0x7f6ab015c990 2023-11-28T13:00:18.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6ab769a700 1 -- 172.21.15.6:0/3937155658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab015c990 msgr2=0x7f6ab00cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6ab769a700 1 --2- 172.21.15.6:0/3937155658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab015c990 0x7f6ab00cf070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00ca080 tx=0x7f6aac009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6ab769a700 1 -- 172.21.15.6:0/3937155658 shutdown_connections 2023-11-28T13:00:18.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6ab769a700 1 --2- 172.21.15.6:0/3937155658 >> v2:172.21.15.61:3300/0 conn(0x7f6ab00caeb0 0x7f6ab00cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6ab769a700 1 --2- 172.21.15.6:0/3937155658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab015c990 0x7f6ab00cf070 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6ab769a700 1 --2- 172.21.15.6:0/3937155658 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ab0161090 0x7f6ab00ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.630+0000 7f6ab769a700 1 -- 172.21.15.6:0/3937155658 >> 172.21.15.6:0/3937155658 conn(0x7f6ab00b1650 msgr2=0x7f6ab00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:18.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.631+0000 7f6ab769a700 1 -- 172.21.15.6:0/3937155658 shutdown_connections 2023-11-28T13:00:18.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.631+0000 7f6ab769a700 1 -- 172.21.15.6:0/3937155658 wait complete. 2023-11-28T13:00:18.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.632+0000 7f6ab769a700 1 Processor -- start 2023-11-28T13:00:18.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.632+0000 7f6ab769a700 1 -- start start 2023-11-28T13:00:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.633+0000 7f6ab769a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ab00caeb0 0x7f6ab00ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.633+0000 7f6ab769a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ab015c990 0x7f6ab00c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.633+0000 7f6ab769a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab0161090 0x7f6ab00c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.633+0000 7f6ab769a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00c43a0 con 0x7f6ab0161090 2023-11-28T13:00:18.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.633+0000 7f6ab769a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab014b1d0 con 0x7f6ab015c990 2023-11-28T13:00:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.633+0000 7f6ab769a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00c9a30 con 0x7f6ab00caeb0 2023-11-28T13:00:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.633+0000 7f6ab5c37700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab0161090 0x7f6ab00c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.633+0000 7f6ab5c37700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab0161090 0x7f6ab00c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57442/0 (socket says 172.21.15.6:57442) 2023-11-28T13:00:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.633+0000 7f6ab5c37700 1 -- 172.21.15.6:0/345762516 learned_addr learned my addr 172.21.15.6:0/345762516 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6ab4c35700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ab015c990 0x7f6ab00c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6ab5436700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ab00caeb0 0x7f6ab00ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6ab5c37700 1 -- 172.21.15.6:0/345762516 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ab00caeb0 msgr2=0x7f6ab00ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6ab5c37700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ab00caeb0 0x7f6ab00ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6ab5c37700 1 -- 172.21.15.6:0/345762516 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ab015c990 msgr2=0x7f6ab00c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6ab5c37700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ab015c990 0x7f6ab00c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6ab5c37700 1 -- 172.21.15.6:0/345762516 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac009550 con 0x7f6ab0161090 2023-11-28T13:00:18.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6ab5c37700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab0161090 0x7f6ab00c5660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800eed0 tx=0x7f6aa800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6aa5ffb700 1 -- 172.21.15.6:0/345762516 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8010040 con 0x7f6ab0161090 2023-11-28T13:00:18.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6aa5ffb700 1 -- 172.21.15.6:0/345762516 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa800bd30 con 0x7f6ab0161090 2023-11-28T13:00:18.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6ab769a700 1 -- 172.21.15.6:0/345762516 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab00cf5b0 con 0x7f6ab0161090 2023-11-28T13:00:18.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6aa5ffb700 1 -- 172.21.15.6:0/345762516 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa800bea0 con 0x7f6ab0161090 2023-11-28T13:00:18.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.634+0000 7f6ab769a700 1 -- 172.21.15.6:0/345762516 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab00ce870 con 0x7f6ab0161090 2023-11-28T13:00:18.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.635+0000 7f6ab769a700 1 -- 172.21.15.6:0/345762516 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab00cf740 con 0x7f6ab0161090 2023-11-28T13:00:18.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.636+0000 7f6aa5ffb700 1 -- 172.21.15.6:0/345762516 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f6aa8014620 con 0x7f6ab0161090 2023-11-28T13:00:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.636+0000 7f6aa5ffb700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a940682a0 0x7f6a9406a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.636+0000 7f6aa5ffb700 1 -- 172.21.15.6:0/345762516 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6aa8085ea0 con 0x7f6ab0161090 2023-11-28T13:00:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.636+0000 7f6ab5436700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a940682a0 0x7f6a9406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.637+0000 7f6ab5436700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a940682a0 0x7f6a9406a750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009c10 tx=0x7f6aac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.640+0000 7f6aa5ffb700 1 -- 172.21.15.6:0/345762516 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6aa8052a40 con 0x7f6ab0161090 2023-11-28T13:00:18.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.838+0000 7f6ab769a700 1 -- 172.21.15.6:0/345762516 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6ab00675a0 con 0x7f6ab0161090 2023-11-28T13:00:18.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.838+0000 7f6aa5ffb700 1 -- 172.21.15.6:0/345762516 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f6aa8018090 con 0x7f6ab0161090 2023-11-28T13:00:18.839 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:18.840 INFO:teuthology.orchestra.run.smithi006.stdout:[ 2023-11-28T13:00:18.840 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:00:18.840 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 1, 2023-11-28T13:00:18.840 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": "rbd" 2023-11-28T13:00:18.840 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:18.840 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:00:18.840 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 2, 2023-11-28T13:00:18.840 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": ".mgr" 2023-11-28T13:00:18.840 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:00:18.840 INFO:teuthology.orchestra.run.smithi006.stdout:] 2023-11-28T13:00:18.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 -- 172.21.15.6:0/345762516 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a940682a0 msgr2=0x7f6a9406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a940682a0 0x7f6a9406a750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009c10 tx=0x7f6aac008030 comp rx=0 tx=0).stop 2023-11-28T13:00:18.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 -- 172.21.15.6:0/345762516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab0161090 msgr2=0x7f6ab00c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab0161090 0x7f6ab00c5660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800eed0 tx=0x7f6aa800b390 comp rx=0 tx=0).stop 2023-11-28T13:00:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 -- 172.21.15.6:0/345762516 shutdown_connections 2023-11-28T13:00:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ab0161090 0x7f6ab00c5660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a940682a0 0x7f6a9406a750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ab00caeb0 0x7f6ab00ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 --2- 172.21.15.6:0/345762516 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ab015c990 0x7f6ab00c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 -- 172.21.15.6:0/345762516 >> 172.21.15.6:0/345762516 conn(0x7f6ab00b1650 msgr2=0x7f6ab00b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 -- 172.21.15.6:0/345762516 shutdown_connections 2023-11-28T13:00:18.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.840+0000 7f6ab769a700 1 -- 172.21.15.6:0/345762516 wait complete. 2023-11-28T13:00:18.853 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:00:18.853 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:00:18.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.980+0000 7fea7f9ab700 1 Processor -- start 2023-11-28T13:00:18.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.980+0000 7fea7f9ab700 1 -- start start 2023-11-28T13:00:18.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7f9ab700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fea78159590 0x7fea7806e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7f9ab700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fea7815db70 0x7fea780ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7f9ab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea780ce890 0x7fea780ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7f9ab700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea780cf250 con 0x7fea78159590 2023-11-28T13:00:18.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7f9ab700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7815d120 con 0x7fea7815db70 2023-11-28T13:00:18.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7f9ab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7815d6e0 con 0x7fea780ce890 2023-11-28T13:00:18.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7d747700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fea78159590 0x7fea7806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7d747700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fea78159590 0x7fea7806e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57446/0 (socket says 172.21.15.6:57446) 2023-11-28T13:00:18.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7d747700 1 -- 172.21.15.6:0/1032915227 learned_addr learned my addr 172.21.15.6:0/1032915227 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:18.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7cf46700 1 --2- 172.21.15.6:0/1032915227 >> v2:172.21.15.61:3300/0 conn(0x7fea7815db70 0x7fea780ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.981+0000 7fea7df48700 1 --2- 172.21.15.6:0/1032915227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea780ce890 0x7fea780ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea7d747700 1 -- 172.21.15.6:0/1032915227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea780ce890 msgr2=0x7fea780ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:18.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea7d747700 1 --2- 172.21.15.6:0/1032915227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea780ce890 0x7fea780ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea7d747700 1 -- 172.21.15.6:0/1032915227 >> v2:172.21.15.61:3300/0 conn(0x7fea7815db70 msgr2=0x7fea780ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:18.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea7d747700 1 --2- 172.21.15.6:0/1032915227 >> v2:172.21.15.61:3300/0 conn(0x7fea7815db70 0x7fea780ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea7d747700 1 -- 172.21.15.6:0/1032915227 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780aef60 con 0x7fea78159590 2023-11-28T13:00:18.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea7d747700 1 --2- 172.21.15.6:0/1032915227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea78159590 0x7fea7806e400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea7815d080 tx=0x7fea68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd717a9514a4c69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea6ffff700 1 -- 172.21.15.6:0/1032915227 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68011830 con 0x7fea78159590 2023-11-28T13:00:18.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea6ffff700 1 -- 172.21.15.6:0/1032915227 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea68011990 con 0x7fea78159590 2023-11-28T13:00:18.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea6ffff700 1 -- 172.21.15.6:0/1032915227 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68011d20 con 0x7fea78159590 2023-11-28T13:00:18.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea7f9ab700 1 -- 172.21.15.6:0/1032915227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea78159590 msgr2=0x7fea7806e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.982+0000 7fea7f9ab700 1 --2- 172.21.15.6:0/1032915227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea78159590 0x7fea7806e400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea7815d080 tx=0x7fea68009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.983+0000 7fea7f9ab700 1 -- 172.21.15.6:0/1032915227 shutdown_connections 2023-11-28T13:00:18.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.983+0000 7fea7f9ab700 1 --2- 172.21.15.6:0/1032915227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea780ce890 0x7fea780ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.983+0000 7fea7f9ab700 1 --2- 172.21.15.6:0/1032915227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea78159590 0x7fea7806e400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.983+0000 7fea7f9ab700 1 --2- 172.21.15.6:0/1032915227 >> v2:172.21.15.61:3300/0 conn(0x7fea7815db70 0x7fea780ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.983+0000 7fea7f9ab700 1 -- 172.21.15.6:0/1032915227 >> 172.21.15.6:0/1032915227 conn(0x7fea780b1650 msgr2=0x7fea780b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:18.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.983+0000 7fea7f9ab700 1 -- 172.21.15.6:0/1032915227 shutdown_connections 2023-11-28T13:00:18.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.983+0000 7fea7f9ab700 1 -- 172.21.15.6:0/1032915227 wait complete. 2023-11-28T13:00:18.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.984+0000 7fea7f9ab700 1 Processor -- start 2023-11-28T13:00:18.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.984+0000 7fea7f9ab700 1 -- start start 2023-11-28T13:00:18.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.985+0000 7fea7f9ab700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea780ce890 0x7fea780c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.985+0000 7fea7f9ab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea78159590 0x7fea780c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.985+0000 7fea7f9ab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7815db70 0x7fea781587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.985+0000 7fea7f9ab700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea780b3f10 con 0x7fea7815db70 2023-11-28T13:00:18.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.985+0000 7fea7f9ab700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea780b40f0 con 0x7fea780ce890 2023-11-28T13:00:18.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.985+0000 7fea7f9ab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea780b43d0 con 0x7fea78159590 2023-11-28T13:00:18.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.985+0000 7fea7df48700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7815db70 0x7fea781587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.985+0000 7fea7df48700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7815db70 0x7fea781587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57456/0 (socket says 172.21.15.6:57456) 2023-11-28T13:00:18.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.985+0000 7fea7df48700 1 -- 172.21.15.6:0/733231798 learned_addr learned my addr 172.21.15.6:0/733231798 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:18.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea7d747700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea780ce890 0x7fea780c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea7cf46700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea78159590 0x7fea780c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea7df48700 1 -- 172.21.15.6:0/733231798 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea78159590 msgr2=0x7fea780c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea7df48700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea78159590 0x7fea780c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea7df48700 1 -- 172.21.15.6:0/733231798 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea780ce890 msgr2=0x7fea780c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea7df48700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea780ce890 0x7fea780c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea7df48700 1 -- 172.21.15.6:0/733231798 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea68009550 con 0x7fea7815db70 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea7df48700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7815db70 0x7fea781587c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea7000efc0 tx=0x7fea7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea6dffb700 1 -- 172.21.15.6:0/733231798 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70009ea0 con 0x7fea7815db70 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea6dffb700 1 -- 172.21.15.6:0/733231798 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7000f040 con 0x7fea7815db70 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea7f9ab700 1 -- 172.21.15.6:0/733231798 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7815d370 con 0x7fea7815db70 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea6dffb700 1 -- 172.21.15.6:0/733231798 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7000ced0 con 0x7fea7815db70 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.986+0000 7fea7f9ab700 1 -- 172.21.15.6:0/733231798 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea780af240 con 0x7fea7815db70 2023-11-28T13:00:18.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.987+0000 7fea7f9ab700 1 -- 172.21.15.6:0/733231798 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7815e010 con 0x7fea7815db70 2023-11-28T13:00:18.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.988+0000 7fea6dffb700 1 -- 172.21.15.6:0/733231798 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fea70014480 con 0x7fea7815db70 2023-11-28T13:00:18.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.988+0000 7fea6dffb700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fea5c0682a0 0x7fea5c06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:18.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.988+0000 7fea6dffb700 1 -- 172.21.15.6:0/733231798 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fea70086220 con 0x7fea7815db70 2023-11-28T13:00:18.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.988+0000 7fea7d747700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fea5c0682a0 0x7fea5c06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:18.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.988+0000 7fea7d747700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fea5c0682a0 0x7fea5c06a750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea68009840 tx=0x7fea68008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:18.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:18.992+0000 7fea6dffb700 1 -- 172.21.15.6:0/733231798 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fea70052dc0 con 0x7fea7815db70 2023-11-28T13:00:19.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.187+0000 7fea7f9ab700 1 -- 172.21.15.6:0/733231798 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fea780739f0 con 0x7fea7815db70 2023-11-28T13:00:19.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.971+0000 7fea6dffb700 1 -- 172.21.15.6:0/733231798 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea70053190 con 0x7fea7815db70 2023-11-28T13:00:19.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.977+0000 7fea6dffb700 1 -- 172.21.15.6:0/733231798 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fea70052dc0 con 0x7fea7815db70 2023-11-28T13:00:19.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.982+0000 7fea7f9ab700 1 -- 172.21.15.6:0/733231798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fea5c0682a0 msgr2=0x7fea5c06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:19.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.982+0000 7fea7f9ab700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fea5c0682a0 0x7fea5c06a750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fea68009840 tx=0x7fea68008030 comp rx=0 tx=0).stop 2023-11-28T13:00:19.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.982+0000 7fea7f9ab700 1 -- 172.21.15.6:0/733231798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7815db70 msgr2=0x7fea781587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:19.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.982+0000 7fea7f9ab700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7815db70 0x7fea781587c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea7000efc0 tx=0x7fea7000c3a0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.983+0000 7fea7f9ab700 1 -- 172.21.15.6:0/733231798 shutdown_connections 2023-11-28T13:00:19.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.983+0000 7fea7f9ab700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea7815db70 0x7fea781587c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.983+0000 7fea7f9ab700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea780ce890 0x7fea780c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.983+0000 7fea7f9ab700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fea5c0682a0 0x7fea5c06a750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.983+0000 7fea7f9ab700 1 --2- 172.21.15.6:0/733231798 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea78159590 0x7fea780c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:19.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.983+0000 7fea7f9ab700 1 -- 172.21.15.6:0/733231798 >> 172.21.15.6:0/733231798 conn(0x7fea780b1650 msgr2=0x7fea7806d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:19.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.984+0000 7fea7f9ab700 1 -- 172.21.15.6:0/733231798 shutdown_connections 2023-11-28T13:00:19.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:19.984+0000 7fea7f9ab700 1 -- 172.21.15.6:0/733231798 wait complete. 2023-11-28T13:00:19.986 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' created 2023-11-28T13:00:20.003 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-28T13:00:20.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.140+0000 7fb5cd1b8700 1 Processor -- start 2023-11-28T13:00:20.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.140+0000 7fb5cd1b8700 1 -- start start 2023-11-28T13:00:20.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5cd1b8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb5c81592b0 0x7fb5c80cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5cd1b8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5c80cea90 0x7fb5c80cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5cd1b8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c80ce0a0 con 0x7fb5c815a4a0 2023-11-28T13:00:20.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5cd1b8700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c81599c0 con 0x7fb5c81592b0 2023-11-28T13:00:20.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5cd1b8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8159ca0 con 0x7fb5c80cea90 2023-11-28T13:00:20.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c6d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb5c81592b0 0x7fb5c80cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c6d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb5c81592b0 0x7fb5c80cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57460/0 (socket says 172.21.15.6:57460) 2023-11-28T13:00:20.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c6d9d700 1 -- 172.21.15.6:0/2881037966 learned_addr learned my addr 172.21.15.6:0/2881037966 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:20.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c759e700 1 --2- 172.21.15.6:0/2881037966 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5c80cea90 0x7fb5c80cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c6d9d700 1 -- 172.21.15.6:0/2881037966 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5c80cea90 msgr2=0x7fb5c80cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:20.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c6d9d700 1 --2- 172.21.15.6:0/2881037966 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5c80cea90 0x7fb5c80cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c6d9d700 1 -- 172.21.15.6:0/2881037966 >> v1:172.21.15.61:6789/0 conn(0x7fb5c815a4a0 legacy=0x7fb5c80ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c6d9d700 1 -- 172.21.15.6:0/2881037966 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c8159f80 con 0x7fb5c81592b0 2023-11-28T13:00:20.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c6d9d700 1 --2- 172.21.15.6:0/2881037966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c81592b0 0x7fb5c80cd530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800bb90 tx=0x7fb5b800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab8c1338445f2eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:20.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c5d9b700 1 -- 172.21.15.6:0/2881037966 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800dd30 con 0x7fb5c81592b0 2023-11-28T13:00:20.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c5d9b700 1 -- 172.21.15.6:0/2881037966 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b800de90 con 0x7fb5c81592b0 2023-11-28T13:00:20.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.141+0000 7fb5c5d9b700 1 -- 172.21.15.6:0/2881037966 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8010690 con 0x7fb5c81592b0 2023-11-28T13:00:20.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.142+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/2881037966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c81592b0 msgr2=0x7fb5c80cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.142+0000 7fb5cd1b8700 1 --2- 172.21.15.6:0/2881037966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c81592b0 0x7fb5c80cd530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800bb90 tx=0x7fb5b800bea0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.142+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/2881037966 shutdown_connections 2023-11-28T13:00:20.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.142+0000 7fb5cd1b8700 1 --2- 172.21.15.6:0/2881037966 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5c80cea90 0x7fb5c80cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.142+0000 7fb5cd1b8700 1 --2- 172.21.15.6:0/2881037966 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c81592b0 0x7fb5c80cd530 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.142+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/2881037966 >> 172.21.15.6:0/2881037966 conn(0x7fb5c80cc470 msgr2=0x7fb5c80cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:20.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.142+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/2881037966 shutdown_connections 2023-11-28T13:00:20.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.142+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/2881037966 wait complete. 2023-11-28T13:00:20.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.143+0000 7fb5cd1b8700 1 Processor -- start 2023-11-28T13:00:20.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.144+0000 7fb5cd1b8700 1 -- start start 2023-11-28T13:00:20.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.144+0000 7fb5cd1b8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5c80cea90 0x7fb5c8146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.144+0000 7fb5cd1b8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5c81592b0 0x7fb5c80b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.144+0000 7fb5cd1b8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c815a4a0 0x7fb5c80b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.144+0000 7fb5cd1b8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8147210 con 0x7fb5c815a4a0 2023-11-28T13:00:20.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.144+0000 7fb5cd1b8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80c34f0 con 0x7fb5c80cea90 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.144+0000 7fb5cd1b8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80c37d0 con 0x7fb5c81592b0 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.144+0000 7fb5c759e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c815a4a0 0x7fb5c80b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.144+0000 7fb5c759e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c815a4a0 0x7fb5c80b4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57470/0 (socket says 172.21.15.6:57470) 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.144+0000 7fb5c759e700 1 -- 172.21.15.6:0/4021107280 learned_addr learned my addr 172.21.15.6:0/4021107280 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5c659c700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5c81592b0 0x7fb5c80b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5c6d9d700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5c80cea90 0x7fb5c8146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5c759e700 1 -- 172.21.15.6:0/4021107280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5c81592b0 msgr2=0x7fb5c80b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5c759e700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5c81592b0 0x7fb5c80b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5c759e700 1 -- 172.21.15.6:0/4021107280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5c80cea90 msgr2=0x7fb5c8146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5c759e700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5c80cea90 0x7fb5c8146cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5c759e700 1 -- 172.21.15.6:0/4021107280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b800b7f0 con 0x7fb5c815a4a0 2023-11-28T13:00:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5c759e700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c815a4a0 0x7fb5c80b4500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000efc0 tx=0x7fb5c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5af7fe700 1 -- 172.21.15.6:0/4021107280 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0009ea0 con 0x7fb5c815a4a0 2023-11-28T13:00:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5af7fe700 1 -- 172.21.15.6:0/4021107280 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c000f040 con 0x7fb5c815a4a0 2023-11-28T13:00:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/4021107280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c8159680 con 0x7fb5c815a4a0 2023-11-28T13:00:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5af7fe700 1 -- 172.21.15.6:0/4021107280 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000ced0 con 0x7fb5c815a4a0 2023-11-28T13:00:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.145+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/4021107280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c806dd80 con 0x7fb5c815a4a0 2023-11-28T13:00:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.146+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/4021107280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c80b1470 con 0x7fb5c815a4a0 2023-11-28T13:00:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.147+0000 7fb5af7fe700 1 -- 172.21.15.6:0/4021107280 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb5c0014480 con 0x7fb5c815a4a0 2023-11-28T13:00:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.147+0000 7fb5af7fe700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb5b40682f0 0x7fb5b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:20.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.147+0000 7fb5af7fe700 1 -- 172.21.15.6:0/4021107280 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fb5c00867b0 con 0x7fb5c815a4a0 2023-11-28T13:00:20.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.147+0000 7fb5c6d9d700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb5b40682f0 0x7fb5b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:20.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.147+0000 7fb5c6d9d700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb5b40682f0 0x7fb5b406a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800bee0 tx=0x7fb5b8013000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:20.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.152+0000 7fb5af7fe700 1 -- 172.21.15.6:0/4021107280 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb5c0053100 con 0x7fb5c815a4a0 2023-11-28T13:00:20.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.344+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/4021107280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fb5c80739f0 con 0x7fb5c815a4a0 2023-11-28T13:00:20.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.985+0000 7fb5af7fe700 1 -- 172.21.15.6:0/4021107280 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7fb5c0018020 con 0x7fb5c815a4a0 2023-11-28T13:00:20.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.991+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/4021107280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb5b40682f0 msgr2=0x7fb5b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.991+0000 7fb5cd1b8700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb5b40682f0 0x7fb5b406a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800bee0 tx=0x7fb5b8013000 comp rx=0 tx=0).stop 2023-11-28T13:00:20.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.991+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/4021107280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c815a4a0 msgr2=0x7fb5c80b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:20.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.991+0000 7fb5cd1b8700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c815a4a0 0x7fb5c80b4500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000efc0 tx=0x7fb5c000c3a0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.992+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/4021107280 shutdown_connections 2023-11-28T13:00:20.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.992+0000 7fb5cd1b8700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5c815a4a0 0x7fb5c80b4500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.992+0000 7fb5cd1b8700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5c80cea90 0x7fb5c8146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.992+0000 7fb5cd1b8700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb5b40682f0 0x7fb5b406a7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.992+0000 7fb5cd1b8700 1 --2- 172.21.15.6:0/4021107280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5c81592b0 0x7fb5c80b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:20.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.992+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/4021107280 >> 172.21.15.6:0/4021107280 conn(0x7fb5c80cc470 msgr2=0x7fb5c806c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:20.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.992+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/4021107280 shutdown_connections 2023-11-28T13:00:20.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:20.992+0000 7fb5cd1b8700 1 -- 172.21.15.6:0/4021107280 wait complete. 2023-11-28T13:00:21.000 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' created 2023-11-28T13:00:21.018 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:00:21.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.150+0000 7f6524c6e700 1 Processor -- start 2023-11-28T13:00:21.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.150+0000 7f6524c6e700 1 -- start start 2023-11-28T13:00:21.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f6524c6e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f652015a640 0x7f65200c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f6524c6e700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65200ae730 con 0x7f65200cd140 2023-11-28T13:00:21.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f6524c6e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652006e9e0 con 0x7f652006b900 2023-11-28T13:00:21.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f6524c6e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65200cb210 con 0x7f652015a640 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651dd9b700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f65200cd140 0x7f652006b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44796/0 (socket says 172.21.15.6:44796) 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651dd9b700 1 -- 172.21.15.6:0/3428100494 learned_addr learned my addr 172.21.15.6:0/3428100494 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233217916 0 0) 0x7f65200ae730 con 0x7f65200cd140 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651e59c700 1 --2- 172.21.15.6:0/3428100494 >> v2:172.21.15.61:3300/0 conn(0x7f652015a640 0x7f65200c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f650c0035b0 con 0x7f65200cd140 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1491679185 0 0) 0x7f650c0035b0 con 0x7f65200cd140 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 >> v2:172.21.15.61:3300/0 conn(0x7f652015a640 msgr2=0x7f65200c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 --2- 172.21.15.6:0/3428100494 >> v2:172.21.15.61:3300/0 conn(0x7f652015a640 0x7f65200c3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 >> v1:172.21.15.61:6789/0 conn(0x7f652006b900 legacy=0x7f65200cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200ae980 con 0x7f65200cd140 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f6510001670 con 0x7f65200cd140 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 >> v1:172.21.15.6:6789/0 conn(0x7f65200cd140 legacy=0x7f652006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f650c003a00 0x7f650c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f650c004850 0x7f650c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f650c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65200cb210 con 0x7f650c005200 2023-11-28T13:00:21.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c00c080 con 0x7f650c003a00 2023-11-28T13:00:21.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.151+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650c00c240 con 0x7f650c004850 2023-11-28T13:00:21.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651ed9d700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f650c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651e59c700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f650c003a00 0x7f650c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651dd9b700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f650c004850 0x7f650c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651ed9d700 1 -- 172.21.15.6:0/3428100494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f650c004850 msgr2=0x7f650c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:21.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651ed9d700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f650c004850 0x7f650c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651ed9d700 1 -- 172.21.15.6:0/3428100494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f650c003a00 msgr2=0x7f650c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651ed9d700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f650c003a00 0x7f650c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651ed9d700 1 -- 172.21.15.6:0/3428100494 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c0035b0 con 0x7f650c005200 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651ed9d700 1 -- 172.21.15.6:0/3428100494 reap_dead start 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651ed9d700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f650c00ba10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f651800c160 tx=0x7f650c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c036292d47f519 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518001db0 con 0x7f650c005200 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651800ccc0 con 0x7f650c005200 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.152+0000 7f651d59a700 1 -- 172.21.15.6:0/3428100494 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518003c40 con 0x7f650c005200 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.153+0000 7f6524c6e700 1 -- 172.21.15.6:0/3428100494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 msgr2=0x7f650c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.153+0000 7f6524c6e700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f650c00ba10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f651800c160 tx=0x7f650c005880 comp rx=0 tx=0).stop 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.153+0000 7f6524c6e700 1 -- 172.21.15.6:0/3428100494 shutdown_connections 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.153+0000 7f6524c6e700 1 --2- 172.21.15.6:0/3428100494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f650c00ba10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.153+0000 7f6524c6e700 1 -- 172.21.15.6:0/3428100494 >> 172.21.15.6:0/3428100494 conn(0x7f65200cc630 msgr2=0x7f65200cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:21.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.153+0000 7f6524c6e700 1 -- 172.21.15.6:0/3428100494 shutdown_connections 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.153+0000 7f6524c6e700 1 -- 172.21.15.6:0/3428100494 wait complete. 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.154+0000 7f6524c6e700 1 Processor -- start 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.154+0000 7f6524c6e700 1 -- start start 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f6524c6e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f65201f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f6524c6e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f652015c880 0x7f652015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f6524c6e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65200b2bb0 0x7f65200b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f6524c6e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65200b4600 con 0x7f650c005200 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f6524c6e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652015d230 con 0x7f652015c880 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f6524c6e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65200b3980 con 0x7f65200b2bb0 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f65201f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f65201f2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57486/0 (socket says 172.21.15.6:57486) 2023-11-28T13:00:21.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651e59c700 1 -- 172.21.15.6:0/1931088120 learned_addr learned my addr 172.21.15.6:0/1931088120 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651ed9d700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65200b2bb0 0x7f65200b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651dd9b700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f652015c880 0x7f652015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651e59c700 1 -- 172.21.15.6:0/1931088120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65200b2bb0 msgr2=0x7f65200b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651e59c700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65200b2bb0 0x7f65200b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651e59c700 1 -- 172.21.15.6:0/1931088120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f652015c880 msgr2=0x7f652015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651e59c700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f652015c880 0x7f652015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651e59c700 1 -- 172.21.15.6:0/1931088120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200aef40 con 0x7f650c005200 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.155+0000 7f651e59c700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f65201f2910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65080119d0 tx=0x7f6508011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.156+0000 7f6516ffd700 1 -- 172.21.15.6:0/1931088120 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508011e60 con 0x7f650c005200 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.156+0000 7f6516ffd700 1 -- 172.21.15.6:0/1931088120 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650800f940 con 0x7f650c005200 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.156+0000 7f6524c6e700 1 -- 172.21.15.6:0/1931088120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65200aec60 con 0x7f650c005200 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.156+0000 7f6516ffd700 1 -- 172.21.15.6:0/1931088120 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650800fcd0 con 0x7f650c005200 2023-11-28T13:00:21.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.156+0000 7f6524c6e700 1 -- 172.21.15.6:0/1931088120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65200af220 con 0x7f650c005200 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.156+0000 7f6524c6e700 1 -- 172.21.15.6:0/1931088120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65200739f0 con 0x7f650c005200 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.157+0000 7f6516ffd700 1 -- 172.21.15.6:0/1931088120 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6508014440 con 0x7f650c005200 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.157+0000 7f6516ffd700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65000682f0 0x7f650006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.157+0000 7f6516ffd700 1 -- 172.21.15.6:0/1931088120 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f6508086310 con 0x7f650c005200 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.157+0000 7f651dd9b700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65000682f0 0x7f650006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.158+0000 7f651dd9b700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65000682f0 0x7f650006a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6510001c90 tx=0x7f6510001b20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.162+0000 7f6516ffd700 1 -- 172.21.15.6:0/1931088120 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6508052b70 con 0x7f650c005200 2023-11-28T13:00:21.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.402+0000 7f6524c6e700 1 -- 172.21.15.6:0/1931088120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f65201e4190 con 0x7f650c005200 2023-11-28T13:00:21.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.412+0000 7f6516ffd700 1 -- 172.21.15.6:0/1931088120 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6508056190 con 0x7f650c005200 2023-11-28T13:00:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.416+0000 7f6524c6e700 1 -- 172.21.15.6:0/1931088120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65000682f0 msgr2=0x7f650006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.416+0000 7f6524c6e700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65000682f0 0x7f650006a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6510001c90 tx=0x7f6510001b20 comp rx=0 tx=0).stop 2023-11-28T13:00:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.416+0000 7f6524c6e700 1 -- 172.21.15.6:0/1931088120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 msgr2=0x7f65201f2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.416+0000 7f6524c6e700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f65201f2910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65080119d0 tx=0x7f6508011ce0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.416+0000 7f6524c6e700 1 -- 172.21.15.6:0/1931088120 shutdown_connections 2023-11-28T13:00:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.416+0000 7f6524c6e700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65000682f0 0x7f650006a7a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.416+0000 7f6524c6e700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f65200b2bb0 0x7f65200b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.416+0000 7f6524c6e700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f650c005200 0x7f65201f2910 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65080119d0 tx=0x7f6508011ce0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.416+0000 7f6524c6e700 1 --2- 172.21.15.6:0/1931088120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f652015c880 0x7f652015ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.416+0000 7f6524c6e700 1 -- 172.21.15.6:0/1931088120 >> 172.21.15.6:0/1931088120 conn(0x7f65200cc630 msgr2=0x7f65200b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:21.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.418+0000 7f6524c6e700 1 -- 172.21.15.6:0/1931088120 shutdown_connections 2023-11-28T13:00:21.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.419+0000 7f6524c6e700 1 -- 172.21.15.6:0/1931088120 wait complete. 2023-11-28T13:00:21.434 INFO:teuthology.orchestra.run.smithi006.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T13:00:21.444 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-28T13:00:21.444 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-28T13:00:21.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.573+0000 7fdc1c323700 1 Processor -- start 2023-11-28T13:00:21.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.573+0000 7fdc1c323700 1 -- start start 2023-11-28T13:00:21.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1c323700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140cb230 0x7fdc140ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1c323700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdc140b2660 0x7fdc140b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1c323700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc140cfb90 con 0x7fdc1415a6c0 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1c323700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc140b3020 con 0x7fdc140b2660 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1c323700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc140cb680 con 0x7fdc140cb230 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1a0bf700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fdc1415a6c0 0x7fdc140d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44826/0 (socket says 172.21.15.6:44826) 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1a0bf700 1 -- 172.21.15.6:0/3170700440 learned_addr learned my addr 172.21.15.6:0/3170700440 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1a8c0700 1 --2- 172.21.15.6:0/3170700440 >> v2:172.21.15.6:3300/0 conn(0x7fdc140b2660 0x7fdc140b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc198be700 1 --2- 172.21.15.6:0/3170700440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140cb230 0x7fdc140ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1a8c0700 1 -- 172.21.15.6:0/3170700440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140cb230 msgr2=0x7fdc140ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1a8c0700 1 --2- 172.21.15.6:0/3170700440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140cb230 0x7fdc140ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1a8c0700 1 -- 172.21.15.6:0/3170700440 >> v1:172.21.15.6:6789/0 conn(0x7fdc1415a6c0 legacy=0x7fdc140d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1a8c0700 1 -- 172.21.15.6:0/3170700440 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc140cb930 con 0x7fdc140b2660 2023-11-28T13:00:21.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc190bd700 1 -- 172.21.15.6:0/3170700440 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3564336790 0 0) 0x7fdc140cfb90 con 0x7fdc1415a6c0 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.574+0000 7fdc1a8c0700 1 --2- 172.21.15.6:0/3170700440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 0x7fdc140b2ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009d90 tx=0x7fdc0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4875056bd39ae47e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc190bd700 1 -- 172.21.15.6:0/3170700440 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00e070 con 0x7fdc140b2660 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc190bd700 1 -- 172.21.15.6:0/3170700440 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c00bbd0 con 0x7fdc140b2660 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc190bd700 1 -- 172.21.15.6:0/3170700440 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c012430 con 0x7fdc140b2660 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc1c323700 1 -- 172.21.15.6:0/3170700440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 msgr2=0x7fdc140b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc1c323700 1 --2- 172.21.15.6:0/3170700440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 0x7fdc140b2ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009d90 tx=0x7fdc0c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc1c323700 1 -- 172.21.15.6:0/3170700440 shutdown_connections 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc1c323700 1 --2- 172.21.15.6:0/3170700440 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 0x7fdc140b2ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc1c323700 1 --2- 172.21.15.6:0/3170700440 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc140cb230 0x7fdc140ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc1c323700 1 -- 172.21.15.6:0/3170700440 >> 172.21.15.6:0/3170700440 conn(0x7fdc140cd660 msgr2=0x7fdc140cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc1c323700 1 -- 172.21.15.6:0/3170700440 shutdown_connections 2023-11-28T13:00:21.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.575+0000 7fdc1c323700 1 -- 172.21.15.6:0/3170700440 wait complete. 2023-11-28T13:00:21.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.576+0000 7fdc1c323700 1 Processor -- start 2023-11-28T13:00:21.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.576+0000 7fdc1c323700 1 -- start start 2023-11-28T13:00:21.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc1c323700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 0x7fdc140c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc1c323700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc140cb230 0x7fdc140ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc1c323700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc1415a6c0 0x7fdc140ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc1c323700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc140c8780 con 0x7fdc140b2660 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc1c323700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1406ce70 con 0x7fdc140cb230 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc1c323700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc140aefa0 con 0x7fdc1415a6c0 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc1a0bf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 0x7fdc140c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc1a0bf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 0x7fdc140c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57512/0 (socket says 172.21.15.6:57512) 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc1a0bf700 1 -- 172.21.15.6:0/2659604986 learned_addr learned my addr 172.21.15.6:0/2659604986 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc1a8c0700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc1415a6c0 0x7fdc140ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.577+0000 7fdc198be700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc140cb230 0x7fdc140ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc1a0bf700 1 -- 172.21.15.6:0/2659604986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc1415a6c0 msgr2=0x7fdc140ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc1a0bf700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc1415a6c0 0x7fdc140ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc1a0bf700 1 -- 172.21.15.6:0/2659604986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc140cb230 msgr2=0x7fdc140ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:21.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc1a0bf700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc140cb230 0x7fdc140ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc1a0bf700 1 -- 172.21.15.6:0/2659604986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c009a70 con 0x7fdc140b2660 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc1a0bf700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 0x7fdc140c7090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc100011e0 tx=0x7fdc1000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc0affd700 1 -- 172.21.15.6:0/2659604986 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004c80 con 0x7fdc140b2660 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc0affd700 1 -- 172.21.15.6:0/2659604986 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc10004e00 con 0x7fdc140b2660 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc1c323700 1 -- 172.21.15.6:0/2659604986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc140cf540 con 0x7fdc140b2660 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc0affd700 1 -- 172.21.15.6:0/2659604986 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10007310 con 0x7fdc140b2660 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.578+0000 7fdc1c323700 1 -- 172.21.15.6:0/2659604986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc140cf260 con 0x7fdc140b2660 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.579+0000 7fdc1c323700 1 -- 172.21.15.6:0/2659604986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc140cf820 con 0x7fdc140b2660 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.579+0000 7fdc0affd700 1 -- 172.21.15.6:0/2659604986 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdc1001a020 con 0x7fdc140b2660 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.580+0000 7fdc0affd700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbf80682f0 0x7fdbf806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:21.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.580+0000 7fdc0affd700 1 -- 172.21.15.6:0/2659604986 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc100857c0 con 0x7fdc140b2660 2023-11-28T13:00:21.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.580+0000 7fdc198be700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbf80682f0 0x7fdbf806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:21.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.580+0000 7fdc198be700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbf80682f0 0x7fdbf806a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc140b1e40 tx=0x7fdc04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:21.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.584+0000 7fdc0affd700 1 -- 172.21.15.6:0/2659604986 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdc10051f90 con 0x7fdc140b2660 2023-11-28T13:00:21.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:21.789+0000 7fdc1c323700 1 -- 172.21.15.6:0/2659604986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fdc140739f0 con 0x7fdc140b2660 2023-11-28T13:00:22.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.410+0000 7fdc0affd700 1 -- 172.21.15.6:0/2659604986 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7fdc100555b0 con 0x7fdc140b2660 2023-11-28T13:00:22.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 -- 172.21.15.6:0/2659604986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbf80682f0 msgr2=0x7fdbf806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbf80682f0 0x7fdbf806a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdc140b1e40 tx=0x7fdc04008040 comp rx=0 tx=0).stop 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 -- 172.21.15.6:0/2659604986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 msgr2=0x7fdc140c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 0x7fdc140c7090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc100011e0 tx=0x7fdc1000ca30 comp rx=0 tx=0).stop 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 -- 172.21.15.6:0/2659604986 shutdown_connections 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbf80682f0 0x7fdbf806a7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc1415a6c0 0x7fdc140ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc140b2660 0x7fdc140c7090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 --2- 172.21.15.6:0/2659604986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc140cb230 0x7fdc140ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 -- 172.21.15.6:0/2659604986 >> 172.21.15.6:0/2659604986 conn(0x7fdc140cd660 msgr2=0x7fdc140b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 -- 172.21.15.6:0/2659604986 shutdown_connections 2023-11-28T13:00:22.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.413+0000 7fdc1c323700 1 -- 172.21.15.6:0/2659604986 wait complete. 2023-11-28T13:00:22.430 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:00:22.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.569+0000 7f40672f8700 1 Processor -- start 2023-11-28T13:00:22.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.569+0000 7f40672f8700 1 -- start start 2023-11-28T13:00:22.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f40672f8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f406015c8a0 0x7f40600b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f40672f8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f40600acd70 0x7f40600cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f40672f8700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40600d0280 con 0x7f40600b2290 2023-11-28T13:00:22.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f40672f8700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40600d0560 con 0x7f40600acd70 2023-11-28T13:00:22.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f40672f8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40600d0840 con 0x7f406015c8a0 2023-11-28T13:00:22.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f4065895700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f40600acd70 0x7f40600cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f4064893700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f40600b2290 0x7f40600ac660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44842/0 (socket says 172.21.15.6:44842) 2023-11-28T13:00:22.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f4064893700 1 -- 172.21.15.6:0/2757598607 learned_addr learned my addr 172.21.15.6:0/2757598607 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:22.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f4065094700 1 --2- 172.21.15.6:0/2757598607 >> v2:172.21.15.61:3300/0 conn(0x7f406015c8a0 0x7f40600b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f4065895700 1 -- 172.21.15.6:0/2757598607 >> v2:172.21.15.61:3300/0 conn(0x7f406015c8a0 msgr2=0x7f40600b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f4065895700 1 --2- 172.21.15.6:0/2757598607 >> v2:172.21.15.61:3300/0 conn(0x7f406015c8a0 0x7f40600b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f4065895700 1 -- 172.21.15.6:0/2757598607 >> v1:172.21.15.6:6789/0 conn(0x7f40600b2290 legacy=0x7f40600ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f4065895700 1 -- 172.21.15.6:0/2757598607 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40600b1550 con 0x7f40600acd70 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.570+0000 7f4057fff700 1 -- 172.21.15.6:0/2757598607 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692087618 0 0) 0x7f40600d0280 con 0x7f40600b2290 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f4065895700 1 --2- 172.21.15.6:0/2757598607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40600acd70 0x7f40600cd610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4060072990 tx=0x7f405800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483fd83e0e1d8ca1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f4057fff700 1 -- 172.21.15.6:0/2757598607 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405800e070 con 0x7f40600acd70 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f4057fff700 1 -- 172.21.15.6:0/2757598607 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405800bb20 con 0x7f40600acd70 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f4057fff700 1 -- 172.21.15.6:0/2757598607 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058012430 con 0x7f40600acd70 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f40672f8700 1 -- 172.21.15.6:0/2757598607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40600acd70 msgr2=0x7f40600cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f40672f8700 1 --2- 172.21.15.6:0/2757598607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40600acd70 0x7f40600cd610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4060072990 tx=0x7f405800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:22.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f40672f8700 1 -- 172.21.15.6:0/2757598607 shutdown_connections 2023-11-28T13:00:22.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f40672f8700 1 --2- 172.21.15.6:0/2757598607 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40600acd70 0x7f40600cd610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f40672f8700 1 --2- 172.21.15.6:0/2757598607 >> v2:172.21.15.61:3300/0 conn(0x7f406015c8a0 0x7f40600b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f40672f8700 1 -- 172.21.15.6:0/2757598607 >> 172.21.15.6:0/2757598607 conn(0x7f40600b3610 msgr2=0x7f40600cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:22.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f40672f8700 1 -- 172.21.15.6:0/2757598607 shutdown_connections 2023-11-28T13:00:22.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.571+0000 7f40672f8700 1 -- 172.21.15.6:0/2757598607 wait complete. 2023-11-28T13:00:22.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.573+0000 7f40672f8700 1 Processor -- start 2023-11-28T13:00:22.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.573+0000 7f40672f8700 1 -- start start 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.573+0000 7f40672f8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40600acd70 0x7f4060153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.573+0000 7f40672f8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40600b2290 0x7f40600ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.573+0000 7f40672f8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f406015c8a0 0x7f40600ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.573+0000 7f40672f8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40600c33a0 con 0x7f406015c8a0 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.573+0000 7f40672f8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4060155150 con 0x7f40600b2290 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.573+0000 7f40672f8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4060154540 con 0x7f40600acd70 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4065895700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f406015c8a0 0x7f40600ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4065895700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f406015c8a0 0x7f40600ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57530/0 (socket says 172.21.15.6:57530) 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4065895700 1 -- 172.21.15.6:0/3468510945 learned_addr learned my addr 172.21.15.6:0/3468510945 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4065895700 1 -- 172.21.15.6:0/3468510945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40600acd70 msgr2=0x7f4060153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4065895700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40600acd70 0x7f4060153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4064893700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40600b2290 0x7f40600ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4065895700 1 -- 172.21.15.6:0/3468510945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40600b2290 msgr2=0x7f40600ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4065895700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40600b2290 0x7f40600ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4065895700 1 -- 172.21.15.6:0/3468510945 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4058009a70 con 0x7f406015c8a0 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4065895700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f406015c8a0 0x7f40600ca590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f40600708f0 tx=0x7f4058004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4055ffb700 1 -- 172.21.15.6:0/3468510945 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058012430 con 0x7f406015c8a0 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4055ffb700 1 -- 172.21.15.6:0/3468510945 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4058020070 con 0x7f406015c8a0 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f40672f8700 1 -- 172.21.15.6:0/3468510945 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406015cc70 con 0x7f406015c8a0 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f4055ffb700 1 -- 172.21.15.6:0/3468510945 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4058004280 con 0x7f406015c8a0 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.574+0000 7f40672f8700 1 -- 172.21.15.6:0/3468510945 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40600b1830 con 0x7f406015c8a0 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.575+0000 7f40672f8700 1 -- 172.21.15.6:0/3468510945 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406006a6c0 con 0x7f406015c8a0 2023-11-28T13:00:22.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.576+0000 7f4055ffb700 1 -- 172.21.15.6:0/3468510945 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f40580043e0 con 0x7f406015c8a0 2023-11-28T13:00:22.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.576+0000 7f4055ffb700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40440682f0 0x7f404406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:22.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.576+0000 7f4055ffb700 1 -- 172.21.15.6:0/3468510945 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f4058022070 con 0x7f406015c8a0 2023-11-28T13:00:22.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.576+0000 7f4065094700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40440682f0 0x7f404406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:22.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.577+0000 7f4065094700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40440682f0 0x7f404406a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4050006b20 tx=0x7f4050006ab0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:22.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.580+0000 7f4055ffb700 1 -- 172.21.15.6:0/3468510945 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f40580b95e0 con 0x7f406015c8a0 2023-11-28T13:00:22.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:22.777+0000 7f40672f8700 1 -- 172.21.15.6:0/3468510945 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4060155300 con 0x7f406015c8a0 2023-11-28T13:00:24.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.431+0000 7f4055ffb700 1 -- 172.21.15.6:0/3468510945 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f4060155300 con 0x7f406015c8a0 2023-11-28T13:00:24.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.436+0000 7f40672f8700 1 -- 172.21.15.6:0/3468510945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40440682f0 msgr2=0x7f404406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.436+0000 7f40672f8700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40440682f0 0x7f404406a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4050006b20 tx=0x7f4050006ab0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.436+0000 7f40672f8700 1 -- 172.21.15.6:0/3468510945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f406015c8a0 msgr2=0x7f40600ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.436+0000 7f40672f8700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f406015c8a0 0x7f40600ca590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f40600708f0 tx=0x7f4058004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.437+0000 7f40672f8700 1 -- 172.21.15.6:0/3468510945 shutdown_connections 2023-11-28T13:00:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.437+0000 7f40672f8700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f406015c8a0 0x7f40600ca590 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.437+0000 7f40672f8700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40440682f0 0x7f404406a7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.437+0000 7f40672f8700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40600acd70 0x7f4060153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.437+0000 7f40672f8700 1 --2- 172.21.15.6:0/3468510945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40600b2290 0x7f40600ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.437+0000 7f40672f8700 1 -- 172.21.15.6:0/3468510945 >> 172.21.15.6:0/3468510945 conn(0x7f40600b3610 msgr2=0x7f40600ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:24.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.437+0000 7f40672f8700 1 -- 172.21.15.6:0/3468510945 shutdown_connections 2023-11-28T13:00:24.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.437+0000 7f40672f8700 1 -- 172.21.15.6:0/3468510945 wait complete. 2023-11-28T13:00:24.446 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data_ec' created 2023-11-28T13:00:24.459 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-28T13:00:24.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.592+0000 7f932bbe8700 1 Processor -- start 2023-11-28T13:00:24.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.592+0000 7f932bbe8700 1 -- start start 2023-11-28T13:00:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f932bbe8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932406e400 0x7f932415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f932bbe8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f93240ce890 0x7f93240ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f932bbe8700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932415e0b0 con 0x7f9324159590 2023-11-28T13:00:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f932bbe8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93240cf250 con 0x7f93240ce890 2023-11-28T13:00:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f932bbe8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932415d120 con 0x7f932406e400 2023-11-28T13:00:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f9329984700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f9324159590 0x7f93240ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44854/0 (socket says 172.21.15.6:44854) 2023-11-28T13:00:24.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f9329984700 1 -- 172.21.15.6:0/2743270390 learned_addr learned my addr 172.21.15.6:0/2743270390 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f932a185700 1 --2- 172.21.15.6:0/2743270390 >> v2:172.21.15.61:3300/0 conn(0x7f93240ce890 0x7f93240ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f9329183700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932406e400 0x7f932415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154745072 0 0) 0x7f9314001110 con 0x7f9324159590 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.593+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93100035b0 con 0x7f9324159590 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1295906167 0 0) 0x7f93100035b0 con 0x7f9324159590 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932406e400 msgr2=0x7f932415db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932406e400 0x7f932415db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 >> v2:172.21.15.61:3300/0 conn(0x7f93240ce890 msgr2=0x7f93240ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 --2- 172.21.15.6:0/2743270390 >> v2:172.21.15.61:3300/0 conn(0x7f93240ce890 0x7f93240ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932415d6e0 con 0x7f9324159590 2023-11-28T13:00:24.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f9314001800 con 0x7f9324159590 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 >> v1:172.21.15.6:6789/0 conn(0x7f9324159590 legacy=0x7f93240ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9310003a00 0x7f9310003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9310004430 0x7f93100048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932406e400 0x7f9310004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93240cf250 con 0x7f9310003a00 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932415d120 con 0x7f9310004430 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931000ba10 con 0x7f932406e400 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9329984700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9310003a00 0x7f9310003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9329183700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9310004430 0x7f93100048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f932a185700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932406e400 0x7f9310004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:24.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9329984700 1 -- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932406e400 msgr2=0x7f9310004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9329984700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932406e400 0x7f9310004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9329984700 1 -- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9310004430 msgr2=0x7f93100048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9329984700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9310004430 0x7f93100048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9329984700 1 -- 172.21.15.6:0/2743270390 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93100035b0 con 0x7f9310003a00 2023-11-28T13:00:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9329984700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9310003a00 0x7f9310003e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f931400c240 tx=0x7f9310005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeeed939ca363965 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314007db0 con 0x7f9310003a00 2023-11-28T13:00:24.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931400cde0 con 0x7f9310003a00 2023-11-28T13:00:24.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.594+0000 7f9328982700 1 -- 172.21.15.6:0/2743270390 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314005b80 con 0x7f9310003a00 2023-11-28T13:00:24.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.595+0000 7f932bbe8700 1 -- 172.21.15.6:0/2743270390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9310003a00 msgr2=0x7f9310003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:24.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.595+0000 7f932bbe8700 1 --2- 172.21.15.6:0/2743270390 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9310003a00 0x7f9310003e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f931400c240 tx=0x7f9310005880 comp rx=0 tx=0).stop 2023-11-28T13:00:24.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.595+0000 7f932a185700 1 -- 172.21.15.6:0/2743270390 reap_dead start 2023-11-28T13:00:24.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.595+0000 7f932bbe8700 1 -- 172.21.15.6:0/2743270390 shutdown_connections 2023-11-28T13:00:24.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.595+0000 7f932bbe8700 1 -- 172.21.15.6:0/2743270390 >> 172.21.15.6:0/2743270390 conn(0x7f93240b1650 msgr2=0x7f93240b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:24.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.595+0000 7f932bbe8700 1 -- 172.21.15.6:0/2743270390 shutdown_connections 2023-11-28T13:00:24.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.595+0000 7f932bbe8700 1 -- 172.21.15.6:0/2743270390 wait complete. 2023-11-28T13:00:24.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.596+0000 7f932bbe8700 1 Processor -- start 2023-11-28T13:00:24.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.596+0000 7f932bbe8700 1 -- start start 2023-11-28T13:00:24.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f932bbe8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9310003a00 0x7f9324153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:24.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f932bbe8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932406ab50 0x7f9324157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:24.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f932bbe8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93241541a0 0x7f9324154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:24.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f932bbe8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324154b60 con 0x7f932406ab50 2023-11-28T13:00:24.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f932bbe8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93240ac3e0 con 0x7f93241541a0 2023-11-28T13:00:24.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f932bbe8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93240ac6c0 con 0x7f9310003a00 2023-11-28T13:00:24.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f9329183700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932406ab50 0x7f9324157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:24.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f9329183700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932406ab50 0x7f9324157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57548/0 (socket says 172.21.15.6:57548) 2023-11-28T13:00:24.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f9329183700 1 -- 172.21.15.6:0/2498372787 learned_addr learned my addr 172.21.15.6:0/2498372787 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f9329984700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9310003a00 0x7f9324153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.597+0000 7f932a185700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93241541a0 0x7f9324154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f9329183700 1 -- 172.21.15.6:0/2498372787 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9310003a00 msgr2=0x7f9324153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f9329183700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9310003a00 0x7f9324153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f9329183700 1 -- 172.21.15.6:0/2498372787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93241541a0 msgr2=0x7f9324154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f9329183700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93241541a0 0x7f9324154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f9329183700 1 -- 172.21.15.6:0/2498372787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240af240 con 0x7f932406ab50 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f9329183700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932406ab50 0x7f9324157a10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93200119a0 tx=0x7f9320011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f931a7fc700 1 -- 172.21.15.6:0/2498372787 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932000f780 con 0x7f932406ab50 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f931a7fc700 1 -- 172.21.15.6:0/2498372787 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932000f8e0 con 0x7f932406ab50 2023-11-28T13:00:24.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f932bbe8700 1 -- 172.21.15.6:0/2498372787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93240aef60 con 0x7f932406ab50 2023-11-28T13:00:24.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f931a7fc700 1 -- 172.21.15.6:0/2498372787 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932000fc70 con 0x7f932406ab50 2023-11-28T13:00:24.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.598+0000 7f932bbe8700 1 -- 172.21.15.6:0/2498372787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932415d370 con 0x7f932406ab50 2023-11-28T13:00:24.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.599+0000 7f932bbe8700 1 -- 172.21.15.6:0/2498372787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93240739f0 con 0x7f932406ab50 2023-11-28T13:00:24.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.600+0000 7f931a7fc700 1 -- 172.21.15.6:0/2498372787 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f932001d400 con 0x7f932406ab50 2023-11-28T13:00:24.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.600+0000 7f931a7fc700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93080682f0 0x7f930806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:24.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.600+0000 7f931a7fc700 1 -- 172.21.15.6:0/2498372787 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f9320086c40 con 0x7f932406ab50 2023-11-28T13:00:24.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.600+0000 7f9329984700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93080682f0 0x7f930806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:24.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.601+0000 7f9329984700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93080682f0 0x7f930806a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9314000c00 tx=0x7f931400a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:24.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.604+0000 7f931a7fc700 1 -- 172.21.15.6:0/2498372787 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9320053100 con 0x7f932406ab50 2023-11-28T13:00:24.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:24.799+0000 7f932bbe8700 1 -- 172.21.15.6:0/2498372787 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f932406c1b0 con 0x7f932406ab50 2023-11-28T13:00:25.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.443+0000 7f931a7fc700 1 -- 172.21.15.6:0/2498372787 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9320056720 con 0x7f932406ab50 2023-11-28T13:00:25.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.447+0000 7f932bbe8700 1 -- 172.21.15.6:0/2498372787 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93080682f0 msgr2=0x7f930806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:25.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.447+0000 7f932bbe8700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93080682f0 0x7f930806a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9314000c00 tx=0x7f931400a000 comp rx=0 tx=0).stop 2023-11-28T13:00:25.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.447+0000 7f932bbe8700 1 -- 172.21.15.6:0/2498372787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932406ab50 msgr2=0x7f9324157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:25.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.447+0000 7f932bbe8700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932406ab50 0x7f9324157a10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93200119a0 tx=0x7f9320011cb0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.449+0000 7f932bbe8700 1 -- 172.21.15.6:0/2498372787 shutdown_connections 2023-11-28T13:00:25.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.449+0000 7f932bbe8700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93241541a0 0x7f9324154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.449+0000 7f932bbe8700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93080682f0 0x7f930806a7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.449+0000 7f932bbe8700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9310003a00 0x7f9324153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.449+0000 7f932bbe8700 1 --2- 172.21.15.6:0/2498372787 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932406ab50 0x7f9324157a10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.449+0000 7f932bbe8700 1 -- 172.21.15.6:0/2498372787 >> 172.21.15.6:0/2498372787 conn(0x7f93240b1650 msgr2=0x7f93240cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:25.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.451+0000 7f932bbe8700 1 -- 172.21.15.6:0/2498372787 shutdown_connections 2023-11-28T13:00:25.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.452+0000 7f932bbe8700 1 -- 172.21.15.6:0/2498372787 wait complete. 2023-11-28T13:00:25.455 INFO:teuthology.orchestra.run.smithi006.stderr:set pool 5 allow_ec_overwrites to true 2023-11-28T13:00:25.481 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-28T13:00:25.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.611+0000 7f12d4036700 1 Processor -- start 2023-11-28T13:00:25.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.611+0000 7f12d4036700 1 -- start start 2023-11-28T13:00:25.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.612+0000 7f12d4036700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc0b1630 0x7f12cc0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:25.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.612+0000 7f12d4036700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0ce550 con 0x7f12cc06afc0 2023-11-28T13:00:25.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.612+0000 7f12d4036700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0ce830 con 0x7f12cc0b1a00 2023-11-28T13:00:25.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.612+0000 7f12d4036700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0ceb10 con 0x7f12cc0b1630 2023-11-28T13:00:25.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.612+0000 7f12d25d3700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f12cc06afc0 0x7f12cc06b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44878/0 (socket says 172.21.15.6:44878) 2023-11-28T13:00:25.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.612+0000 7f12d25d3700 1 -- 172.21.15.6:0/3678577456 learned_addr learned my addr 172.21.15.6:0/3678577456 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:25.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.612+0000 7f12d1dd2700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc0b1630 0x7f12cc0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:25.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.612+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032567213 0 0) 0x7f12cc0ce550 con 0x7f12cc06afc0 2023-11-28T13:00:25.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.612+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b80035b0 con 0x7f12cc06afc0 2023-11-28T13:00:25.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 6424313 0 0) 0x7f12b80035b0 con 0x7f12cc06afc0 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc0b1630 msgr2=0x7f12cc0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc0b1630 0x7f12cc0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 >> v1:172.21.15.61:6789/0 conn(0x7f12cc0b1a00 legacy=0x7f12cc0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0cedf0 con 0x7f12cc06afc0 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f12c40019f0 con 0x7f12cc06afc0 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 >> v1:172.21.15.6:6789/0 conn(0x7f12cc06afc0 legacy=0x7f12cc06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12b8003a30 0x7f12b8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8004850 0x7f12b8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc0b1630 0x7f12b8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0ceb10 con 0x7f12b8003a30 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b800bac0 con 0x7f12b8004850 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b800bc50 con 0x7f12cc0b1630 2023-11-28T13:00:25.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d1dd2700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12b8003a30 0x7f12b8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d15d1700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8004850 0x7f12b8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d25d3700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc0b1630 0x7f12b8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d1dd2700 1 -- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc0b1630 msgr2=0x7f12b8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d1dd2700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc0b1630 0x7f12b8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d1dd2700 1 -- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8004850 msgr2=0x7f12b8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d1dd2700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8004850 0x7f12b8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d1dd2700 1 -- 172.21.15.6:0/3678577456 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80035b0 con 0x7f12b8003a30 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d15d1700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8004850 0x7f12b8004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d1dd2700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12b8003a30 0x7f12b8003e40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009b60 tx=0x7f12b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb64fc90b884270 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000e6c0 con 0x7f12b8003a30 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.613+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c000e820 con 0x7f12b8003a30 2023-11-28T13:00:25.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.614+0000 7f12d0dd0700 1 -- 172.21.15.6:0/3678577456 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000eb90 con 0x7f12b8003a30 2023-11-28T13:00:25.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.614+0000 7f12d4036700 1 -- 172.21.15.6:0/3678577456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12b8003a30 msgr2=0x7f12b8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:25.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.614+0000 7f12d4036700 1 --2- 172.21.15.6:0/3678577456 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12b8003a30 0x7f12b8003e40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009b60 tx=0x7f12b8005880 comp rx=0 tx=0).stop 2023-11-28T13:00:25.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.614+0000 7f12d25d3700 1 -- 172.21.15.6:0/3678577456 reap_dead start 2023-11-28T13:00:25.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.614+0000 7f12d4036700 1 -- 172.21.15.6:0/3678577456 shutdown_connections 2023-11-28T13:00:25.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.614+0000 7f12d4036700 1 -- 172.21.15.6:0/3678577456 >> 172.21.15.6:0/3678577456 conn(0x7f12cc0cc630 msgr2=0x7f12cc0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:25.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.614+0000 7f12d4036700 1 -- 172.21.15.6:0/3678577456 shutdown_connections 2023-11-28T13:00:25.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.614+0000 7f12d4036700 1 -- 172.21.15.6:0/3678577456 wait complete. 2023-11-28T13:00:25.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.616+0000 7f12d4036700 1 Processor -- start 2023-11-28T13:00:25.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.616+0000 7f12d4036700 1 -- start start 2023-11-28T13:00:25.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d4036700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8003a30 0x7f12cc14f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:25.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d4036700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12cc14fb20 0x7f12cc15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:25.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d4036700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc14ff90 0x7f12cc0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:25.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d4036700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc15db20 con 0x7f12cc14fb20 2023-11-28T13:00:25.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d4036700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0b4880 con 0x7f12b8003a30 2023-11-28T13:00:25.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d4036700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc15d560 con 0x7f12cc14ff90 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d15d1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12cc14fb20 0x7f12cc15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d15d1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12cc14fb20 0x7f12cc15caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57568/0 (socket says 172.21.15.6:57568) 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d15d1700 1 -- 172.21.15.6:0/2800774904 learned_addr learned my addr 172.21.15.6:0/2800774904 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d25d3700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc14ff90 0x7f12cc0c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d1dd2700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8003a30 0x7f12cc14f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d15d1700 1 -- 172.21.15.6:0/2800774904 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc14ff90 msgr2=0x7f12cc0c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d15d1700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc14ff90 0x7f12cc0c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d15d1700 1 -- 172.21.15.6:0/2800774904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8003a30 msgr2=0x7f12cc14f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d15d1700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8003a30 0x7f12cc14f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d15d1700 1 -- 172.21.15.6:0/2800774904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0b1e70 con 0x7f12cc14fb20 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d1dd2700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8003a30 0x7f12cc14f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:25.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12d15d1700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12cc14fb20 0x7f12cc15caa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e580 tx=0x7f12c800e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12be7fc700 1 -- 172.21.15.6:0/2800774904 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800eee0 con 0x7f12cc14fb20 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.617+0000 7f12be7fc700 1 -- 172.21.15.6:0/2800774904 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c80052e0 con 0x7f12cc14fb20 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.618+0000 7f12d4036700 1 -- 172.21.15.6:0/2800774904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc0cf0d0 con 0x7f12cc14fb20 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.618+0000 7f12be7fc700 1 -- 172.21.15.6:0/2800774904 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8003680 con 0x7f12cc14fb20 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.618+0000 7f12d4036700 1 -- 172.21.15.6:0/2800774904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc0cb5b0 con 0x7f12cc14fb20 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.618+0000 7f12d4036700 1 -- 172.21.15.6:0/2800774904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc0b4a10 con 0x7f12cc14fb20 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.619+0000 7f12be7fc700 1 -- 172.21.15.6:0/2800774904 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f12c801b030 con 0x7f12cc14fb20 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.619+0000 7f12be7fc700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f12b00682f0 0x7f12b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.620+0000 7f12be7fc700 1 -- 172.21.15.6:0/2800774904 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f12c8085c30 con 0x7f12cc14fb20 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.620+0000 7f12d1dd2700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f12b00682f0 0x7f12b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.620+0000 7f12d1dd2700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f12b00682f0 0x7f12b006a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f12c00090b0 tx=0x7f12c0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:25.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.623+0000 7f12be7fc700 1 -- 172.21.15.6:0/2800774904 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f12c8052200 con 0x7f12cc14fb20 2023-11-28T13:00:25.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.863+0000 7f12d4036700 1 -- 172.21.15.6:0/2800774904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f12cc0739f0 con 0x7f12cc14fb20 2023-11-28T13:00:25.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.868+0000 7f12be7fc700 1 -- 172.21.15.6:0/2800774904 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f12c8055820 con 0x7f12cc14fb20 2023-11-28T13:00:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 -- 172.21.15.6:0/2800774904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f12b00682f0 msgr2=0x7f12b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f12b00682f0 0x7f12b006a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f12c00090b0 tx=0x7f12c0008030 comp rx=0 tx=0).stop 2023-11-28T13:00:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 -- 172.21.15.6:0/2800774904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12cc14fb20 msgr2=0x7f12cc15caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12cc14fb20 0x7f12cc15caa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e580 tx=0x7f12c800e890 comp rx=0 tx=0).stop 2023-11-28T13:00:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 -- 172.21.15.6:0/2800774904 shutdown_connections 2023-11-28T13:00:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f12b00682f0 0x7f12b006a7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12cc14ff90 0x7f12cc0c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12b8003a30 0x7f12cc14f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 --2- 172.21.15.6:0/2800774904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f12cc14fb20 0x7f12cc15caa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:25.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 -- 172.21.15.6:0/2800774904 >> 172.21.15.6:0/2800774904 conn(0x7f12cc0cc630 msgr2=0x7f12cc0caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:25.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 -- 172.21.15.6:0/2800774904 shutdown_connections 2023-11-28T13:00:25.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:25.870+0000 7f12d4036700 1 -- 172.21.15.6:0/2800774904 wait complete. 2023-11-28T13:00:25.873 INFO:teuthology.orchestra.run.smithi006.stderr:added data pool 5 to fsmap 2023-11-28T13:00:25.883 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:00:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.006+0000 7fedb8a25700 1 Processor -- start 2023-11-28T13:00:26.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.007+0000 7fedb8a25700 1 -- start start 2023-11-28T13:00:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.007+0000 7fedb8a25700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fedb40cd560 0x7fedb40cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.007+0000 7fedb8a25700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fedb40ce410 0x7fedb40ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.007+0000 7fedb8a25700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb40ced90 con 0x7fedb4159270 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.007+0000 7fedb8a25700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb406bf40 con 0x7fedb40cd560 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.007+0000 7fedb8a25700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb40cdfe0 con 0x7fedb40ce410 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb1d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fedb40cd560 0x7fedb40cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb1d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fedb40cd560 0x7fedb40cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57576/0 (socket says 172.21.15.6:57576) 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb259c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fedb4159270 0x7fedb40cab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44894/0 (socket says 172.21.15.6:44894) 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb1d9b700 1 -- 172.21.15.6:0/937381691 learned_addr learned my addr 172.21.15.6:0/937381691 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb1d9b700 1 -- 172.21.15.6:0/937381691 >> v2:172.21.15.61:3300/0 conn(0x7fedb40ce410 msgr2=0x7fedb40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb2d9d700 1 --2- 172.21.15.6:0/937381691 >> v2:172.21.15.61:3300/0 conn(0x7fedb40ce410 0x7fedb40ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb1d9b700 1 --2- 172.21.15.6:0/937381691 >> v2:172.21.15.61:3300/0 conn(0x7fedb40ce410 0x7fedb40ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb159a700 1 -- 172.21.15.6:0/937381691 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358108738 0 0) 0x7fedb40ced90 con 0x7fedb4159270 2023-11-28T13:00:26.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb1d9b700 1 -- 172.21.15.6:0/937381691 >> v1:172.21.15.6:6789/0 conn(0x7fedb4159270 legacy=0x7fedb40cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb1d9b700 1 -- 172.21.15.6:0/937381691 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb406c150 con 0x7fedb40cd560 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb1d9b700 1 --2- 172.21.15.6:0/937381691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb40cd560 0x7fedb40cda00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fedb40af380 tx=0x7feda8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25d23ed9852a57b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb159a700 1 -- 172.21.15.6:0/937381691 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800b870 con 0x7fedb40cd560 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb159a700 1 -- 172.21.15.6:0/937381691 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda800b9d0 con 0x7fedb40cd560 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.008+0000 7fedb159a700 1 -- 172.21.15.6:0/937381691 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800bd60 con 0x7fedb40cd560 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.009+0000 7fedb8a25700 1 -- 172.21.15.6:0/937381691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb40cd560 msgr2=0x7fedb40cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.009+0000 7fedb8a25700 1 --2- 172.21.15.6:0/937381691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb40cd560 0x7fedb40cda00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fedb40af380 tx=0x7feda8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.009+0000 7fedb8a25700 1 -- 172.21.15.6:0/937381691 shutdown_connections 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.009+0000 7fedb8a25700 1 --2- 172.21.15.6:0/937381691 >> v2:172.21.15.61:3300/0 conn(0x7fedb40ce410 0x7fedb40ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.009+0000 7fedb8a25700 1 --2- 172.21.15.6:0/937381691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb40cd560 0x7fedb40cda00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.009+0000 7fedb8a25700 1 -- 172.21.15.6:0/937381691 >> 172.21.15.6:0/937381691 conn(0x7fedb406dad0 msgr2=0x7fedb406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.009+0000 7fedb8a25700 1 -- 172.21.15.6:0/937381691 shutdown_connections 2023-11-28T13:00:26.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.009+0000 7fedb8a25700 1 -- 172.21.15.6:0/937381691 wait complete. 2023-11-28T13:00:26.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.010+0000 7fedb8a25700 1 Processor -- start 2023-11-28T13:00:26.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.010+0000 7fedb8a25700 1 -- start start 2023-11-28T13:00:26.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb8a25700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedb40cd560 0x7fedb40c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb8a25700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedb40ce410 0x7fedb40cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb8a25700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb4159270 0x7fedb40ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.016 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb8a25700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40cab60 con 0x7fedb4159270 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb8a25700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40c8f30 con 0x7fedb40ce410 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb8a25700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40c9210 con 0x7fedb40cd560 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb4159270 0x7fedb40ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb4159270 0x7fedb40ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57582/0 (socket says 172.21.15.6:57582) 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb2d9d700 1 -- 172.21.15.6:0/769964863 learned_addr learned my addr 172.21.15.6:0/769964863 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb1d9b700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedb40ce410 0x7fedb40cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb259c700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedb40cd560 0x7fedb40c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb2d9d700 1 -- 172.21.15.6:0/769964863 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedb40cd560 msgr2=0x7fedb40c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb2d9d700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedb40cd560 0x7fedb40c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb2d9d700 1 -- 172.21.15.6:0/769964863 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedb40ce410 msgr2=0x7fedb40cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb2d9d700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedb40ce410 0x7fedb40cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb2d9d700 1 -- 172.21.15.6:0/769964863 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda8009550 con 0x7fedb4159270 2023-11-28T13:00:26.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.011+0000 7fedb1d9b700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedb40ce410 0x7fedb40cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:26.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.012+0000 7fedb2d9d700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb4159270 0x7fedb40ad7b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fedac00cd70 tx=0x7fedac00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.012+0000 7feda2ffd700 1 -- 172.21.15.6:0/769964863 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac003a20 con 0x7fedb4159270 2023-11-28T13:00:26.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.012+0000 7feda2ffd700 1 -- 172.21.15.6:0/769964863 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac003b80 con 0x7fedb4159270 2023-11-28T13:00:26.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.012+0000 7fedb8a25700 1 -- 172.21.15.6:0/769964863 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb406c710 con 0x7fedb4159270 2023-11-28T13:00:26.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.012+0000 7feda2ffd700 1 -- 172.21.15.6:0/769964863 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac004020 con 0x7fedb4159270 2023-11-28T13:00:26.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.012+0000 7fedb8a25700 1 -- 172.21.15.6:0/769964863 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb406c430 con 0x7fedb4159270 2023-11-28T13:00:26.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.013+0000 7fedb8a25700 1 -- 172.21.15.6:0/769964863 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb4159640 con 0x7fedb4159270 2023-11-28T13:00:26.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.013+0000 7feda2ffd700 1 -- 172.21.15.6:0/769964863 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fedac01c020 con 0x7fedb4159270 2023-11-28T13:00:26.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.014+0000 7feda2ffd700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed940682f0 0x7fed9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.014+0000 7feda2ffd700 1 -- 172.21.15.6:0/769964863 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fedac012070 con 0x7fedb4159270 2023-11-28T13:00:26.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.014+0000 7fedb259c700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed940682f0 0x7fed9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.014+0000 7fedb259c700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed940682f0 0x7fed9406a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feda4000f60 tx=0x7feda4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.018+0000 7feda2ffd700 1 -- 172.21.15.6:0/769964863 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fedac052d20 con 0x7fedb4159270 2023-11-28T13:00:26.080 INFO:tasks.ceph.mds.c.smithi006.stderr:2023-11-28T13:00:26.079+0000 7fe43c474700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:00:26.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.258+0000 7fedb8a25700 1 -- 172.21.15.6:0/769964863 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedb40b2240 con 0x7fedb4159270 2023-11-28T13:00:26.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.258+0000 7feda2ffd700 1 -- 172.21.15.6:0/769964863 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4638 (secure 0 0 0) 0x7fedac056340 con 0x7fedb4159270 2023-11-28T13:00:26.260 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:26.260 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6834/1568686201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":1568686201},{"type":"v1","addr":"172.21.15.61:6834","nonce":1568686201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/3333487635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6833","nonce":3333487635},{"type":"v1","addr":"172.21.15.61:6835","nonce":3333487635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3917075730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3917075730},{"type":"v1","addr":"172.21.15.6:6835","nonce":3917075730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:21.404170+0000","modified":"2023-11-28T13:00:25.865537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/1668558371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1668558371},{"type":"v1","addr":"172.21.15.6:6837","nonce":1668558371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T13:00:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.260+0000 7fedb8a25700 1 -- 172.21.15.6:0/769964863 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed940682f0 msgr2=0x7fed9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.260+0000 7fedb8a25700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed940682f0 0x7fed9406a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feda4000f60 tx=0x7feda4009040 comp rx=0 tx=0).stop 2023-11-28T13:00:26.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.260+0000 7fedb8a25700 1 -- 172.21.15.6:0/769964863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb4159270 msgr2=0x7fedb40ad7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.260+0000 7fedb8a25700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb4159270 0x7fedb40ad7b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fedac00cd70 tx=0x7fedac00a570 comp rx=0 tx=0).stop 2023-11-28T13:00:26.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.261+0000 7fedb8a25700 1 -- 172.21.15.6:0/769964863 shutdown_connections 2023-11-28T13:00:26.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.261+0000 7fedb8a25700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedb4159270 0x7fedb40ad7b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.261+0000 7fedb8a25700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed940682f0 0x7fed9406a7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.261+0000 7fedb8a25700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedb40cd560 0x7fedb40c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.261+0000 7fedb8a25700 1 --2- 172.21.15.6:0/769964863 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedb40ce410 0x7fedb40cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.261+0000 7fedb8a25700 1 -- 172.21.15.6:0/769964863 >> 172.21.15.6:0/769964863 conn(0x7fedb406dad0 msgr2=0x7fedb4155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:26.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.261+0000 7fedb8a25700 1 -- 172.21.15.6:0/769964863 shutdown_connections 2023-11-28T13:00:26.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.261+0000 7fedb8a25700 1 -- 172.21.15.6:0/769964863 wait complete. 2023-11-28T13:00:26.264 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 7 2023-11-28T13:00:26.274 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.397+0000 7ff6e0f6a700 1 Processor -- start 2023-11-28T13:00:26.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.397+0000 7ff6e0f6a700 1 -- start start 2023-11-28T13:00:26.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6e0f6a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff6dc06b930 0x7ff6dc0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6e0f6a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6dc0cc4b0 0x7ff6dc0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6e0f6a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6dc0cb570 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6e0f6a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc06bda0 con 0x7ff6dc06b930 2023-11-28T13:00:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6e0f6a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0cce70 con 0x7ff6dc0cc4b0 2023-11-28T13:00:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6da59c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff6dc15a5a0 0x7ff6dc0cba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44908/0 (socket says 172.21.15.6:44908) 2023-11-28T13:00:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6da59c700 1 -- 172.21.15.6:0/1981342703 learned_addr learned my addr 172.21.15.6:0/1981342703 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6d9d9b700 1 --2- 172.21.15.6:0/1981342703 >> v2:172.21.15.61:3300/0 conn(0x7ff6dc06b930 0x7ff6dc0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6dad9d700 1 --2- 172.21.15.6:0/1981342703 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6dc0cc4b0 0x7ff6dc0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6d959a700 1 -- 172.21.15.6:0/1981342703 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616770718 0 0) 0x7ff6dc0cb570 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6d959a700 1 -- 172.21.15.6:0/1981342703 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6c40035b0 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6d9d9b700 1 -- 172.21.15.6:0/1981342703 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6dc0cc4b0 msgr2=0x7ff6dc0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6d9d9b700 1 --2- 172.21.15.6:0/1981342703 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6dc0cc4b0 0x7ff6dc0cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6d9d9b700 1 -- 172.21.15.6:0/1981342703 >> v1:172.21.15.6:6789/0 conn(0x7ff6dc15a5a0 legacy=0x7ff6dc0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.398+0000 7ff6d9d9b700 1 -- 172.21.15.6:0/1981342703 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc0cc110 con 0x7ff6dc06b930 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.399+0000 7ff6d9d9b700 1 --2- 172.21.15.6:0/1981342703 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6dc06b930 0x7ff6dc0ca280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009840 tx=0x7ff6cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66bc311c7f89795d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.399+0000 7ff6d959a700 1 -- 172.21.15.6:0/1981342703 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc011830 con 0x7ff6dc06b930 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.399+0000 7ff6d959a700 1 -- 172.21.15.6:0/1981342703 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc011990 con 0x7ff6dc06b930 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.399+0000 7ff6d959a700 1 -- 172.21.15.6:0/1981342703 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc011d00 con 0x7ff6dc06b930 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.399+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1981342703 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6dc06b930 msgr2=0x7ff6dc0ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.399+0000 7ff6e0f6a700 1 --2- 172.21.15.6:0/1981342703 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6dc06b930 0x7ff6dc0ca280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009840 tx=0x7ff6cc009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:26.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.399+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1981342703 shutdown_connections 2023-11-28T13:00:26.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.399+0000 7ff6e0f6a700 1 --2- 172.21.15.6:0/1981342703 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6dc0cc4b0 0x7ff6dc0cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.400+0000 7ff6e0f6a700 1 --2- 172.21.15.6:0/1981342703 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6dc06b930 0x7ff6dc0ca280 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.400+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1981342703 >> 172.21.15.6:0/1981342703 conn(0x7ff6dc0b1650 msgr2=0x7ff6dc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:26.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.400+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1981342703 shutdown_connections 2023-11-28T13:00:26.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.400+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1981342703 wait complete. 2023-11-28T13:00:26.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.401+0000 7ff6e0f6a700 1 Processor -- start 2023-11-28T13:00:26.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.401+0000 7ff6e0f6a700 1 -- start start 2023-11-28T13:00:26.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6e0f6a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6dc06b930 0x7ff6dc0c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6e0f6a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6dc0cc4b0 0x7ff6dc0c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6e0f6a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6dc15a5a0 0x7ff6dc0b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6e0f6a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc157dd0 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6e0f6a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0c7310 con 0x7ff6dc0cc4b0 2023-11-28T13:00:26.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6e0f6a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0c70b0 con 0x7ff6dc06b930 2023-11-28T13:00:26.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6dad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6dc15a5a0 0x7ff6dc0b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6dad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6dc15a5a0 0x7ff6dc0b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57598/0 (socket says 172.21.15.6:57598) 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6dad9d700 1 -- 172.21.15.6:0/1967949113 learned_addr learned my addr 172.21.15.6:0/1967949113 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6da59c700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6dc06b930 0x7ff6dc0c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6d9d9b700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6dc0cc4b0 0x7ff6dc0c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6dad9d700 1 -- 172.21.15.6:0/1967949113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6dc06b930 msgr2=0x7ff6dc0c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6dad9d700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6dc06b930 0x7ff6dc0c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6dad9d700 1 -- 172.21.15.6:0/1967949113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6dc0cc4b0 msgr2=0x7ff6dc0c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6dad9d700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6dc0cc4b0 0x7ff6dc0c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6dad9d700 1 -- 172.21.15.6:0/1967949113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc0094a0 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6d9d9b700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6dc0cc4b0 0x7ff6dc0c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:26.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6dad9d700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6dc15a5a0 0x7ff6dc0b4c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400efc0 tx=0x7ff6d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6caffd700 1 -- 172.21.15.6:0/1967949113 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400cd40 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6caffd700 1 -- 172.21.15.6:0/1967949113 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d400f040 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1967949113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc0cac40 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.402+0000 7ff6caffd700 1 -- 172.21.15.6:0/1967949113 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40146c0 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.403+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1967949113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc0ca960 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.403+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1967949113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6dc0caf20 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.404+0000 7ff6caffd700 1 -- 172.21.15.6:0/1967949113 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff6d4014820 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.404+0000 7ff6caffd700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6bc0682f0 0x7ff6bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.404+0000 7ff6caffd700 1 -- 172.21.15.6:0/1967949113 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff6d4086ac0 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.404+0000 7ff6da59c700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6bc0682f0 0x7ff6bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.405+0000 7ff6da59c700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6bc0682f0 0x7ff6bc06a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005910 tx=0x7ff6d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.408+0000 7ff6caffd700 1 -- 172.21.15.6:0/1967949113 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff6d4052fb0 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.503 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:00:26.505 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:00:26.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.606+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1967949113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6dc0675a0 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.607+0000 7ff6caffd700 1 -- 172.21.15.6:0/1967949113 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7ff6d400cea0 con 0x7ff6dc15a5a0 2023-11-28T13:00:26.609 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:26.609 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:26.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1967949113 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6bc0682f0 msgr2=0x7ff6bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6bc0682f0 0x7ff6bc06a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0005910 tx=0x7ff6d0009040 comp rx=0 tx=0).stop 2023-11-28T13:00:26.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1967949113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6dc15a5a0 msgr2=0x7ff6dc0b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6dc15a5a0 0x7ff6dc0b4c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400efc0 tx=0x7ff6d400c3a0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1967949113 shutdown_connections 2023-11-28T13:00:26.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6dc15a5a0 0x7ff6dc0b4c40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6bc0682f0 0x7ff6bc06a7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6dc06b930 0x7ff6dc0c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 --2- 172.21.15.6:0/1967949113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6dc0cc4b0 0x7ff6dc0c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1967949113 >> 172.21.15.6:0/1967949113 conn(0x7ff6dc0b1650 msgr2=0x7ff6dc15ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:26.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1967949113 shutdown_connections 2023-11-28T13:00:26.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.609+0000 7ff6e0f6a700 1 -- 172.21.15.6:0/1967949113 wait complete. 2023-11-28T13:00:26.623 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:26.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.746+0000 7f6ef1a8a700 1 Processor -- start 2023-11-28T13:00:26.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.746+0000 7f6ef1a8a700 1 -- start start 2023-11-28T13:00:26.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.747+0000 7f6ef1a8a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6eec0b2660 0x7f6eec0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.747+0000 7f6ef1a8a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eec0b30b0 con 0x7f6eec15c560 2023-11-28T13:00:26.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.747+0000 7f6ef1a8a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eec06c020 con 0x7f6eec0cecf0 2023-11-28T13:00:26.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.747+0000 7f6ef1a8a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eec0cf110 con 0x7f6eec0b2660 2023-11-28T13:00:26.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.747+0000 7f6eeb7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6eec15c560 0x7f6eec06f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44922/0 (socket says 172.21.15.6:44922) 2023-11-28T13:00:26.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.747+0000 7f6eeb7fe700 1 -- 172.21.15.6:0/1787632361 learned_addr learned my addr 172.21.15.6:0/1787632361 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:26.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eeaffd700 1 --2- 172.21.15.6:0/1787632361 >> v2:172.21.15.61:3300/0 conn(0x7f6eec0b2660 0x7f6eec0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889145792 0 0) 0x7f6eec0b30b0 con 0x7f6eec15c560 2023-11-28T13:00:26.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed80035b0 con 0x7f6eec15c560 2023-11-28T13:00:26.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2227010707 0 0) 0x7f6ed80035b0 con 0x7f6eec15c560 2023-11-28T13:00:26.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 >> v2:172.21.15.61:3300/0 conn(0x7f6eec0b2660 msgr2=0x7f6eec0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:26.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 --2- 172.21.15.6:0/1787632361 >> v2:172.21.15.61:3300/0 conn(0x7f6eec0b2660 0x7f6eec0ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 >> v1:172.21.15.61:6789/0 conn(0x7f6eec0cecf0 legacy=0x7f6eec06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec0d09c0 con 0x7f6eec15c560 2023-11-28T13:00:26.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f6edc001670 con 0x7f6eec15c560 2023-11-28T13:00:26.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 >> v1:172.21.15.6:6789/0 conn(0x7f6eec15c560 legacy=0x7f6eec06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed8003a00 0x7f6ed8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed8004850 0x7f6ed8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ed8005200 0x7f6ed800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0cf110 con 0x7f6ed8003a00 2023-11-28T13:00:26.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed800c110 con 0x7f6ed8005200 2023-11-28T13:00:26.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed800c2d0 con 0x7f6ed8004850 2023-11-28T13:00:26.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eeb7fe700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed8003a00 0x7f6ed8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eebfff700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ed8005200 0x7f6ed800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eeaffd700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed8004850 0x7f6ed8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eeb7fe700 1 -- 172.21.15.6:0/1787632361 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed8004850 msgr2=0x7f6ed8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:26.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eeb7fe700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed8004850 0x7f6ed8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eeb7fe700 1 -- 172.21.15.6:0/1787632361 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ed8005200 msgr2=0x7f6ed800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:26.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eeb7fe700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6ed8005200 0x7f6ed800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eebfff700 1 -- 172.21.15.6:0/1787632361 reap_dead start 2023-11-28T13:00:26.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.748+0000 7f6eeb7fe700 1 -- 172.21.15.6:0/1787632361 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80035b0 con 0x7f6ed8003a00 2023-11-28T13:00:26.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6eeb7fe700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed8003a00 0x7f6ed8003e10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c240 tx=0x7f6ed8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df9249d87c2849b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc007c10 con 0x7f6ed8003a00 2023-11-28T13:00:26.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc00cde0 con 0x7f6ed8003a00 2023-11-28T13:00:26.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6eea7fc700 1 -- 172.21.15.6:0/1787632361 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc005ba0 con 0x7f6ed8003a00 2023-11-28T13:00:26.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1787632361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed8003a00 msgr2=0x7f6ed8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6ef1a8a700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed8003a00 0x7f6ed8003e10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c240 tx=0x7f6ed8005880 comp rx=0 tx=0).stop 2023-11-28T13:00:26.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1787632361 shutdown_connections 2023-11-28T13:00:26.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6ef1a8a700 1 --2- 172.21.15.6:0/1787632361 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6ed8003a00 0x7f6ed8003e10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1787632361 >> 172.21.15.6:0/1787632361 conn(0x7f6eec0cd660 msgr2=0x7f6eec0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:26.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1787632361 shutdown_connections 2023-11-28T13:00:26.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.749+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1787632361 wait complete. 2023-11-28T13:00:26.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.750+0000 7f6ef1a8a700 1 Processor -- start 2023-11-28T13:00:26.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.751+0000 7f6ef1a8a700 1 -- start start 2023-11-28T13:00:26.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.751+0000 7f6ef1a8a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed8003a00 0x7f6eec0abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.751+0000 7f6ef1a8a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6eec14d040 0x7f6eec0acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.751+0000 7f6ef1a8a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6eec0c57d0 0x7f6eec0c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.751+0000 7f6ef1a8a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0c5340 con 0x7f6eec0c57d0 2023-11-28T13:00:26.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.751+0000 7f6ef1a8a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0c4770 con 0x7f6eec14d040 2023-11-28T13:00:26.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.751+0000 7f6ef1a8a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0c4a10 con 0x7f6ed8003a00 2023-11-28T13:00:26.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.751+0000 7f6eebfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6eec0c57d0 0x7f6eec0c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.751+0000 7f6eebfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6eec0c57d0 0x7f6eec0c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57618/0 (socket says 172.21.15.6:57618) 2023-11-28T13:00:26.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.751+0000 7f6eebfff700 1 -- 172.21.15.6:0/1263713185 learned_addr learned my addr 172.21.15.6:0/1263713185 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:26.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6eeb7fe700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed8003a00 0x7f6eec0abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6eeaffd700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6eec14d040 0x7f6eec0acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6eebfff700 1 -- 172.21.15.6:0/1263713185 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed8003a00 msgr2=0x7f6eec0abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6eebfff700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed8003a00 0x7f6eec0abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6eebfff700 1 -- 172.21.15.6:0/1263713185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6eec14d040 msgr2=0x7f6eec0acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6eebfff700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6eec14d040 0x7f6eec0acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6eebfff700 1 -- 172.21.15.6:0/1263713185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec0b2dd0 con 0x7f6eec0c57d0 2023-11-28T13:00:26.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6eebfff700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6eec0c57d0 0x7f6eec0c5c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0d00b0 tx=0x7f6ee400d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6ee3fff700 1 -- 172.21.15.6:0/1263713185 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4003cd0 con 0x7f6eec0c57d0 2023-11-28T13:00:26.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6ee3fff700 1 -- 172.21.15.6:0/1263713185 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee4003e70 con 0x7f6eec0c57d0 2023-11-28T13:00:26.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1263713185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eec0b2af0 con 0x7f6eec0c57d0 2023-11-28T13:00:26.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6ee3fff700 1 -- 172.21.15.6:0/1263713185 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee400d9f0 con 0x7f6eec0c57d0 2023-11-28T13:00:26.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.752+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1263713185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eec06bcb0 con 0x7f6eec0c57d0 2023-11-28T13:00:26.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.753+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1263713185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eec0c54b0 con 0x7f6eec0c57d0 2023-11-28T13:00:26.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.754+0000 7f6ee3fff700 1 -- 172.21.15.6:0/1263713185 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6ee400db50 con 0x7f6eec0c57d0 2023-11-28T13:00:26.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.755+0000 7f6ee3fff700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6ecc0682f0 0x7f6ecc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:26.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.755+0000 7f6ee3fff700 1 -- 172.21.15.6:0/1263713185 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6ee4086d20 con 0x7f6eec0c57d0 2023-11-28T13:00:26.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.755+0000 7f6eeb7fe700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6ecc0682f0 0x7f6ecc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:26.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.755+0000 7f6eeb7fe700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6ecc0682f0 0x7f6ecc06a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6edc015040 tx=0x7f6edc007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:26.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.758+0000 7f6ee3fff700 1 -- 172.21.15.6:0/1263713185 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6ee40532d0 con 0x7f6eec0c57d0 2023-11-28T13:00:26.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.951+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1263713185 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6eec0675a0 con 0x7f6eec0c57d0 2023-11-28T13:00:26.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.952+0000 7f6ee3fff700 1 -- 172.21.15.6:0/1263713185 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f6ee40568f0 con 0x7f6eec0c57d0 2023-11-28T13:00:26.954 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:26.954 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1263713185 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6ecc0682f0 msgr2=0x7f6ecc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6ecc0682f0 0x7f6ecc06a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6edc015040 tx=0x7f6edc007000 comp rx=0 tx=0).stop 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1263713185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6eec0c57d0 msgr2=0x7f6eec0c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6eec0c57d0 0x7f6eec0c5c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0d00b0 tx=0x7f6ee400d150 comp rx=0 tx=0).stop 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1263713185 shutdown_connections 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6eec0c57d0 0x7f6eec0c5c40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6ecc0682f0 0x7f6ecc06a7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6ed8003a00 0x7f6eec0abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 --2- 172.21.15.6:0/1263713185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6eec14d040 0x7f6eec0acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1263713185 >> 172.21.15.6:0/1263713185 conn(0x7f6eec0cd660 msgr2=0x7f6eec0b4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:26.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1263713185 shutdown_connections 2023-11-28T13:00:26.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:26.954+0000 7f6ef1a8a700 1 -- 172.21.15.6:0/1263713185 wait complete. 2023-11-28T13:00:26.982 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:00:27.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.105+0000 7f4be9025700 1 Processor -- start 2023-11-28T13:00:27.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.105+0000 7f4be9025700 1 -- start start 2023-11-28T13:00:27.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.106+0000 7f4be9025700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41505b0 0x7f4be40ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.106+0000 7f4be9025700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be406c140 con 0x7f4be40cb470 2023-11-28T13:00:27.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.106+0000 7f4be9025700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be40c9ab0 con 0x7f4be40cb0a0 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.106+0000 7f4be9025700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be40caad0 con 0x7f4be41505b0 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.106+0000 7f4be259c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f4be40cb470 0x7f4be414fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44944/0 (socket says 172.21.15.6:44944) 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.106+0000 7f4be259c700 1 -- 172.21.15.6:0/3031066733 learned_addr learned my addr 172.21.15.6:0/3031066733 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709589933 0 0) 0x7f4be406c140 con 0x7f4be40cb470 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bcc0035b0 con 0x7f4be40cb470 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be359e700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41505b0 0x7f4be40ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2524329101 0 0) 0x7f4bcc0035b0 con 0x7f4be40cb470 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41505b0 msgr2=0x7f4be40ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41505b0 0x7f4be40ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 >> v1:172.21.15.61:6789/0 conn(0x7f4be40cb0a0 legacy=0x7f4be415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40c9d90 con 0x7f4be40cb470 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f4bd8001670 con 0x7f4be40cb470 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 >> v1:172.21.15.6:6789/0 conn(0x7f4be40cb470 legacy=0x7f4be414fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41505b0 0x7f4bcc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bcc003fb0 0x7f4bcc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bcc004d90 0x7f4bcc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc0053b0 con 0x7f4bcc004d90 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc005520 con 0x7f4bcc003fb0 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc0056e0 con 0x7f4be41505b0 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be359e700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bcc004d90 0x7f4bcc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be2d9d700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41505b0 0x7f4bcc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be259c700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bcc003fb0 0x7f4bcc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be359e700 1 -- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41505b0 msgr2=0x7f4bcc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be359e700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41505b0 0x7f4bcc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be359e700 1 -- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bcc003fb0 msgr2=0x7f4bcc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be359e700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4bcc003fb0 0x7f4bcc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.107+0000 7f4be359e700 1 -- 172.21.15.6:0/3031066733 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc0035b0 con 0x7f4bcc004d90 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be359e700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bcc004d90 0x7f4bcc00ba10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009fc0 tx=0x7f4bcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a96aa19489aa1fc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00ec30 con 0x7f4bcc004d90 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc00f040 con 0x7f4bcc004d90 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be1d9b700 1 -- 172.21.15.6:0/3031066733 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc014480 con 0x7f4bcc004d90 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be9025700 1 -- 172.21.15.6:0/3031066733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bcc004d90 msgr2=0x7f4bcc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be9025700 1 --2- 172.21.15.6:0/3031066733 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bcc004d90 0x7f4bcc00ba10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009fc0 tx=0x7f4bcc005880 comp rx=0 tx=0).stop 2023-11-28T13:00:27.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be359e700 1 -- 172.21.15.6:0/3031066733 reap_dead start 2023-11-28T13:00:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be9025700 1 -- 172.21.15.6:0/3031066733 shutdown_connections 2023-11-28T13:00:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be9025700 1 -- 172.21.15.6:0/3031066733 >> 172.21.15.6:0/3031066733 conn(0x7f4be40b0dd0 msgr2=0x7f4be40b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be9025700 1 -- 172.21.15.6:0/3031066733 shutdown_connections 2023-11-28T13:00:27.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.108+0000 7f4be9025700 1 -- 172.21.15.6:0/3031066733 wait complete. 2023-11-28T13:00:27.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.110+0000 7f4be9025700 1 Processor -- start 2023-11-28T13:00:27.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.110+0000 7f4be9025700 1 -- start start 2023-11-28T13:00:27.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.110+0000 7f4be9025700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4be41f0850 0x7f4be40c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.110+0000 7f4be9025700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4be41f0c20 0x7f4be40c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.110+0000 7f4be9025700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41f10d0 0x7f4be41f1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.110+0000 7f4be9025700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be40c8290 con 0x7f4be41f0850 2023-11-28T13:00:27.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.110+0000 7f4be9025700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be41f1a90 con 0x7f4be41f0c20 2023-11-28T13:00:27.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.110+0000 7f4be9025700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be40c7610 con 0x7f4be41f10d0 2023-11-28T13:00:27.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4be41f0850 0x7f4be40c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4be41f0850 0x7f4be40c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57634/0 (socket says 172.21.15.6:57634) 2023-11-28T13:00:27.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be2d9d700 1 -- 172.21.15.6:0/3481651218 learned_addr learned my addr 172.21.15.6:0/3481651218 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:27.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be359e700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41f10d0 0x7f4be41f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be259c700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4be41f0c20 0x7f4be40c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be2d9d700 1 -- 172.21.15.6:0/3481651218 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41f10d0 msgr2=0x7f4be41f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be2d9d700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41f10d0 0x7f4be41f1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be2d9d700 1 -- 172.21.15.6:0/3481651218 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4be41f0c20 msgr2=0x7f4be40c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be2d9d700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4be41f0c20 0x7f4be40c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be2d9d700 1 -- 172.21.15.6:0/3481651218 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40ca590 con 0x7f4be41f0850 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be2d9d700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4be41f0850 0x7f4be40c57a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400f820 tx=0x7f4bd400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4bcb7fe700 1 -- 172.21.15.6:0/3481651218 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40052e0 con 0x7f4be41f0850 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4bcb7fe700 1 -- 172.21.15.6:0/3481651218 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd4005440 con 0x7f4be41f0850 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be9025700 1 -- 172.21.15.6:0/3481651218 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be40ca070 con 0x7f4be41f0850 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4bcb7fe700 1 -- 172.21.15.6:0/3481651218 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40038c0 con 0x7f4be41f0850 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.111+0000 7f4be9025700 1 -- 172.21.15.6:0/3481651218 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be40ca720 con 0x7f4be41f0850 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.112+0000 7f4be9025700 1 -- 172.21.15.6:0/3481651218 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be40675a0 con 0x7f4be41f0850 2023-11-28T13:00:27.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.113+0000 7f4bcb7fe700 1 -- 172.21.15.6:0/3481651218 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4bd4003a20 con 0x7f4be41f0850 2023-11-28T13:00:27.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.113+0000 7f4bcb7fe700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4bd00682f0 0x7f4bd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.113+0000 7f4bcb7fe700 1 -- 172.21.15.6:0/3481651218 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4bd4085df0 con 0x7f4be41f0850 2023-11-28T13:00:27.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.113+0000 7f4be259c700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4bd00682f0 0x7f4bd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.113+0000 7f4be259c700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4bd00682f0 0x7f4bd006a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8001200 tx=0x7f4bd8001150 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.117+0000 7f4bcb7fe700 1 -- 172.21.15.6:0/3481651218 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4bd4052360 con 0x7f4be41f0850 2023-11-28T13:00:27.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.356+0000 7f4be9025700 1 -- 172.21.15.6:0/3481651218 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4be406bcd0 con 0x7f4be41f0850 2023-11-28T13:00:27.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.356+0000 7f4bcb7fe700 1 -- 172.21.15.6:0/3481651218 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4638 (secure 0 0 0) 0x7f4bd4003cb0 con 0x7f4be41f0850 2023-11-28T13:00:27.358 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:27.358 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6834/1568686201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":1568686201},{"type":"v1","addr":"172.21.15.61:6834","nonce":1568686201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/3333487635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6833","nonce":3333487635},{"type":"v1","addr":"172.21.15.61:6835","nonce":3333487635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3917075730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3917075730},{"type":"v1","addr":"172.21.15.6:6835","nonce":3917075730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:21.404170+0000","modified":"2023-11-28T13:00:25.865537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/1668558371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1668558371},{"type":"v1","addr":"172.21.15.6:6837","nonce":1668558371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T13:00:27.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.358+0000 7f4be9025700 1 -- 172.21.15.6:0/3481651218 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4bd00682f0 msgr2=0x7f4bd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.358+0000 7f4be9025700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4bd00682f0 0x7f4bd006a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8001200 tx=0x7f4bd8001150 comp rx=0 tx=0).stop 2023-11-28T13:00:27.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.358+0000 7f4be9025700 1 -- 172.21.15.6:0/3481651218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4be41f0850 msgr2=0x7f4be40c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.358+0000 7f4be9025700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4be41f0850 0x7f4be40c57a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400f820 tx=0x7f4bd400fb30 comp rx=0 tx=0).stop 2023-11-28T13:00:27.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.358+0000 7f4be9025700 1 -- 172.21.15.6:0/3481651218 shutdown_connections 2023-11-28T13:00:27.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.358+0000 7f4be9025700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4bd00682f0 0x7f4bd006a7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.358+0000 7f4be9025700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4be41f10d0 0x7f4be41f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.358+0000 7f4be9025700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4be41f0850 0x7f4be40c57a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.359+0000 7f4be9025700 1 --2- 172.21.15.6:0/3481651218 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4be41f0c20 0x7f4be40c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.359+0000 7f4be9025700 1 -- 172.21.15.6:0/3481651218 >> 172.21.15.6:0/3481651218 conn(0x7f4be40b0dd0 msgr2=0x7f4be40af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.359+0000 7f4be9025700 1 -- 172.21.15.6:0/3481651218 shutdown_connections 2023-11-28T13:00:27.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.359+0000 7f4be9025700 1 -- 172.21.15.6:0/3481651218 wait complete. 2023-11-28T13:00:27.361 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 7 2023-11-28T13:00:27.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:00:21.404170+0000', 'modified': '2023-11-28T13:00:25.865537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6837/1668558371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6836', 'nonce': 1668558371}, {'type': 'v1', 'addr': '172.21.15.6:6837', 'nonce': 1668558371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-28T13:00:27.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:00:27.775 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T13:00:27.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.904+0000 7fdf47359700 1 Processor -- start 2023-11-28T13:00:27.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.904+0000 7fdf47359700 1 -- start start 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf47359700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdf4006d9d0 0x7fdf4015b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf47359700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf4006dda0 0x7fdf4015e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf47359700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4006c3a0 con 0x7fdf4006bc30 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf47359700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4015ee90 con 0x7fdf4006d9d0 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf47359700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4006c0c0 con 0x7fdf4006dda0 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf448f4700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fdf4006bc30 0x7fdf40150eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:44962/0 (socket says 172.21.15.6:44962) 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf448f4700 1 -- 172.21.15.6:0/1641112306 learned_addr learned my addr 172.21.15.6:0/1641112306 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf450f5700 1 --2- 172.21.15.6:0/1641112306 >> v2:172.21.15.61:3300/0 conn(0x7fdf4006d9d0 0x7fdf4015b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788685390 0 0) 0x7fdf4006c3a0 con 0x7fdf4006bc30 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf458f6700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf4006dda0 0x7fdf4015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.905+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf2c0035b0 con 0x7fdf4006bc30 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 358377418 0 0) 0x7fdf2c0035b0 con 0x7fdf4006bc30 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf4006dda0 msgr2=0x7fdf4015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf4006dda0 0x7fdf4015e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 >> v2:172.21.15.61:3300/0 conn(0x7fdf4006d9d0 msgr2=0x7fdf4015b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 --2- 172.21.15.6:0/1641112306 >> v2:172.21.15.61:3300/0 conn(0x7fdf4006d9d0 0x7fdf4015b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4015ab60 con 0x7fdf4006bc30 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fdf30001670 con 0x7fdf4006bc30 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 >> v1:172.21.15.6:6789/0 conn(0x7fdf4006bc30 legacy=0x7fdf40150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 0x7fdf2c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf2c004430 0x7fdf2c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf4006dda0 0x7fdf2c004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4015ee90 con 0x7fdf2c003a00 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4006c0c0 con 0x7fdf2c004430 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c00ba10 con 0x7fdf4006dda0 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf450f5700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 0x7fdf2c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf448f4700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf2c004430 0x7fdf2c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf458f6700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf4006dda0 0x7fdf2c004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf450f5700 1 -- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf4006dda0 msgr2=0x7fdf2c004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf450f5700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf4006dda0 0x7fdf2c004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf450f5700 1 -- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf2c004430 msgr2=0x7fdf2c0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf450f5700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf2c004430 0x7fdf2c0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf450f5700 1 -- 172.21.15.6:0/1641112306 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0035b0 con 0x7fdf2c003a00 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf458f6700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf4006dda0 0x7fdf2c004fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf448f4700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf2c004430 0x7fdf2c0048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf450f5700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 0x7fdf2c003e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009b60 tx=0x7fdf2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0ebbe352697fb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00e6c0 con 0x7fdf2c003a00 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c00e820 con 0x7fdf2c003a00 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.906+0000 7fdf37fff700 1 -- 172.21.15.6:0/1641112306 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00eb90 con 0x7fdf2c003a00 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.907+0000 7fdf47359700 1 -- 172.21.15.6:0/1641112306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 msgr2=0x7fdf2c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.907+0000 7fdf47359700 1 --2- 172.21.15.6:0/1641112306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 0x7fdf2c003e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009b60 tx=0x7fdf2c005880 comp rx=0 tx=0).stop 2023-11-28T13:00:27.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.907+0000 7fdf458f6700 1 -- 172.21.15.6:0/1641112306 reap_dead start 2023-11-28T13:00:27.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.907+0000 7fdf47359700 1 -- 172.21.15.6:0/1641112306 shutdown_connections 2023-11-28T13:00:27.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.907+0000 7fdf47359700 1 -- 172.21.15.6:0/1641112306 >> 172.21.15.6:0/1641112306 conn(0x7fdf400b0dd0 msgr2=0x7fdf400b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.907+0000 7fdf47359700 1 -- 172.21.15.6:0/1641112306 shutdown_connections 2023-11-28T13:00:27.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.907+0000 7fdf47359700 1 -- 172.21.15.6:0/1641112306 wait complete. 2023-11-28T13:00:27.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.908+0000 7fdf47359700 1 Processor -- start 2023-11-28T13:00:27.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.908+0000 7fdf47359700 1 -- start start 2023-11-28T13:00:27.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf47359700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 0x7fdf401f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf47359700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf401f31e0 0x7fdf400c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf47359700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf401e3ce0 0x7fdf401e4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf47359700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf401e46a0 con 0x7fdf2c003a00 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf47359700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400c3ab0 con 0x7fdf401f31e0 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf47359700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400c3de0 con 0x7fdf401e3ce0 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf450f5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 0x7fdf401f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf450f5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 0x7fdf401f03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57654/0 (socket says 172.21.15.6:57654) 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf450f5700 1 -- 172.21.15.6:0/2576404926 learned_addr learned my addr 172.21.15.6:0/2576404926 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf458f6700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf401e3ce0 0x7fdf401e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf448f4700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf401f31e0 0x7fdf400c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf450f5700 1 -- 172.21.15.6:0/2576404926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf401e3ce0 msgr2=0x7fdf401e4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf450f5700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf401e3ce0 0x7fdf401e4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf450f5700 1 -- 172.21.15.6:0/2576404926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf401f31e0 msgr2=0x7fdf400c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf450f5700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf401f31e0 0x7fdf400c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf450f5700 1 -- 172.21.15.6:0/2576404926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4015b120 con 0x7fdf2c003a00 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.909+0000 7fdf448f4700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf401f31e0 0x7fdf400c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.910+0000 7fdf450f5700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 0x7fdf401f03a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0056d0 tx=0x7fdf3c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.910+0000 7fdf35ffb700 1 -- 172.21.15.6:0/2576404926 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c021070 con 0x7fdf2c003a00 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.910+0000 7fdf35ffb700 1 -- 172.21.15.6:0/2576404926 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c004520 con 0x7fdf2c003a00 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.910+0000 7fdf47359700 1 -- 172.21.15.6:0/2576404926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4015ae40 con 0x7fdf2c003a00 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.910+0000 7fdf35ffb700 1 -- 172.21.15.6:0/2576404926 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c00e6c0 con 0x7fdf2c003a00 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.910+0000 7fdf47359700 1 -- 172.21.15.6:0/2576404926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4006fac0 con 0x7fdf2c003a00 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.911+0000 7fdf35ffb700 1 -- 172.21.15.6:0/2576404926 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdf3c02e030 con 0x7fdf2c003a00 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.912+0000 7fdf35ffb700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf240682f0 0x7fdf2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.912+0000 7fdf35ffb700 1 -- 172.21.15.6:0/2576404926 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdf3c087110 con 0x7fdf2c003a00 2023-11-28T13:00:27.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.912+0000 7fdf448f4700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf240682f0 0x7fdf2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.912+0000 7fdf448f4700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf240682f0 0x7fdf2406a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf30000fb0 tx=0x7fdf30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.914+0000 7fdf534e2b80 1 Processor -- start 2023-11-28T13:00:27.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.914+0000 7fdf534e2b80 1 -- start start 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e8886a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 0x55d8e88b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e8770bc0 con 0x55d8e88b1120 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e87df350 con 0x55d8e88b0810 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8e87d8880 con 0x55d8e88ae130 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf458f6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf458f6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57666/0 (socket says 172.21.15.6:57666) 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf458f6700 1 -- 172.21.15.6:0/3175740196 learned_addr learned my addr 172.21.15.6:0/3175740196 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf448f4700 1 --2- 172.21.15.6:0/3175740196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 0x55d8e88b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf450f5700 1 --2- 172.21.15.6:0/3175740196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e8886a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf458f6700 1 -- 172.21.15.6:0/3175740196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 msgr2=0x55d8e88b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf458f6700 1 --2- 172.21.15.6:0/3175740196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 0x55d8e88b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf458f6700 1 -- 172.21.15.6:0/3175740196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 msgr2=0x55d8e8886a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf458f6700 1 --2- 172.21.15.6:0/3175740196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e8886a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf458f6700 1 -- 172.21.15.6:0/3175740196 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8e865a5e0 con 0x55d8e88b1120 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf458f6700 1 --2- 172.21.15.6:0/3175740196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88cc770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf400c2de0 tx=0x7fdf38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d0d1875395f542 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.915+0000 7fdf2affd700 1 -- 172.21.15.6:0/3175740196 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3800e070 con 0x55d8e88b1120 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf2affd700 1 -- 172.21.15.6:0/3175740196 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3800f040 con 0x55d8e88b1120 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf2affd700 1 -- 172.21.15.6:0/3175740196 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38007ed0 con 0x55d8e88b1120 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3175740196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 msgr2=0x55d8e88cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/3175740196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88cc770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf400c2de0 tx=0x7fdf38009750 comp rx=0 tx=0).stop 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3175740196 shutdown_connections 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/3175740196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88cc770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/3175740196 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e8886a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/3175740196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 0x55d8e88b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3175740196 >> 172.21.15.6:0/3175740196 conn(0x55d8e8883d60 msgr2=0x55d8e88846f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3175740196 shutdown_connections 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.916+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3175740196 wait complete. 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.917+0000 7fdf534e2b80 1 -- start start 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.917+0000 7fdf534e2b80 1 Processor -- start 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88ae130 0x55d8e88d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e88a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88b1120 0x55d8e88a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e865c8d0 con 0x55d8e88ae130 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e88d1200 con 0x55d8e88b0810 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8e88d3d10 con 0x55d8e88b1120 2023-11-28T13:00:27.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf534e2b80 10 client.0 _ll_register_callbacks cb 0x55d8e867e240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf450f5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88ae130 0x55d8e88d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf450f5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88ae130 0x55d8e88d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57668/0 (socket says 172.21.15.6:57668) 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf450f5700 1 -- 172.21.15.6:0/1015042589 learned_addr learned my addr 172.21.15.6:0/1015042589 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf458f6700 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88b1120 0x55d8e88a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf448f4700 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e88a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf450f5700 1 -- 172.21.15.6:0/1015042589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88b1120 msgr2=0x55d8e88a92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf450f5700 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88b1120 0x55d8e88a92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf450f5700 1 -- 172.21.15.6:0/1015042589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 msgr2=0x55d8e88a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf450f5700 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e88a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.918+0000 7fdf450f5700 1 -- 172.21.15.6:0/1015042589 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf38009e50 con 0x55d8e88ae130 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf450f5700 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88ae130 0x55d8e88d34c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c05eea0 tx=0x7fdf3c056d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bc5c0cb38e5358 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf1ffff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:00:27.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf1ffff700 1 -- 172.21.15.6:0/1015042589 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c05f140 con 0x55d8e88ae130 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf1ffff700 1 -- 172.21.15.6:0/1015042589 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdf3c0562a0 con 0x55d8e88ae130 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf534e2b80 1 -- 172.21.15.6:0/1015042589 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8e87a6140 con 0x55d8e88ae130 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf1ffff700 1 -- 172.21.15.6:0/1015042589 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf3c0850d0 con 0x55d8e88ae130 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf534e2b80 10 client.4443 fetch_fsmap learned FSMap version 7 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf534e2b80 1 -- 172.21.15.6:0/1015042589 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55d8e8606d90 con 0x55d8e88ae130 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf1ffff700 1 -- 172.21.15.6:0/1015042589 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8e87a6140 con 0x55d8e88ae130 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf1ffff700 1 -- 172.21.15.6:0/1015042589 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdf3c056460 con 0x55d8e88ae130 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf534e2b80 10 client.4443 fetch_fsmap finished waiting for FSMap version 7 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf534e2b80 10 client.4443 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf534e2b80 20 client.4443 populate_metadata read hostname 'smithi006' 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x55d8e88b3fa0 0x55d8e88b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf534e2b80 4 client.4443 mds_command: new command op to 4287 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:00:27.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf534e2b80 1 -- 172.21.15.6:0/1015042589 --> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8e865cf10 con 0x55d8e88b3fa0 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.919+0000 7fdf448f4700 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x55d8e88b3fa0 0x55d8e88b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf1ffff700 10 client.4443 ms_handle_connect on v2:172.21.15.6:6836/1668558371 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf448f4700 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x55d8e88b3fa0 0x55d8e88b4370 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d062dc26533ebcdf server_cookie=71e6f34f9b45c08f in_seq=0 out_seq=0 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf1ffff700 1 -- 172.21.15.6:0/1015042589 <== mds.0 v2:172.21.15.6:6836/1668558371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55d8e865cf10 con 0x55d8e88b3fa0 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf1ffff700 10 client.4443 handle_command_reply: tid=0 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf534e2b80 1 client.4443 shutdown 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf534e2b80 1 -- 172.21.15.6:0/1015042589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88ae130 msgr2=0x55d8e88d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88ae130 0x55d8e88d34c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c05eea0 tx=0x7fdf3c056d90 comp rx=0 tx=0).stop 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf534e2b80 1 -- 172.21.15.6:0/1015042589 shutdown_connections 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88b1120 0x55d8e88a92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88ae130 0x55d8e88d34c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x55d8e88b3fa0 0x55d8e88b4370 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/1015042589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e88a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf1ffff700 0 client.4443 ms_handle_reset on v2:172.21.15.6:6836/1668558371 2023-11-28T13:00:27.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.920+0000 7fdf534e2b80 1 -- 172.21.15.6:0/1015042589 >> 172.21.15.6:0/1015042589 conn(0x55d8e8883d60 msgr2=0x55d8e8886990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.921+0000 7fdf534e2b80 1 -- 172.21.15.6:0/1015042589 shutdown_connections 2023-11-28T13:00:27.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.921+0000 7fdf534e2b80 1 -- 172.21.15.6:0/1015042589 wait complete. 2023-11-28T13:00:27.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.921+0000 7fdf534e2b80 20 client.4443 trim_cache size 0 max 16384 2023-11-28T13:00:27.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.938+0000 7fdf534e2b80 1 Processor -- start 2023-11-28T13:00:27.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.939+0000 7fdf534e2b80 1 -- start start 2023-11-28T13:00:27.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.939+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 0x55d8e88a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.939+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e8929d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.939+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e892a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.939+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e88ad6c0 con 0x55d8e88b1120 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.939+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e88cf0a0 con 0x55d8e88b0810 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.939+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8e892a790 con 0x55d8e88ae130 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.939+0000 7fdf458f6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e892a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.939+0000 7fdf458f6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e892a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57678/0 (socket says 172.21.15.6:57678) 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.939+0000 7fdf458f6700 1 -- 172.21.15.6:0/2129906597 learned_addr learned my addr 172.21.15.6:0/2129906597 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf448f4700 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e8929d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf450f5700 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 0x55d8e88a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf458f6700 1 -- 172.21.15.6:0/2129906597 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 msgr2=0x55d8e88a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf458f6700 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 0x55d8e88a84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf458f6700 1 -- 172.21.15.6:0/2129906597 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 msgr2=0x55d8e8929d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf458f6700 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e8929d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf458f6700 1 -- 172.21.15.6:0/2129906597 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8e879f140 con 0x55d8e88b1120 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf450f5700 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 0x55d8e88a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf448f4700 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e8929d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf458f6700 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e892a250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55d8e88820d0 tx=0x7fdf38018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c197984c88c9e55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf2b7fe700 1 -- 172.21.15.6:0/2129906597 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38009410 con 0x55d8e88b1120 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf2b7fe700 1 -- 172.21.15.6:0/2129906597 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf38025040 con 0x55d8e88b1120 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf2b7fe700 1 -- 172.21.15.6:0/2129906597 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38007ed0 con 0x55d8e88b1120 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf534e2b80 1 -- 172.21.15.6:0/2129906597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 msgr2=0x55d8e892a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e892a250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55d8e88820d0 tx=0x7fdf38018000 comp rx=0 tx=0).stop 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf534e2b80 1 -- 172.21.15.6:0/2129906597 shutdown_connections 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e892a250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88ae130 0x55d8e88a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/2129906597 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88b0810 0x55d8e8929d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.940+0000 7fdf534e2b80 1 -- 172.21.15.6:0/2129906597 >> 172.21.15.6:0/2129906597 conn(0x55d8e8883d60 msgr2=0x55d8e88ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.941+0000 7fdf534e2b80 1 -- 172.21.15.6:0/2129906597 shutdown_connections 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.941+0000 7fdf534e2b80 1 -- 172.21.15.6:0/2129906597 wait complete. 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.941+0000 7fdf534e2b80 1 -- start start 2023-11-28T13:00:27.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.941+0000 7fdf534e2b80 1 Processor -- start 2023-11-28T13:00:27.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88ae130 0x55d8e887fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88b0810 0x55d8e88e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf534e2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e8931a90 con 0x55d8e88b1120 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e892a070 con 0x55d8e88ae130 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf534e2b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8e892a5a0 con 0x55d8e88b0810 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf534e2b80 10 client.0 _ll_register_callbacks cb 0x55d8e867e240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf458f6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf458f6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57686/0 (socket says 172.21.15.6:57686) 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf458f6700 1 -- 172.21.15.6:0/3077432137 learned_addr learned my addr 172.21.15.6:0/3077432137 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf448f4700 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88b0810 0x55d8e88e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf450f5700 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88ae130 0x55d8e887fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.942+0000 7fdf458f6700 1 -- 172.21.15.6:0/3077432137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88b0810 msgr2=0x55d8e88e4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf458f6700 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88b0810 0x55d8e88e4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf458f6700 1 -- 172.21.15.6:0/3077432137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88ae130 msgr2=0x55d8e887fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf458f6700 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88ae130 0x55d8e887fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf458f6700 1 -- 172.21.15.6:0/3077432137 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8e865a5e0 con 0x55d8e88b1120 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf450f5700 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88ae130 0x55d8e887fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf458f6700 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88d00b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55d8e88ad800 tx=0x7fdf38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e7ca9cc9b042ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf1ffff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf1ffff700 1 -- 172.21.15.6:0/3077432137 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38009410 con 0x55d8e88b1120 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf1ffff700 1 -- 172.21.15.6:0/3077432137 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdf38018b40 con 0x55d8e88b1120 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3077432137 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8e8796160 con 0x55d8e88b1120 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf1ffff700 1 -- 172.21.15.6:0/3077432137 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf38018d40 con 0x55d8e88b1120 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf534e2b80 10 client.4449 fetch_fsmap learned FSMap version 7 2023-11-28T13:00:27.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3077432137 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55d8e862d200 con 0x55d8e88b1120 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf1ffff700 1 -- 172.21.15.6:0/3077432137 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8e8796160 con 0x55d8e88b1120 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf1ffff700 1 -- 172.21.15.6:0/3077432137 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdf38028070 con 0x55d8e88b1120 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf534e2b80 10 client.4449 fetch_fsmap finished waiting for FSMap version 7 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf534e2b80 10 client.4449 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf534e2b80 20 client.4449 populate_metadata read hostname 'smithi006' 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x55d8e8929240 0x55d8e8930cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf534e2b80 4 client.4449 mds_command: new command op to 4287 tid=0[{"prefix": "status"}] 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3077432137 --> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d8e8780170 con 0x55d8e8929240 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.943+0000 7fdf450f5700 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x55d8e8929240 0x55d8e8930cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf1ffff700 10 client.4449 ms_handle_connect on v2:172.21.15.6:6836/1668558371 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf450f5700 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x55d8e8929240 0x55d8e8930cf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f784518ec1ff905d server_cookie=bdee00f9b14f0647 in_seq=0 out_seq=0 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf1ffff700 1 -- 172.21.15.6:0/3077432137 <== mds.0 v2:172.21.15.6:6836/1668558371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+317 (crc 0 0 0) 0x55d8e8780170 con 0x55d8e8929240 2023-11-28T13:00:27.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf1ffff700 10 client.4449 handle_command_reply: tid=0 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 client.4449 shutdown 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3077432137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 msgr2=0x55d8e88d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88d00b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55d8e88ad800 tx=0x7fdf38004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3077432137 shutdown_connections 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d8e88b1120 0x55d8e88d00b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x55d8e8929240 0x55d8e8930cf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d8e88ae130 0x55d8e887fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf1ffff700 0 client.4449 ms_handle_reset on v2:172.21.15.6:6836/1668558371 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 --2- 172.21.15.6:0/3077432137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d8e88b0810 0x55d8e88e4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3077432137 >> 172.21.15.6:0/3077432137 conn(0x55d8e8883d60 msgr2=0x55d8e88d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3077432137 shutdown_connections 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 1 -- 172.21.15.6:0/3077432137 wait complete. 2023-11-28T13:00:27.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.944+0000 7fdf534e2b80 20 client.4449 trim_cache size 0 max 16384 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "dc643718-e974-4d8a-b829-ff6ced554687", 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4287, 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 6.539637473, 2023-11-28T13:00:27.953 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 7, 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 25, 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 24, 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 11.876322835 2023-11-28T13:00:27.954 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 -- 172.21.15.6:0/2576404926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf240682f0 msgr2=0x7fdf2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf240682f0 0x7fdf2406a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf30000fb0 tx=0x7fdf30009040 comp rx=0 tx=0).stop 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 -- 172.21.15.6:0/2576404926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 msgr2=0x7fdf401f03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 0x7fdf401f03a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0056d0 tx=0x7fdf3c0047c0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 -- 172.21.15.6:0/2576404926 shutdown_connections 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf240682f0 0x7fdf2406a7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf401e3ce0 0x7fdf401e4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf2c003a00 0x7fdf401f03a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 --2- 172.21.15.6:0/2576404926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf401f31e0 0x7fdf400c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 -- 172.21.15.6:0/2576404926 >> 172.21.15.6:0/2576404926 conn(0x7fdf400b0dd0 msgr2=0x7fdf400af510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:27.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 -- 172.21.15.6:0/2576404926 shutdown_connections 2023-11-28T13:00:27.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:27.945+0000 7fdf47359700 1 -- 172.21.15.6:0/2576404926 wait complete. 2023-11-28T13:00:27.958 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:00:27.958 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T13:00:27.958 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:00:28.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.081+0000 7f7b9a7cc700 1 Processor -- start 2023-11-28T13:00:28.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.081+0000 7f7b9a7cc700 1 -- start start 2023-11-28T13:00:28.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.082+0000 7f7b9a7cc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7b9415a5a0 0x7f7b9406b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.082+0000 7f7b9a7cc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7b940ca280 0x7f7b940cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.082+0000 7f7b9a7cc700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b940cb570 con 0x7f7b940cbf40 2023-11-28T13:00:28.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.082+0000 7f7b9a7cc700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b940ca6f0 con 0x7f7b940ca280 2023-11-28T13:00:28.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.082+0000 7f7b9a7cc700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b940ca960 con 0x7f7b9415a5a0 2023-11-28T13:00:28.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.082+0000 7f7b937fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7b940ca280 0x7f7b940cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.082+0000 7f7b98d69700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f7b940cbf40 0x7f7b940cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45014/0 (socket says 172.21.15.6:45014) 2023-11-28T13:00:28.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.082+0000 7f7b98d69700 1 -- 172.21.15.6:0/627498497 learned_addr learned my addr 172.21.15.6:0/627498497 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:28.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.082+0000 7f7b93fff700 1 --2- 172.21.15.6:0/627498497 >> v2:172.21.15.61:3300/0 conn(0x7f7b9415a5a0 0x7f7b9406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b937fe700 1 -- 172.21.15.6:0/627498497 >> v2:172.21.15.61:3300/0 conn(0x7f7b9415a5a0 msgr2=0x7f7b9406b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b937fe700 1 --2- 172.21.15.6:0/627498497 >> v2:172.21.15.61:3300/0 conn(0x7f7b9415a5a0 0x7f7b9406b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b937fe700 1 -- 172.21.15.6:0/627498497 >> v1:172.21.15.6:6789/0 conn(0x7f7b940cbf40 legacy=0x7f7b940cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b92ffd700 1 -- 172.21.15.6:0/627498497 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782741342 0 0) 0x7f7b940cb570 con 0x7f7b940cbf40 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b937fe700 1 -- 172.21.15.6:0/627498497 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b940cac40 con 0x7f7b940ca280 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b937fe700 1 --2- 172.21.15.6:0/627498497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b940ca280 0x7f7b940cba00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7b94071570 tx=0x7f7b84009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b35e9b3942e1e9a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b92ffd700 1 -- 172.21.15.6:0/627498497 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84011830 con 0x7f7b940ca280 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b92ffd700 1 -- 172.21.15.6:0/627498497 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b84011990 con 0x7f7b940ca280 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b92ffd700 1 -- 172.21.15.6:0/627498497 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84011d20 con 0x7f7b940ca280 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/627498497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b940ca280 msgr2=0x7f7b940cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b9a7cc700 1 --2- 172.21.15.6:0/627498497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b940ca280 0x7f7b940cba00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7b94071570 tx=0x7f7b84009b20 comp rx=0 tx=0).stop 2023-11-28T13:00:28.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/627498497 shutdown_connections 2023-11-28T13:00:28.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b9a7cc700 1 --2- 172.21.15.6:0/627498497 >> v2:172.21.15.61:3300/0 conn(0x7f7b9415a5a0 0x7f7b9406b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b9a7cc700 1 --2- 172.21.15.6:0/627498497 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b940ca280 0x7f7b940cba00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.083+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/627498497 >> 172.21.15.6:0/627498497 conn(0x7f7b940b1650 msgr2=0x7f7b940b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.084+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/627498497 shutdown_connections 2023-11-28T13:00:28.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.084+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/627498497 wait complete. 2023-11-28T13:00:28.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.085+0000 7f7b9a7cc700 1 Processor -- start 2023-11-28T13:00:28.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.085+0000 7f7b9a7cc700 1 -- start start 2023-11-28T13:00:28.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b9a7cc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b940ca280 0x7f7b940c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b9a7cc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b940cbf40 0x7f7b941e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b9a7cc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b9415a5a0 0x7f7b941e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b9a7cc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b940c6b70 con 0x7f7b9415a5a0 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b9a7cc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b940c6dc0 con 0x7f7b940ca280 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b9a7cc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b940c70a0 con 0x7f7b940cbf40 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b98d69700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b9415a5a0 0x7f7b941e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b98d69700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b9415a5a0 0x7f7b941e43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57700/0 (socket says 172.21.15.6:57700) 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b98d69700 1 -- 172.21.15.6:0/4149095650 learned_addr learned my addr 172.21.15.6:0/4149095650 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b93fff700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b940ca280 0x7f7b940c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b937fe700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b940cbf40 0x7f7b941e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b98d69700 1 -- 172.21.15.6:0/4149095650 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b940cbf40 msgr2=0x7f7b941e3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b98d69700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b940cbf40 0x7f7b941e3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b98d69700 1 -- 172.21.15.6:0/4149095650 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b940ca280 msgr2=0x7f7b940c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b98d69700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b940ca280 0x7f7b940c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b98d69700 1 -- 172.21.15.6:0/4149095650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840094a0 con 0x7f7b9415a5a0 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b98d69700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b9415a5a0 0x7f7b941e43c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00cc10 tx=0x7f7b8c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b90ff9700 1 -- 172.21.15.6:0/4149095650 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00aa60 con 0x7f7b9415a5a0 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b90ff9700 1 -- 172.21.15.6:0/4149095650 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00abc0 con 0x7f7b9415a5a0 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/4149095650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b940cb200 con 0x7f7b9415a5a0 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b90ff9700 1 -- 172.21.15.6:0/4149095650 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0070d0 con 0x7f7b9415a5a0 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.086+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/4149095650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b940caf20 con 0x7f7b9415a5a0 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.087+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/4149095650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9415a970 con 0x7f7b9415a5a0 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.088+0000 7f7b90ff9700 1 -- 172.21.15.6:0/4149095650 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7b8c01a030 con 0x7f7b9415a5a0 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.088+0000 7f7b90ff9700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7b800682f0 0x7f7b8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:28.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.088+0000 7f7b93fff700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7b800682f0 0x7f7b8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:28.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.088+0000 7f7b90ff9700 1 -- 172.21.15.6:0/4149095650 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7b8c085bb0 con 0x7f7b9415a5a0 2023-11-28T13:00:28.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.089+0000 7f7b93fff700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7b800682f0 0x7f7b8006a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7b940b15b0 tx=0x7f7b88009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:28.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.093+0000 7f7b90ff9700 1 -- 172.21.15.6:0/4149095650 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7b8c08c080 con 0x7f7b9415a5a0 2023-11-28T13:00:28.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.331+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/4149095650 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7b940675a0 con 0x7f7b9415a5a0 2023-11-28T13:00:28.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.331+0000 7f7b90ff9700 1 -- 172.21.15.6:0/4149095650 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7b8c018070 con 0x7f7b9415a5a0 2023-11-28T13:00:28.332 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T13:00:28.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/4149095650 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7b800682f0 msgr2=0x7f7b8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7b800682f0 0x7f7b8006a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7b940b15b0 tx=0x7f7b88009210 comp rx=0 tx=0).stop 2023-11-28T13:00:28.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/4149095650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b9415a5a0 msgr2=0x7f7b941e43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:28.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b9415a5a0 0x7f7b941e43c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00cc10 tx=0x7f7b8c00cf20 comp rx=0 tx=0).stop 2023-11-28T13:00:28.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/4149095650 shutdown_connections 2023-11-28T13:00:28.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b9415a5a0 0x7f7b941e43c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b940ca280 0x7f7b940c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7b800682f0 0x7f7b8006a7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 --2- 172.21.15.6:0/4149095650 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b940cbf40 0x7f7b941e3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:28.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/4149095650 >> 172.21.15.6:0/4149095650 conn(0x7f7b940b1650 msgr2=0x7f7b94157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:28.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/4149095650 shutdown_connections 2023-11-28T13:00:28.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:28.333+0000 7f7b9a7cc700 1 -- 172.21.15.6:0/4149095650 wait complete. 2023-11-28T13:00:28.346 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T13:00:28.346 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:00:28.347 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-28T13:00:28.347 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-28T13:00:28.347 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T13:00:28.347 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T13:00:28.347 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:00:28.347 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:00:28.347 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T13:00:28.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:28.348 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-28T13:00:28.365 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:28.365 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:28.366 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T13:00:28.367 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42335sec preferred_lft 42335sec 2023-11-28T13:00:28.367 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2023-11-28T13:00:28.367 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:28.367 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:28.367 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:28.367 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T13:00:28.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:28.368 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:28.368 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:28.368 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2023-11-28T13:00:28.368 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2023-11-28T13:00:28.368 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2023-11-28T13:00:28.368 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T13:00:28.368 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:28.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:28.538 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:00:28.567 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-28T13:00:28.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:28.569 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-28T13:00:28.650 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-28T13:00:28.650 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:00:28.650 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T13:00:28.651 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T13:00:28.651 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:00:28.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:28.652 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:28.652 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:28.652 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T13:00:28.652 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T13:00:28.652 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T13:00:28.653 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:28.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:28.905 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-28T13:00:28.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:28.923 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2023-11-28T13:00:28.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:28.981 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:28.981 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:28.981 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T13:00:28.981 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-28T13:00:28.981 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:29.078 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-28T13:00:29.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:29.079 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:29.079 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:29.079 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-28T13:00:29.079 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T13:00:29.079 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-28T13:00:29.079 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-28T13:00:29.079 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-28T13:00:29.079 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:29.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:29.290 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:29.290 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:29.290 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2023-11-28T13:00:29.290 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T13:00:29.290 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:29.415 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-28T13:00:29.415 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:29.415 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-28T13:00:29.433 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T13:00:29.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:29.434 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-28T13:00:29.491 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:00:29.560 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:00:29.560 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:00:29.627 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:00:29.628 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:00:29.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:29.630 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:00:29.687 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T13:00:29.688 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_mountpoint=/ --client_fs=cephfs) 2023-11-28T13:00:29.730 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:00:29.758 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:00:29.758 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:00:29.824 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:00:29.825 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:00:29.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:29.826 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:00:29.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:2023-11-28T13:00:29.843+0000 7fe135be95c0 -1 init, newargv = 0x55aff0de30a0 newargc=15 2023-11-28T13:00:29.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:ceph-fuse[111395]: starting ceph client 2023-11-28T13:00:29.850 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:ceph-fuse[111395]: starting fuse 2023-11-28T13:00:29.862 INFO:teuthology.orchestra.run.smithi006.stdout:47 2023-11-28T13:00:29.862 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T13:00:29.862 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:00:29.862 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2023-11-28T13:00:29.863 DEBUG:teuthology.orchestra.run.smithi006:> import re 2023-11-28T13:00:29.863 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:00:29.863 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2023-11-28T13:00:29.863 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:29.863 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2023-11-28T13:00:29.863 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-28T13:00:29.863 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2023-11-28T13:00:29.864 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2023-11-28T13:00:29.865 DEBUG:teuthology.orchestra.run.smithi006:> return f 2023-11-28T13:00:29.865 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T13:00:29.865 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:29.865 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.admin")) 2023-11-28T13:00:29.865 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:00:29.988 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.admin.111395.asok 2023-11-28T13:00:30.170 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.111395.asok 2023-11-28T13:00:30.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:30.170 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.111395.asok status 2023-11-28T13:00:30.331 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:00:30.331 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "admin", 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "111395", 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2023-11-28T13:00:30.332 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4461, 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2023-11-28T13:00:30.333 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 4461 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:00:30.334 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2240193963 2023-11-28T13:00:30.335 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:00:30.335 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:30.335 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:00:30.335 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:00:30.335 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:00:30.336 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2240193963 2023-11-28T13:00:30.336 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:30.336 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.4461 192.168.0.1:0/2240193963", 2023-11-28T13:00:30.336 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.1:0/2240193963", 2023-11-28T13:00:30.336 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2023-11-28T13:00:30.336 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 7, 2023-11-28T13:00:30.336 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 25, 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2023-11-28T13:00:30.337 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:00:30.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:30.341 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2023-11-28T13:00:30.342 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-28T13:00:30.359 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2023-11-28T13:00:30.359 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-28T13:00:30.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:30.359 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-28T13:00:30.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:30.428 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-28T13:00:30.524 INFO:teuthology.orchestra.run.smithi006.stdout:# file: . 2023-11-28T13:00:30.524 INFO:teuthology.orchestra.run.smithi006.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-28T13:00:30.524 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:30.525 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:30.525 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:30.543 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:30.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.706+0000 7f6f5e1a6700 1 Processor -- start 2023-11-28T13:00:30.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.706+0000 7f6f5e1a6700 1 -- start start 2023-11-28T13:00:30.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f5e1a6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6f58159590 0x7f6f5806e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f5e1a6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6f5815db70 0x7f6f580ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f5e1a6700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5815d120 con 0x7f6f580ce890 2023-11-28T13:00:30.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f5e1a6700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5815d6e0 con 0x7f6f58159590 2023-11-28T13:00:30.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f5e1a6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f580aef60 con 0x7f6f5815db70 2023-11-28T13:00:30.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f577fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6f58159590 0x7f6f5806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f577fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6f58159590 0x7f6f5806e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57728/0 (socket says 172.21.15.6:57728) 2023-11-28T13:00:30.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f577fe700 1 -- 172.21.15.6:0/4287673831 learned_addr learned my addr 172.21.15.6:0/4287673831 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:30.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f57fff700 1 --1- 172.21.15.6:0/4287673831 >> v1:172.21.15.6:6789/0 conn(0x7f6f580ce890 0x7f6f580caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45050/0 (socket says 172.21.15.6:45050) 2023-11-28T13:00:30.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f56ffd700 1 --2- 172.21.15.6:0/4287673831 >> v2:172.21.15.61:3300/0 conn(0x7f6f5815db70 0x7f6f580ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f577fe700 1 -- 172.21.15.6:0/4287673831 >> v2:172.21.15.61:3300/0 conn(0x7f6f5815db70 msgr2=0x7f6f580ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:30.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f577fe700 1 --2- 172.21.15.6:0/4287673831 >> v2:172.21.15.61:3300/0 conn(0x7f6f5815db70 0x7f6f580ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f567fc700 1 -- 172.21.15.6:0/4287673831 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720183502 0 0) 0x7f6f5815d120 con 0x7f6f580ce890 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f577fe700 1 -- 172.21.15.6:0/4287673831 >> v1:172.21.15.6:6789/0 conn(0x7f6f580ce890 legacy=0x7f6f580caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f577fe700 1 -- 172.21.15.6:0/4287673831 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f580af240 con 0x7f6f58159590 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f56ffd700 1 --2- 172.21.15.6:0/4287673831 >> v2:172.21.15.61:3300/0 conn(0x7f6f5815db70 0x7f6f580ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.707+0000 7f6f577fe700 1 --2- 172.21.15.6:0/4287673831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 0x7f6f5806e400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f5815d080 tx=0x7f6f48009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d577bcadfd7abd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f567fc700 1 -- 172.21.15.6:0/4287673831 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800b870 con 0x7f6f58159590 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f567fc700 1 -- 172.21.15.6:0/4287673831 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4800b9d0 con 0x7f6f58159590 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f567fc700 1 -- 172.21.15.6:0/4287673831 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4800bd60 con 0x7f6f58159590 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/4287673831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 msgr2=0x7f6f5806e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f5e1a6700 1 --2- 172.21.15.6:0/4287673831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 0x7f6f5806e400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f5815d080 tx=0x7f6f48009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/4287673831 shutdown_connections 2023-11-28T13:00:30.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f5e1a6700 1 --2- 172.21.15.6:0/4287673831 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 0x7f6f5806e400 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f5e1a6700 1 --2- 172.21.15.6:0/4287673831 >> v2:172.21.15.61:3300/0 conn(0x7f6f5815db70 0x7f6f580ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/4287673831 >> 172.21.15.6:0/4287673831 conn(0x7f6f580b1650 msgr2=0x7f6f580b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:30.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/4287673831 shutdown_connections 2023-11-28T13:00:30.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.708+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/4287673831 wait complete. 2023-11-28T13:00:30.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.709+0000 7f6f5e1a6700 1 Processor -- start 2023-11-28T13:00:30.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.710+0000 7f6f5e1a6700 1 -- start start 2023-11-28T13:00:30.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.710+0000 7f6f5e1a6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f580ce890 0x7f6f581501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.710+0000 7f6f5e1a6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 0x7f6f580b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.710+0000 7f6f5e1a6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5815db70 0x7f6f58152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.710+0000 7f6f5e1a6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f580b4760 con 0x7f6f58159590 2023-11-28T13:00:30.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.710+0000 7f6f5e1a6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f581f5af0 con 0x7f6f580ce890 2023-11-28T13:00:30.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.710+0000 7f6f5e1a6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f581f5dd0 con 0x7f6f5815db70 2023-11-28T13:00:30.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.710+0000 7f6f56ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 0x7f6f580b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.710+0000 7f6f56ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 0x7f6f580b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57734/0 (socket says 172.21.15.6:57734) 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.710+0000 7f6f56ffd700 1 -- 172.21.15.6:0/3114457511 learned_addr learned my addr 172.21.15.6:0/3114457511 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f57fff700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5815db70 0x7f6f58152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f577fe700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f580ce890 0x7f6f581501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f56ffd700 1 -- 172.21.15.6:0/3114457511 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5815db70 msgr2=0x7f6f58152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f56ffd700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5815db70 0x7f6f58152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f56ffd700 1 -- 172.21.15.6:0/3114457511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f580ce890 msgr2=0x7f6f581501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f56ffd700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f580ce890 0x7f6f581501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f56ffd700 1 -- 172.21.15.6:0/3114457511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f48009550 con 0x7f6f58159590 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f57fff700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5815db70 0x7f6f58152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f577fe700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f580ce890 0x7f6f581501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:30.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f56ffd700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 0x7f6f580b4220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00c7f0 tx=0x7f6f4c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f3ffff700 1 -- 172.21.15.6:0/3114457511 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c011800 con 0x7f6f58159590 2023-11-28T13:00:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f3ffff700 1 -- 172.21.15.6:0/3114457511 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4c011960 con 0x7f6f58159590 2023-11-28T13:00:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/3114457511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5815e010 con 0x7f6f58159590 2023-11-28T13:00:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f3ffff700 1 -- 172.21.15.6:0/3114457511 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c011d10 con 0x7f6f58159590 2023-11-28T13:00:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.711+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/3114457511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5815d370 con 0x7f6f58159590 2023-11-28T13:00:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.712+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/3114457511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f580aec70 con 0x7f6f58159590 2023-11-28T13:00:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.713+0000 7f6f3ffff700 1 -- 172.21.15.6:0/3114457511 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6f4c021070 con 0x7f6f58159590 2023-11-28T13:00:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.713+0000 7f6f3ffff700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f440682f0 0x7f6f4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.713+0000 7f6f3ffff700 1 -- 172.21.15.6:0/3114457511 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6f4c085e00 con 0x7f6f58159590 2023-11-28T13:00:30.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.713+0000 7f6f577fe700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f440682f0 0x7f6f4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:30.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.713+0000 7f6f577fe700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f440682f0 0x7f6f4406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009c10 tx=0x7f6f48008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:30.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.717+0000 7f6f3ffff700 1 -- 172.21.15.6:0/3114457511 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6f4c052370 con 0x7f6f58159590 2023-11-28T13:00:30.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.912+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/3114457511 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f581e5170 con 0x7f6f58159590 2023-11-28T13:00:30.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.913+0000 7f6f3ffff700 1 -- 172.21.15.6:0/3114457511 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f6f4c055990 con 0x7f6f58159590 2023-11-28T13:00:30.915 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:30.915 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:30.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/3114457511 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f440682f0 msgr2=0x7f6f4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f440682f0 0x7f6f4406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6f48009c10 tx=0x7f6f48008030 comp rx=0 tx=0).stop 2023-11-28T13:00:30.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/3114457511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 msgr2=0x7f6f580b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:30.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 0x7f6f580b4220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00c7f0 tx=0x7f6f4c00cbb0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/3114457511 shutdown_connections 2023-11-28T13:00:30.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f440682f0 0x7f6f4406a7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f5815db70 0x7f6f58152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f580ce890 0x7f6f581501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 --2- 172.21.15.6:0/3114457511 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f58159590 0x7f6f580b4220 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:30.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/3114457511 >> 172.21.15.6:0/3114457511 conn(0x7f6f580b1650 msgr2=0x7f6f5815bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:30.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/3114457511 shutdown_connections 2023-11-28T13:00:30.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:30.915+0000 7f6f5e1a6700 1 -- 172.21.15.6:0/3114457511 wait complete. 2023-11-28T13:00:30.929 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:00:30.929 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:31.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.055+0000 7fbc404e8700 1 Processor -- start 2023-11-28T13:00:31.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.055+0000 7fbc404e8700 1 -- start start 2023-11-28T13:00:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.056+0000 7fbc404e8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc38159500 0x7fbc380cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.056+0000 7fbc404e8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbc3815dbf0 0x7fbc380ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.056+0000 7fbc404e8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3815e040 con 0x7fbc380b1630 2023-11-28T13:00:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.056+0000 7fbc404e8700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc380cede0 con 0x7fbc3815dbf0 2023-11-28T13:00:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.056+0000 7fbc404e8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380ceff0 con 0x7fbc38159500 2023-11-28T13:00:31.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.056+0000 7fbc3da83700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbc3815dbf0 0x7fbc380ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.056+0000 7fbc3da83700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbc3815dbf0 0x7fbc380ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57744/0 (socket says 172.21.15.6:57744) 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.056+0000 7fbc3da83700 1 -- 172.21.15.6:0/3486129035 learned_addr learned my addr 172.21.15.6:0/3486129035 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.057+0000 7fbc3e284700 1 --2- 172.21.15.6:0/3486129035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc38159500 0x7fbc380cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.057+0000 7fbc3da83700 1 -- 172.21.15.6:0/3486129035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc38159500 msgr2=0x7fbc380cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.057+0000 7fbc3da83700 1 --2- 172.21.15.6:0/3486129035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc38159500 0x7fbc380cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.057+0000 7fbc3da83700 1 -- 172.21.15.6:0/3486129035 >> v1:172.21.15.61:6789/0 conn(0x7fbc380b1630 legacy=0x7fbc3806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.057+0000 7fbc3da83700 1 -- 172.21.15.6:0/3486129035 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc380cf2d0 con 0x7fbc3815dbf0 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.057+0000 7fbc3da83700 1 --2- 172.21.15.6:0/3486129035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc3815dbf0 0x7fbc380ce8a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0098f0 tx=0x7fbc2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d330cf3507070b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.057+0000 7fbc3d282700 1 -- 172.21.15.6:0/3486129035 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c004230 con 0x7fbc3815dbf0 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.057+0000 7fbc3d282700 1 -- 172.21.15.6:0/3486129035 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c004390 con 0x7fbc3815dbf0 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.057+0000 7fbc3d282700 1 -- 172.21.15.6:0/3486129035 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0036a0 con 0x7fbc3815dbf0 2023-11-28T13:00:31.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.058+0000 7fbc404e8700 1 -- 172.21.15.6:0/3486129035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc3815dbf0 msgr2=0x7fbc380ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.058+0000 7fbc404e8700 1 --2- 172.21.15.6:0/3486129035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc3815dbf0 0x7fbc380ce8a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0098f0 tx=0x7fbc2c009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.058+0000 7fbc404e8700 1 -- 172.21.15.6:0/3486129035 shutdown_connections 2023-11-28T13:00:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.058+0000 7fbc404e8700 1 --2- 172.21.15.6:0/3486129035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc38159500 0x7fbc380cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.058+0000 7fbc404e8700 1 --2- 172.21.15.6:0/3486129035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc3815dbf0 0x7fbc380ce8a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.058+0000 7fbc404e8700 1 -- 172.21.15.6:0/3486129035 >> 172.21.15.6:0/3486129035 conn(0x7fbc380cc630 msgr2=0x7fbc380cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.058+0000 7fbc404e8700 1 -- 172.21.15.6:0/3486129035 shutdown_connections 2023-11-28T13:00:31.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.058+0000 7fbc404e8700 1 -- 172.21.15.6:0/3486129035 wait complete. 2023-11-28T13:00:31.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.059+0000 7fbc404e8700 1 Processor -- start 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.059+0000 7fbc404e8700 1 -- start start 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.060+0000 7fbc404e8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc380b1630 0x7fbc38149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.060+0000 7fbc404e8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc38159500 0x7fbc380ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.060+0000 7fbc404e8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3815dbf0 0x7fbc380c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.060+0000 7fbc404e8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc38149eb0 con 0x7fbc38159500 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.060+0000 7fbc404e8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc38155350 con 0x7fbc3815dbf0 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.060+0000 7fbc404e8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc38155630 con 0x7fbc380b1630 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.060+0000 7fbc3da83700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc38159500 0x7fbc380ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.060+0000 7fbc3da83700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc38159500 0x7fbc380ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57748/0 (socket says 172.21.15.6:57748) 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.060+0000 7fbc3da83700 1 -- 172.21.15.6:0/843911098 learned_addr learned my addr 172.21.15.6:0/843911098 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc3ea85700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3815dbf0 0x7fbc380c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc3e284700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc380b1630 0x7fbc38149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc3da83700 1 -- 172.21.15.6:0/843911098 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc380b1630 msgr2=0x7fbc38149970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc3da83700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc380b1630 0x7fbc38149970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc3da83700 1 -- 172.21.15.6:0/843911098 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3815dbf0 msgr2=0x7fbc380c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc3da83700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3815dbf0 0x7fbc380c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc3da83700 1 -- 172.21.15.6:0/843911098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c009550 con 0x7fbc38159500 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc3da83700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc38159500 0x7fbc380ae730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009130 tx=0x7fbc2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc36ffd700 1 -- 172.21.15.6:0/843911098 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c003aa0 con 0x7fbc38159500 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc36ffd700 1 -- 172.21.15.6:0/843911098 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c01f070 con 0x7fbc38159500 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc404e8700 1 -- 172.21.15.6:0/843911098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3815d3f0 con 0x7fbc38159500 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc36ffd700 1 -- 172.21.15.6:0/843911098 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c003e40 con 0x7fbc38159500 2023-11-28T13:00:31.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.061+0000 7fbc404e8700 1 -- 172.21.15.6:0/843911098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3815d110 con 0x7fbc38159500 2023-11-28T13:00:31.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.062+0000 7fbc404e8700 1 -- 172.21.15.6:0/843911098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3815d6d0 con 0x7fbc38159500 2023-11-28T13:00:31.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.063+0000 7fbc36ffd700 1 -- 172.21.15.6:0/843911098 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbc2c02a020 con 0x7fbc38159500 2023-11-28T13:00:31.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.063+0000 7fbc36ffd700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc240682f0 0x7fbc2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.063+0000 7fbc36ffd700 1 -- 172.21.15.6:0/843911098 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbc2c0875e0 con 0x7fbc38159500 2023-11-28T13:00:31.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.063+0000 7fbc3e284700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc240682f0 0x7fbc2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.063+0000 7fbc3e284700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc240682f0 0x7fbc2406a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc380c6700 tx=0x7fbc280098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.067+0000 7fbc36ffd700 1 -- 172.21.15.6:0/843911098 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbc2c053b50 con 0x7fbc38159500 2023-11-28T13:00:31.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.261+0000 7fbc404e8700 1 -- 172.21.15.6:0/843911098 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbc3815d6d0 con 0x7fbc38159500 2023-11-28T13:00:31.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.261+0000 7fbc36ffd700 1 -- 172.21.15.6:0/843911098 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbc2c057170 con 0x7fbc38159500 2023-11-28T13:00:31.263 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:00:31.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 -- 172.21.15.6:0/843911098 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc240682f0 msgr2=0x7fbc2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc240682f0 0x7fbc2406a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc380c6700 tx=0x7fbc280098d0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 -- 172.21.15.6:0/843911098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc38159500 msgr2=0x7fbc380ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc38159500 0x7fbc380ae730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009130 tx=0x7fbc2c004750 comp rx=0 tx=0).stop 2023-11-28T13:00:31.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 -- 172.21.15.6:0/843911098 shutdown_connections 2023-11-28T13:00:31.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc3815dbf0 0x7fbc380c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc240682f0 0x7fbc2406a7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc380b1630 0x7fbc38149970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 --2- 172.21.15.6:0/843911098 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc38159500 0x7fbc380ae730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 -- 172.21.15.6:0/843911098 >> 172.21.15.6:0/843911098 conn(0x7fbc380cc630 msgr2=0x7fbc380cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 -- 172.21.15.6:0/843911098 shutdown_connections 2023-11-28T13:00:31.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.263+0000 7fbc404e8700 1 -- 172.21.15.6:0/843911098 wait complete. 2023-11-28T13:00:31.276 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:31.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.402+0000 7f366f8d1700 1 Processor -- start 2023-11-28T13:00:31.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.402+0000 7f366f8d1700 1 -- start start 2023-11-28T13:00:31.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.403+0000 7f366f8d1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f36680b1630 0x7f36680cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.403+0000 7f366f8d1700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36680ce550 con 0x7f366806afc0 2023-11-28T13:00:31.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.403+0000 7f366f8d1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36680ce830 con 0x7f36680b1a00 2023-11-28T13:00:31.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.403+0000 7f366f8d1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36680ceb10 con 0x7f36680b1630 2023-11-28T13:00:31.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.403+0000 7f366de6e700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f366806afc0 0x7f366806b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45078/0 (socket says 172.21.15.6:45078) 2023-11-28T13:00:31.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.403+0000 7f366de6e700 1 -- 172.21.15.6:0/144658912 learned_addr learned my addr 172.21.15.6:0/144658912 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:31.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.403+0000 7f366d66d700 1 --2- 172.21.15.6:0/144658912 >> v2:172.21.15.61:3300/0 conn(0x7f36680b1630 0x7f36680cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.403+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071900935 0 0) 0x7f36680ce550 con 0x7f366806afc0 2023-11-28T13:00:31.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36540035b0 con 0x7f366806afc0 2023-11-28T13:00:31.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1498704330 0 0) 0x7f36540035b0 con 0x7f366806afc0 2023-11-28T13:00:31.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 >> v2:172.21.15.61:3300/0 conn(0x7f36680b1630 msgr2=0x7f36680cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:31.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 --2- 172.21.15.6:0/144658912 >> v2:172.21.15.61:3300/0 conn(0x7f36680b1630 0x7f36680cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 >> v1:172.21.15.61:6789/0 conn(0x7f36680b1a00 legacy=0x7f36680caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680cedf0 con 0x7f366806afc0 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f36600019f0 con 0x7f366806afc0 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 >> v1:172.21.15.6:6789/0 conn(0x7f366806afc0 legacy=0x7f366806b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3654003a00 0x7f3654003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3654004850 0x7f3654004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3654005200 0x7f365400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680ceb10 con 0x7f3654004850 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365400c080 con 0x7f3654005200 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365400c240 con 0x7f3654003a00 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366d66d700 1 --2- 172.21.15.6:0/144658912 >> v2:172.21.15.61:3300/0 conn(0x7f36680b1630 0x7f36680cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:31.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366ce6c700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3654004850 0x7f3654004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366de6e700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3654005200 0x7f365400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366d66d700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3654003a00 0x7f3654003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366ce6c700 1 -- 172.21.15.6:0/144658912 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3654003a00 msgr2=0x7f3654003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:31.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366ce6c700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3654003a00 0x7f3654003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366ce6c700 1 -- 172.21.15.6:0/144658912 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3654005200 msgr2=0x7f365400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:31.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366ce6c700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3654005200 0x7f365400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366de6e700 1 -- 172.21.15.6:0/144658912 reap_dead start 2023-11-28T13:00:31.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366ce6c700 1 -- 172.21.15.6:0/144658912 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36540035b0 con 0x7f3654004850 2023-11-28T13:00:31.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.404+0000 7f366ce6c700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3654004850 0x7f3654004cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f36640017c0 tx=0x7f3654005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357770fb9359e0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.405+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36640052d0 con 0x7f3654004850 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.405+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3664005430 con 0x7f3654004850 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.405+0000 7f365ffff700 1 -- 172.21.15.6:0/144658912 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36640038c0 con 0x7f3654004850 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.405+0000 7f366f8d1700 1 -- 172.21.15.6:0/144658912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3654004850 msgr2=0x7f3654004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.405+0000 7f366f8d1700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3654004850 0x7f3654004cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f36640017c0 tx=0x7f3654005880 comp rx=0 tx=0).stop 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.405+0000 7f366f8d1700 1 -- 172.21.15.6:0/144658912 shutdown_connections 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.405+0000 7f366f8d1700 1 --2- 172.21.15.6:0/144658912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3654004850 0x7f3654004cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.405+0000 7f366f8d1700 1 -- 172.21.15.6:0/144658912 >> 172.21.15.6:0/144658912 conn(0x7f36680cc630 msgr2=0x7f36680cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.405+0000 7f366f8d1700 1 -- 172.21.15.6:0/144658912 shutdown_connections 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.405+0000 7f366f8d1700 1 -- 172.21.15.6:0/144658912 wait complete. 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.406+0000 7f366f8d1700 1 Processor -- start 2023-11-28T13:00:31.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.407+0000 7f366f8d1700 1 -- start start 2023-11-28T13:00:31.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.407+0000 7f366f8d1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3654004850 0x7f3668159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.407+0000 7f366f8d1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36680c6230 0x7f36680ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.407+0000 7f366f8d1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36680ad5d0 0x7f36680ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.407+0000 7f366f8d1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680b5730 con 0x7f36680ad5d0 2023-11-28T13:00:31.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.407+0000 7f366f8d1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680b4b20 con 0x7f36680c6230 2023-11-28T13:00:31.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.407+0000 7f366f8d1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36680b4e00 con 0x7f3654004850 2023-11-28T13:00:31.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.407+0000 7f366de6e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36680ad5d0 0x7f36680ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.407+0000 7f366de6e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36680ad5d0 0x7f36680ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57770/0 (socket says 172.21.15.6:57770) 2023-11-28T13:00:31.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.407+0000 7f366de6e700 1 -- 172.21.15.6:0/2264602331 learned_addr learned my addr 172.21.15.6:0/2264602331 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:31.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366d66d700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3654004850 0x7f3668159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366ce6c700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36680c6230 0x7f36680ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366de6e700 1 -- 172.21.15.6:0/2264602331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3654004850 msgr2=0x7f3668159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366de6e700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3654004850 0x7f3668159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366de6e700 1 -- 172.21.15.6:0/2264602331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36680c6230 msgr2=0x7f36680ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366de6e700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36680c6230 0x7f36680ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366de6e700 1 -- 172.21.15.6:0/2264602331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680b1ea0 con 0x7f36680ad5d0 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366d66d700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3654004850 0x7f3668159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366ce6c700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36680c6230 0x7f36680ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366de6e700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36680ad5d0 0x7f36680ada70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f36680b1dd0 tx=0x7f366000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f365dffb700 1 -- 172.21.15.6:0/2264602331 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366000ed90 con 0x7f36680ad5d0 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f365dffb700 1 -- 172.21.15.6:0/2264602331 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3660013420 con 0x7f36680ad5d0 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f365dffb700 1 -- 172.21.15.6:0/2264602331 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36600137b0 con 0x7f36680ad5d0 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366f8d1700 1 -- 172.21.15.6:0/2264602331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36680cf0d0 con 0x7f36680ad5d0 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.408+0000 7f366f8d1700 1 -- 172.21.15.6:0/2264602331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36680ce140 con 0x7f36680ad5d0 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.409+0000 7f366f8d1700 1 -- 172.21.15.6:0/2264602331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36680739f0 con 0x7f36680ad5d0 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.409+0000 7f365dffb700 1 -- 172.21.15.6:0/2264602331 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3660013910 con 0x7f36680ad5d0 2023-11-28T13:00:31.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.410+0000 7f365dffb700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f364c0682f0 0x7f364c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.410+0000 7f365dffb700 1 -- 172.21.15.6:0/2264602331 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3660086770 con 0x7f36680ad5d0 2023-11-28T13:00:31.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.410+0000 7f366d66d700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f364c0682f0 0x7f364c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.410+0000 7f366d66d700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f364c0682f0 0x7f364c06a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36680c5150 tx=0x7f3658008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.414+0000 7f365dffb700 1 -- 172.21.15.6:0/2264602331 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3660052ce0 con 0x7f36680ad5d0 2023-11-28T13:00:31.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.605+0000 7f366f8d1700 1 -- 172.21.15.6:0/2264602331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36680739f0 con 0x7f36680ad5d0 2023-11-28T13:00:31.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.606+0000 7f365dffb700 1 -- 172.21.15.6:0/2264602331 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3660056300 con 0x7f36680ad5d0 2023-11-28T13:00:31.607 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:00:31.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 -- 172.21.15.6:0/2264602331 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f364c0682f0 msgr2=0x7f364c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f364c0682f0 0x7f364c06a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36680c5150 tx=0x7f3658008040 comp rx=0 tx=0).stop 2023-11-28T13:00:31.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 -- 172.21.15.6:0/2264602331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36680ad5d0 msgr2=0x7f36680ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36680ad5d0 0x7f36680ada70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f36680b1dd0 tx=0x7f366000e440 comp rx=0 tx=0).stop 2023-11-28T13:00:31.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 -- 172.21.15.6:0/2264602331 shutdown_connections 2023-11-28T13:00:31.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36680ad5d0 0x7f36680ada70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f364c0682f0 0x7f364c06a7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3654004850 0x7f3668159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 --2- 172.21.15.6:0/2264602331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36680c6230 0x7f36680ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 -- 172.21.15.6:0/2264602331 >> 172.21.15.6:0/2264602331 conn(0x7f36680cc630 msgr2=0x7f36680b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 -- 172.21.15.6:0/2264602331 shutdown_connections 2023-11-28T13:00:31.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.608+0000 7f366f8d1700 1 -- 172.21.15.6:0/2264602331 wait complete. 2023-11-28T13:00:31.621 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:31.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.748+0000 7fd486a04700 1 Processor -- start 2023-11-28T13:00:31.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.748+0000 7fd486a04700 1 -- start start 2023-11-28T13:00:31.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd486a04700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd48015c4b0 0x7fd48015d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd486a04700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd4800c9950 0x7fd4800cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd486a04700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4800cae80 con 0x7fd48015c880 2023-11-28T13:00:31.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd486a04700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48015dba0 con 0x7fd4800c9950 2023-11-28T13:00:31.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd486a04700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800c9de0 con 0x7fd48015c4b0 2023-11-28T13:00:31.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd484fa1700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd4800c9950 0x7fd4800cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd484fa1700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd4800c9950 0x7fd4800cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57778/0 (socket says 172.21.15.6:57778) 2023-11-28T13:00:31.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd484fa1700 1 -- 172.21.15.6:0/1507375207 learned_addr learned my addr 172.21.15.6:0/1507375207 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:31.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd47ffff700 1 --2- 172.21.15.6:0/1507375207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd48015c4b0 0x7fd48015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd484fa1700 1 -- 172.21.15.6:0/1507375207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd48015c4b0 msgr2=0x7fd48015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:31.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd484fa1700 1 --2- 172.21.15.6:0/1507375207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd48015c4b0 0x7fd48015d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd484fa1700 1 -- 172.21.15.6:0/1507375207 >> v1:172.21.15.61:6789/0 conn(0x7fd48015c880 legacy=0x7fd48006a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd484fa1700 1 -- 172.21.15.6:0/1507375207 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4800ca0c0 con 0x7fd4800c9950 2023-11-28T13:00:31.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd47ffff700 1 --2- 172.21.15.6:0/1507375207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd48015c4b0 0x7fd48015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:31.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.749+0000 7fd484fa1700 1 --2- 172.21.15.6:0/1507375207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4800c9950 0x7fd4800cced0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd478009e50 tx=0x7fd47800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e375638d51e3a935 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd47effd700 1 -- 172.21.15.6:0/1507375207 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47800e070 con 0x7fd4800c9950 2023-11-28T13:00:31.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd47effd700 1 -- 172.21.15.6:0/1507375207 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47800b9c0 con 0x7fd4800c9950 2023-11-28T13:00:31.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd47effd700 1 -- 172.21.15.6:0/1507375207 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47800bd70 con 0x7fd4800c9950 2023-11-28T13:00:31.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd486a04700 1 -- 172.21.15.6:0/1507375207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4800c9950 msgr2=0x7fd4800cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd486a04700 1 --2- 172.21.15.6:0/1507375207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4800c9950 0x7fd4800cced0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd478009e50 tx=0x7fd47800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:31.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd486a04700 1 -- 172.21.15.6:0/1507375207 shutdown_connections 2023-11-28T13:00:31.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd486a04700 1 --2- 172.21.15.6:0/1507375207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4800c9950 0x7fd4800cced0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd486a04700 1 --2- 172.21.15.6:0/1507375207 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd48015c4b0 0x7fd48015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd486a04700 1 -- 172.21.15.6:0/1507375207 >> 172.21.15.6:0/1507375207 conn(0x7fd4800b0df0 msgr2=0x7fd4800b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd486a04700 1 -- 172.21.15.6:0/1507375207 shutdown_connections 2023-11-28T13:00:31.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.750+0000 7fd486a04700 1 -- 172.21.15.6:0/1507375207 wait complete. 2023-11-28T13:00:31.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.751+0000 7fd486a04700 1 Processor -- start 2023-11-28T13:00:31.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd486a04700 1 -- start start 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd486a04700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4800c9950 0x7fd480148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd486a04700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd48015c4b0 0x7fd4800ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd486a04700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd48015c880 0x7fd480159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd486a04700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800c6440 con 0x7fd48015c4b0 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd486a04700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800adcf0 con 0x7fd48015c880 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd486a04700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800adfd0 con 0x7fd4800c9950 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd47f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd48015c4b0 0x7fd4800ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd47f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd48015c4b0 0x7fd4800ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57782/0 (socket says 172.21.15.6:57782) 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd47f7fe700 1 -- 172.21.15.6:0/3002153552 learned_addr learned my addr 172.21.15.6:0/3002153552 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.752+0000 7fd47ffff700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4800c9950 0x7fd480148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd484fa1700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd48015c880 0x7fd480159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd47f7fe700 1 -- 172.21.15.6:0/3002153552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4800c9950 msgr2=0x7fd480148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd47f7fe700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4800c9950 0x7fd480148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd47f7fe700 1 -- 172.21.15.6:0/3002153552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd48015c880 msgr2=0x7fd480159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd47f7fe700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd48015c880 0x7fd480159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd47f7fe700 1 -- 172.21.15.6:0/3002153552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd478009b30 con 0x7fd48015c4b0 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd47f7fe700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd48015c4b0 0x7fd4800ad2a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd47000c450 tx=0x7fd47000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd47cff9700 1 -- 172.21.15.6:0/3002153552 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4700052d0 con 0x7fd48015c4b0 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd47cff9700 1 -- 172.21.15.6:0/3002153552 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd470005430 con 0x7fd48015c4b0 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd486a04700 1 -- 172.21.15.6:0/3002153552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4800ca680 con 0x7fd48015c4b0 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd47cff9700 1 -- 172.21.15.6:0/3002153552 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4700038c0 con 0x7fd48015c4b0 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.753+0000 7fd486a04700 1 -- 172.21.15.6:0/3002153552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4800ca3a0 con 0x7fd48015c4b0 2023-11-28T13:00:31.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.754+0000 7fd486a04700 1 -- 172.21.15.6:0/3002153552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4800ca960 con 0x7fd48015c4b0 2023-11-28T13:00:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.755+0000 7fd47cff9700 1 -- 172.21.15.6:0/3002153552 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd470019030 con 0x7fd48015c4b0 2023-11-28T13:00:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.755+0000 7fd47cff9700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd46c0682f0 0x7fd46c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.755+0000 7fd47cff9700 1 -- 172.21.15.6:0/3002153552 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd470085120 con 0x7fd48015c4b0 2023-11-28T13:00:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.755+0000 7fd47ffff700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd46c0682f0 0x7fd46c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.755+0000 7fd47ffff700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd46c0682f0 0x7fd46c06a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd4740099d0 tx=0x7fd474008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:31.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.759+0000 7fd47cff9700 1 -- 172.21.15.6:0/3002153552 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd470051690 con 0x7fd48015c4b0 2023-11-28T13:00:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.954+0000 7fd486a04700 1 -- 172.21.15.6:0/3002153552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4800c6580 con 0x7fd48015c4b0 2023-11-28T13:00:31.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.954+0000 7fd47cff9700 1 -- 172.21.15.6:0/3002153552 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd4800c6580 con 0x7fd48015c4b0 2023-11-28T13:00:31.956 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:00:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 -- 172.21.15.6:0/3002153552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd46c0682f0 msgr2=0x7fd46c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd46c0682f0 0x7fd46c06a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd4740099d0 tx=0x7fd474008040 comp rx=0 tx=0).stop 2023-11-28T13:00:31.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 -- 172.21.15.6:0/3002153552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd48015c4b0 msgr2=0x7fd4800ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd48015c4b0 0x7fd4800ad2a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd47000c450 tx=0x7fd47000c760 comp rx=0 tx=0).stop 2023-11-28T13:00:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 -- 172.21.15.6:0/3002153552 shutdown_connections 2023-11-28T13:00:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd48015c880 0x7fd480159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd46c0682f0 0x7fd46c06a7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4800c9950 0x7fd480148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 --2- 172.21.15.6:0/3002153552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd48015c4b0 0x7fd4800ad2a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 -- 172.21.15.6:0/3002153552 >> 172.21.15.6:0/3002153552 conn(0x7fd4800b0df0 msgr2=0x7fd48015b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:31.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 -- 172.21.15.6:0/3002153552 shutdown_connections 2023-11-28T13:00:31.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:31.957+0000 7fd486a04700 1 -- 172.21.15.6:0/3002153552 wait complete. 2023-11-28T13:00:31.970 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:32.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.096+0000 7fda17659700 1 Processor -- start 2023-11-28T13:00:32.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.097+0000 7fda17659700 1 -- start start 2023-11-28T13:00:32.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.097+0000 7fda17659700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fda100d1560 0x7fda100d1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.097+0000 7fda17659700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda100cdc20 con 0x7fda1015c6e0 2023-11-28T13:00:32.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.097+0000 7fda17659700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda100cd940 con 0x7fda100cd360 2023-11-28T13:00:32.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.097+0000 7fda17659700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda100d1ec0 con 0x7fda100d1560 2023-11-28T13:00:32.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda153f5700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fda1015c6e0 0x7fda1006ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45108/0 (socket says 172.21.15.6:45108) 2023-11-28T13:00:32.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda153f5700 1 -- 172.21.15.6:0/1587832460 learned_addr learned my addr 172.21.15.6:0/1587832460 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:32.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda15bf6700 1 --2- 172.21.15.6:0/1587832460 >> v2:172.21.15.61:3300/0 conn(0x7fda100d1560 0x7fda100d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578388837 0 0) 0x7fda0c001110 con 0x7fda1015c6e0 2023-11-28T13:00:32.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9fc0035b0 con 0x7fda1015c6e0 2023-11-28T13:00:32.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525407029 0 0) 0x7fda100cd940 con 0x7fda100cd360 2023-11-28T13:00:32.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda0c001110 con 0x7fda100cd360 2023-11-28T13:00:32.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2435996896 0 0) 0x7fd9fc0035b0 con 0x7fda1015c6e0 2023-11-28T13:00:32.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 >> v2:172.21.15.61:3300/0 conn(0x7fda100d1560 msgr2=0x7fda100d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 --2- 172.21.15.6:0/1587832460 >> v2:172.21.15.61:3300/0 conn(0x7fda100d1560 0x7fda100d1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 >> v1:172.21.15.61:6789/0 conn(0x7fda100cd360 legacy=0x7fda100b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda100d0a80 con 0x7fda1015c6e0 2023-11-28T13:00:32.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fda0c001800 con 0x7fda1015c6e0 2023-11-28T13:00:32.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 >> v1:172.21.15.6:6789/0 conn(0x7fda1015c6e0 legacy=0x7fda1006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9fc003be0 0x7fd9fc003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc004850 0x7fd9fc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd9fc005200 0x7fd9fc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda100d1ec0 con 0x7fd9fc003be0 2023-11-28T13:00:32.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc00c0e0 con 0x7fd9fc005200 2023-11-28T13:00:32.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc00c270 con 0x7fd9fc004850 2023-11-28T13:00:32.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda153f5700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9fc003be0 0x7fd9fc003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda14bf4700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc004850 0x7fd9fc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.098+0000 7fda15bf6700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd9fc005200 0x7fd9fc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda153f5700 1 -- 172.21.15.6:0/1587832460 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc004850 msgr2=0x7fd9fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda153f5700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc004850 0x7fd9fc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda153f5700 1 -- 172.21.15.6:0/1587832460 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd9fc005200 msgr2=0x7fd9fc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda153f5700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd9fc005200 0x7fd9fc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda153f5700 1 -- 172.21.15.6:0/1587832460 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc0035b0 con 0x7fd9fc003be0 2023-11-28T13:00:32.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda14bf4700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc004850 0x7fd9fc004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:32.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda15bf6700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd9fc005200 0x7fd9fc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:32.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda15bf6700 1 -- 172.21.15.6:0/1587832460 reap_dead start 2023-11-28T13:00:32.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda153f5700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9fc003be0 0x7fd9fc003fd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00c0e0 tx=0x7fd9fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4df4c490cc0f4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00cc80 con 0x7fd9fc003be0 2023-11-28T13:00:32.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c00cde0 con 0x7fd9fc003be0 2023-11-28T13:00:32.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda03fff700 1 -- 172.21.15.6:0/1587832460 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c005ba0 con 0x7fd9fc003be0 2023-11-28T13:00:32.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda17659700 1 -- 172.21.15.6:0/1587832460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9fc003be0 msgr2=0x7fd9fc003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.099+0000 7fda17659700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9fc003be0 0x7fd9fc003fd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00c0e0 tx=0x7fd9fc005880 comp rx=0 tx=0).stop 2023-11-28T13:00:32.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.100+0000 7fda17659700 1 -- 172.21.15.6:0/1587832460 shutdown_connections 2023-11-28T13:00:32.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.100+0000 7fda17659700 1 --2- 172.21.15.6:0/1587832460 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9fc003be0 0x7fd9fc003fd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.100+0000 7fda17659700 1 -- 172.21.15.6:0/1587832460 >> 172.21.15.6:0/1587832460 conn(0x7fda100cf790 msgr2=0x7fda100cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.100+0000 7fda17659700 1 -- 172.21.15.6:0/1587832460 shutdown_connections 2023-11-28T13:00:32.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.100+0000 7fda17659700 1 -- 172.21.15.6:0/1587832460 wait complete. 2023-11-28T13:00:32.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.101+0000 7fda17659700 1 Processor -- start 2023-11-28T13:00:32.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.101+0000 7fda17659700 1 -- start start 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda17659700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc003be0 0x7fda1014c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda17659700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda100b64c0 0x7fda100b8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda17659700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda100b6930 0x7fda100b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda17659700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda100b7b90 con 0x7fda100b64c0 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda17659700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda100b8b10 con 0x7fda100b6930 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda17659700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda100cbee0 con 0x7fd9fc003be0 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda14bf4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda100b64c0 0x7fda100b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda14bf4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda100b64c0 0x7fda100b8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57802/0 (socket says 172.21.15.6:57802) 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda14bf4700 1 -- 172.21.15.6:0/3994096922 learned_addr learned my addr 172.21.15.6:0/3994096922 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda14bf4700 1 -- 172.21.15.6:0/3994096922 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc003be0 msgr2=0x7fda1014c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda15bf6700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda100b6930 0x7fda100b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda153f5700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc003be0 0x7fda1014c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda14bf4700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc003be0 0x7fda1014c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda14bf4700 1 -- 172.21.15.6:0/3994096922 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda100b6930 msgr2=0x7fda100b85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda14bf4700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda100b6930 0x7fda100b85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda14bf4700 1 -- 172.21.15.6:0/3994096922 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda100d1040 con 0x7fda100b64c0 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda15bf6700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda100b6930 0x7fda100b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda153f5700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc003be0 0x7fda1014c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda14bf4700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda100b64c0 0x7fda100b8090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fda0400e7b0 tx=0x7fda0400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda01ffb700 1 -- 172.21.15.6:0/3994096922 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040053a0 con 0x7fda100b64c0 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda01ffb700 1 -- 172.21.15.6:0/3994096922 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda04005500 con 0x7fda100b64c0 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda01ffb700 1 -- 172.21.15.6:0/3994096922 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04006a80 con 0x7fda100b64c0 2023-11-28T13:00:32.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.102+0000 7fda17659700 1 -- 172.21.15.6:0/3994096922 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda100d0d60 con 0x7fda100b64c0 2023-11-28T13:00:32.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.103+0000 7fda17659700 1 -- 172.21.15.6:0/3994096922 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1015cab0 con 0x7fda100b64c0 2023-11-28T13:00:32.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.103+0000 7fda17659700 1 -- 172.21.15.6:0/3994096922 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda10158fe0 con 0x7fda100b64c0 2023-11-28T13:00:32.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.104+0000 7fda01ffb700 1 -- 172.21.15.6:0/3994096922 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fda04006be0 con 0x7fda100b64c0 2023-11-28T13:00:32.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.104+0000 7fda01ffb700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9f40682f0 0x7fd9f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.104+0000 7fda01ffb700 1 -- 172.21.15.6:0/3994096922 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda040850b0 con 0x7fda100b64c0 2023-11-28T13:00:32.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.104+0000 7fda153f5700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9f40682f0 0x7fd9f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.105+0000 7fda153f5700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9f40682f0 0x7fd9f406a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda0c015040 tx=0x7fda0c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.108+0000 7fda01ffb700 1 -- 172.21.15.6:0/3994096922 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fda0404c5e0 con 0x7fda100b64c0 2023-11-28T13:00:32.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.300+0000 7fda17659700 1 -- 172.21.15.6:0/3994096922 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda100739f0 con 0x7fda100b64c0 2023-11-28T13:00:32.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.300+0000 7fda01ffb700 1 -- 172.21.15.6:0/3994096922 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda04019020 con 0x7fda100b64c0 2023-11-28T13:00:32.302 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:00:32.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 -- 172.21.15.6:0/3994096922 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9f40682f0 msgr2=0x7fd9f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9f40682f0 0x7fd9f406a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda0c015040 tx=0x7fda0c007000 comp rx=0 tx=0).stop 2023-11-28T13:00:32.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 -- 172.21.15.6:0/3994096922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda100b64c0 msgr2=0x7fda100b8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda100b64c0 0x7fda100b8090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fda0400e7b0 tx=0x7fda0400eb70 comp rx=0 tx=0).stop 2023-11-28T13:00:32.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 -- 172.21.15.6:0/3994096922 shutdown_connections 2023-11-28T13:00:32.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda100b6930 0x7fda100b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9f40682f0 0x7fd9f406a7a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9fc003be0 0x7fda1014c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 --2- 172.21.15.6:0/3994096922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda100b64c0 0x7fda100b8090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 -- 172.21.15.6:0/3994096922 >> 172.21.15.6:0/3994096922 conn(0x7fda100cf790 msgr2=0x7fda100b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 -- 172.21.15.6:0/3994096922 shutdown_connections 2023-11-28T13:00:32.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.303+0000 7fda17659700 1 -- 172.21.15.6:0/3994096922 wait complete. 2023-11-28T13:00:32.316 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:00:32.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.441+0000 7fda6ccf4700 1 Processor -- start 2023-11-28T13:00:32.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.441+0000 7fda6ccf4700 1 -- start start 2023-11-28T13:00:32.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.442+0000 7fda6ccf4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda680ca360 0x7fda6806f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.442+0000 7fda6ccf4700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fda680cae50 0x7fda680cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.442+0000 7fda6ccf4700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda680caab0 con 0x7fda6815c950 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.442+0000 7fda6ccf4700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda680cb810 con 0x7fda680cae50 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.442+0000 7fda6ccf4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6815cd20 con 0x7fda680ca360 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda66d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fda680cae50 0x7fda680cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda66d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fda680cae50 0x7fda680cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57812/0 (socket says 172.21.15.6:57812) 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda66d9d700 1 -- 172.21.15.6:0/520277327 learned_addr learned my addr 172.21.15.6:0/520277327 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda65d9b700 1 --2- 172.21.15.6:0/520277327 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda680ca360 0x7fda6806f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda66d9d700 1 -- 172.21.15.6:0/520277327 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda680ca360 msgr2=0x7fda6806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda66d9d700 1 --2- 172.21.15.6:0/520277327 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda680ca360 0x7fda6806f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda66d9d700 1 -- 172.21.15.6:0/520277327 >> v1:172.21.15.61:6789/0 conn(0x7fda6815c950 legacy=0x7fda6806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda66d9d700 1 -- 172.21.15.6:0/520277327 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6806e770 con 0x7fda680cae50 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda66d9d700 1 --2- 172.21.15.6:0/520277327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680cae50 0x7fda680cb2d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda60009e50 tx=0x7fda6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b302d608325abb1f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda6559a700 1 -- 172.21.15.6:0/520277327 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6000bb10 con 0x7fda680cae50 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda6559a700 1 -- 172.21.15.6:0/520277327 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6000bc70 con 0x7fda680cae50 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.443+0000 7fda6559a700 1 -- 172.21.15.6:0/520277327 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600104d0 con 0x7fda680cae50 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.444+0000 7fda6ccf4700 1 -- 172.21.15.6:0/520277327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680cae50 msgr2=0x7fda680cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.444+0000 7fda6ccf4700 1 --2- 172.21.15.6:0/520277327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680cae50 0x7fda680cb2d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda60009e50 tx=0x7fda6000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.444+0000 7fda6ccf4700 1 -- 172.21.15.6:0/520277327 shutdown_connections 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.444+0000 7fda6ccf4700 1 --2- 172.21.15.6:0/520277327 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680cae50 0x7fda680cb2d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.444+0000 7fda6ccf4700 1 --2- 172.21.15.6:0/520277327 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda680ca360 0x7fda6806f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.444+0000 7fda6ccf4700 1 -- 172.21.15.6:0/520277327 >> 172.21.15.6:0/520277327 conn(0x7fda680b0df0 msgr2=0x7fda680b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.444+0000 7fda6ccf4700 1 -- 172.21.15.6:0/520277327 shutdown_connections 2023-11-28T13:00:32.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.444+0000 7fda6ccf4700 1 -- 172.21.15.6:0/520277327 wait complete. 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.445+0000 7fda6ccf4700 1 Processor -- start 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.445+0000 7fda6ccf4700 1 -- start start 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6ccf4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680ca360 0x7fda680b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6ccf4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda680cae50 0x7fda6815af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6ccf4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda6815c950 0x7fda6815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6ccf4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda680c63d0 con 0x7fda680ca360 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6ccf4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda680b5080 con 0x7fda6815c950 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6ccf4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda68140940 con 0x7fda680cae50 2023-11-28T13:00:32.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680ca360 0x7fda680b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680ca360 0x7fda680b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57814/0 (socket says 172.21.15.6:57814) 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6659c700 1 -- 172.21.15.6:0/3838401573 learned_addr learned my addr 172.21.15.6:0/3838401573 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda66d9d700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda6815c950 0x7fda6815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda65d9b700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda680cae50 0x7fda6815af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6659c700 1 -- 172.21.15.6:0/3838401573 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda680cae50 msgr2=0x7fda6815af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6659c700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda680cae50 0x7fda6815af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6659c700 1 -- 172.21.15.6:0/3838401573 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda6815c950 msgr2=0x7fda6815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.446+0000 7fda6659c700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda6815c950 0x7fda6815b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.447+0000 7fda6659c700 1 -- 172.21.15.6:0/3838401573 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda60009b30 con 0x7fda680ca360 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.447+0000 7fda6659c700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680ca360 0x7fda680b4b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c450 tx=0x7fda5800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.447+0000 7fda5effd700 1 -- 172.21.15.6:0/3838401573 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda580052d0 con 0x7fda680ca360 2023-11-28T13:00:32.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.447+0000 7fda5effd700 1 -- 172.21.15.6:0/3838401573 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda58005430 con 0x7fda680ca360 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.447+0000 7fda6ccf4700 1 -- 172.21.15.6:0/3838401573 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6806ea10 con 0x7fda680ca360 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.447+0000 7fda5effd700 1 -- 172.21.15.6:0/3838401573 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda580038c0 con 0x7fda680ca360 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.447+0000 7fda6ccf4700 1 -- 172.21.15.6:0/3838401573 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda680ca7d0 con 0x7fda680ca360 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.448+0000 7fda6ccf4700 1 -- 172.21.15.6:0/3838401573 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6806ecf0 con 0x7fda680ca360 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.448+0000 7fda5effd700 1 -- 172.21.15.6:0/3838401573 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fda58019030 con 0x7fda680ca360 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.449+0000 7fda5effd700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda480682f0 0x7fda4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.449+0000 7fda5effd700 1 -- 172.21.15.6:0/3838401573 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda58085120 con 0x7fda680ca360 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.449+0000 7fda65d9b700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda480682f0 0x7fda4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.449+0000 7fda65d9b700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda480682f0 0x7fda4806a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fda681459a0 tx=0x7fda50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.453+0000 7fda5effd700 1 -- 172.21.15.6:0/3838401573 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fda58051690 con 0x7fda680ca360 2023-11-28T13:00:32.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.645+0000 7fda6ccf4700 1 -- 172.21.15.6:0/3838401573 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fda680739f0 con 0x7fda680ca360 2023-11-28T13:00:32.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.646+0000 7fda5effd700 1 -- 172.21.15.6:0/3838401573 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fda5800ce60 con 0x7fda680ca360 2023-11-28T13:00:32.647 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:00:32.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 -- 172.21.15.6:0/3838401573 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda480682f0 msgr2=0x7fda4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda480682f0 0x7fda4806a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fda681459a0 tx=0x7fda50008040 comp rx=0 tx=0).stop 2023-11-28T13:00:32.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 -- 172.21.15.6:0/3838401573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680ca360 msgr2=0x7fda680b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680ca360 0x7fda680b4b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c450 tx=0x7fda5800c760 comp rx=0 tx=0).stop 2023-11-28T13:00:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 -- 172.21.15.6:0/3838401573 shutdown_connections 2023-11-28T13:00:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda6815c950 0x7fda6815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda680ca360 0x7fda680b4b40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda480682f0 0x7fda4806a7a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 --2- 172.21.15.6:0/3838401573 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda680cae50 0x7fda6815af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 -- 172.21.15.6:0/3838401573 >> 172.21.15.6:0/3838401573 conn(0x7fda680b0df0 msgr2=0x7fda6806b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 -- 172.21.15.6:0/3838401573 shutdown_connections 2023-11-28T13:00:32.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.648+0000 7fda6ccf4700 1 -- 172.21.15.6:0/3838401573 wait complete. 2023-11-28T13:00:32.661 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:00:32.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.785+0000 7ff18e078700 1 Processor -- start 2023-11-28T13:00:32.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.785+0000 7ff18e078700 1 -- start start 2023-11-28T13:00:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.786+0000 7ff18e078700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff188161090 0x7ff1880ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.786+0000 7ff18e078700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1880ce340 con 0x7ff1880caeb0 2023-11-28T13:00:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.786+0000 7ff18e078700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1880cebe0 con 0x7ff18815c990 2023-11-28T13:00:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.786+0000 7ff18e078700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1880cf780 con 0x7ff188161090 2023-11-28T13:00:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.786+0000 7ff186ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff188161090 0x7ff1880ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.786+0000 7ff187fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff1880caeb0 0x7ff1880cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45146/0 (socket says 172.21.15.6:45146) 2023-11-28T13:00:32.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.786+0000 7ff187fff700 1 -- 172.21.15.6:0/364479651 learned_addr learned my addr 172.21.15.6:0/364479651 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.787+0000 7ff186ffd700 1 -- 172.21.15.6:0/364479651 >> v1:172.21.15.61:6789/0 conn(0x7ff18815c990 legacy=0x7ff1880cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.787+0000 7ff186ffd700 1 -- 172.21.15.6:0/364479651 >> v1:172.21.15.6:6789/0 conn(0x7ff1880caeb0 legacy=0x7ff1880cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.787+0000 7ff1867fc700 1 -- 172.21.15.6:0/364479651 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513853645 0 0) 0x7ff1880ce340 con 0x7ff1880caeb0 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.787+0000 7ff186ffd700 1 -- 172.21.15.6:0/364479651 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1880cb3f0 con 0x7ff188161090 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.787+0000 7ff186ffd700 1 --2- 172.21.15.6:0/364479651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff188161090 0x7ff1880ca970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009840 tx=0x7ff17c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7b53a82d75d676 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.787+0000 7ff1867fc700 1 -- 172.21.15.6:0/364479651 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c011830 con 0x7ff188161090 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.787+0000 7ff1867fc700 1 -- 172.21.15.6:0/364479651 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c011990 con 0x7ff188161090 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.787+0000 7ff1867fc700 1 -- 172.21.15.6:0/364479651 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c011d20 con 0x7ff188161090 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.788+0000 7ff18e078700 1 -- 172.21.15.6:0/364479651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff188161090 msgr2=0x7ff1880ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.788+0000 7ff18e078700 1 --2- 172.21.15.6:0/364479651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff188161090 0x7ff1880ca970 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009840 tx=0x7ff17c009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.788+0000 7ff18e078700 1 -- 172.21.15.6:0/364479651 shutdown_connections 2023-11-28T13:00:32.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.788+0000 7ff18e078700 1 --2- 172.21.15.6:0/364479651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff188161090 0x7ff1880ca970 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.788+0000 7ff18e078700 1 -- 172.21.15.6:0/364479651 >> 172.21.15.6:0/364479651 conn(0x7ff1880b1650 msgr2=0x7ff1880b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.788+0000 7ff18e078700 1 -- 172.21.15.6:0/364479651 shutdown_connections 2023-11-28T13:00:32.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.788+0000 7ff18e078700 1 -- 172.21.15.6:0/364479651 wait complete. 2023-11-28T13:00:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.789+0000 7ff18e078700 1 Processor -- start 2023-11-28T13:00:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.789+0000 7ff18e078700 1 -- start start 2023-11-28T13:00:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff18e078700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1880caeb0 0x7ff188149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff18e078700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff18815c990 0x7ff1881e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff18e078700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff188161090 0x7ff1880c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff18e078700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1881e7c20 con 0x7ff18815c990 2023-11-28T13:00:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff18e078700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18814a420 con 0x7ff188161090 2023-11-28T13:00:32.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff18e078700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1881e6b80 con 0x7ff1880caeb0 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff186ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff18815c990 0x7ff1881e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff186ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff18815c990 0x7ff1881e61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57830/0 (socket says 172.21.15.6:57830) 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff186ffd700 1 -- 172.21.15.6:0/1226932672 learned_addr learned my addr 172.21.15.6:0/1226932672 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff1877fe700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1880caeb0 0x7ff188149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff187fff700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff188161090 0x7ff1880c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff186ffd700 1 -- 172.21.15.6:0/1226932672 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1880caeb0 msgr2=0x7ff188149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff186ffd700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1880caeb0 0x7ff188149ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff186ffd700 1 -- 172.21.15.6:0/1226932672 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff188161090 msgr2=0x7ff1880c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff186ffd700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff188161090 0x7ff1880c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff186ffd700 1 -- 172.21.15.6:0/1226932672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0094a0 con 0x7ff18815c990 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.790+0000 7ff186ffd700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff18815c990 0x7ff1881e61c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff17c014040 tx=0x7ff17c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.791+0000 7ff16ffff700 1 -- 172.21.15.6:0/1226932672 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c01a860 con 0x7ff18815c990 2023-11-28T13:00:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.791+0000 7ff16ffff700 1 -- 172.21.15.6:0/1226932672 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c01a9c0 con 0x7ff18815c990 2023-11-28T13:00:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.791+0000 7ff16ffff700 1 -- 172.21.15.6:0/1226932672 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c01ad50 con 0x7ff18815c990 2023-11-28T13:00:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.791+0000 7ff18e078700 1 -- 172.21.15.6:0/1226932672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1880ce870 con 0x7ff18815c990 2023-11-28T13:00:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.791+0000 7ff18e078700 1 -- 172.21.15.6:0/1226932672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1880ce590 con 0x7ff18815c990 2023-11-28T13:00:32.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.792+0000 7ff18e078700 1 -- 172.21.15.6:0/1226932672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1880cca20 con 0x7ff18815c990 2023-11-28T13:00:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.792+0000 7ff16ffff700 1 -- 172.21.15.6:0/1226932672 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff17c00f050 con 0x7ff18815c990 2023-11-28T13:00:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.793+0000 7ff16ffff700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1740682f0 0x7ff17406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.793+0000 7ff16ffff700 1 -- 172.21.15.6:0/1226932672 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff17c086f70 con 0x7ff18815c990 2023-11-28T13:00:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.793+0000 7ff1877fe700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1740682f0 0x7ff17406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:32.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.793+0000 7ff1877fe700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1740682f0 0x7ff17406a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff178007fd0 tx=0x7ff178009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:32.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:32.797+0000 7ff16ffff700 1 -- 172.21.15.6:0/1226932672 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff17c0b85e0 con 0x7ff18815c990 2023-11-28T13:00:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.008+0000 7ff18e078700 1 -- 172.21.15.6:0/1226932672 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff1880675a0 con 0x7ff18815c990 2023-11-28T13:00:33.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.008+0000 7ff16ffff700 1 -- 172.21.15.6:0/1226932672 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff17c029020 con 0x7ff18815c990 2023-11-28T13:00:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.010+0000 7ff18e078700 1 -- 172.21.15.6:0/1226932672 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1740682f0 msgr2=0x7ff17406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.010+0000 7ff18e078700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1740682f0 0x7ff17406a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff178007fd0 tx=0x7ff178009040 comp rx=0 tx=0).stop 2023-11-28T13:00:33.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.010+0000 7ff18e078700 1 -- 172.21.15.6:0/1226932672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff18815c990 msgr2=0x7ff1881e61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.010+0000 7ff18e078700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff18815c990 0x7ff1881e61c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff17c014040 tx=0x7ff17c004750 comp rx=0 tx=0).stop 2023-11-28T13:00:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.010+0000 7ff18e078700 1 -- 172.21.15.6:0/1226932672 shutdown_connections 2023-11-28T13:00:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.010+0000 7ff18e078700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff188161090 0x7ff1880c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.010+0000 7ff18e078700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1740682f0 0x7ff17406a7a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.010+0000 7ff18e078700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1880caeb0 0x7ff188149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.010+0000 7ff18e078700 1 --2- 172.21.15.6:0/1226932672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff18815c990 0x7ff1881e61c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.010+0000 7ff18e078700 1 -- 172.21.15.6:0/1226932672 >> 172.21.15.6:0/1226932672 conn(0x7ff1880b1650 msgr2=0x7ff1880af270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:33.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.011+0000 7ff18e078700 1 -- 172.21.15.6:0/1226932672 shutdown_connections 2023-11-28T13:00:33.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.011+0000 7ff18e078700 1 -- 172.21.15.6:0/1226932672 wait complete. 2023-11-28T13:00:33.014 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:00:33.024 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:00:33.024 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:00:33.042 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:33.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.205+0000 7f783d249700 1 Processor -- start 2023-11-28T13:00:33.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.205+0000 7f783d249700 1 -- start start 2023-11-28T13:00:33.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.205+0000 7f783d249700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f783815c990 0x7f78380cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.205+0000 7f783d249700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7838161090 0x7f78380ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.205+0000 7f783d249700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78380ce340 con 0x7f78380caeb0 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.205+0000 7f783d249700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78380cebe0 con 0x7f7838161090 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.205+0000 7f783d249700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78380cb3f0 con 0x7f783815c990 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f783759e700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f78380caeb0 0x7f78380cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45160/0 (socket says 172.21.15.6:45160) 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f783759e700 1 -- 172.21.15.6:0/2800394206 learned_addr learned my addr 172.21.15.6:0/2800394206 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7836d9d700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f783815c990 0x7f78380cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f783659c700 1 --2- 172.21.15.6:0/2800394206 >> v2:172.21.15.61:3300/0 conn(0x7f7838161090 0x7f78380ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803308626 0 0) 0x7f78380ce340 con 0x7f78380caeb0 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78200035b0 con 0x7f78380caeb0 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3234104022 0 0) 0x7f78200035b0 con 0x7f78380caeb0 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f783815c990 msgr2=0x7f78380cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:33.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f783815c990 0x7f78380cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 >> v2:172.21.15.61:3300/0 conn(0x7f7838161090 msgr2=0x7f78380ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 --2- 172.21.15.6:0/2800394206 >> v2:172.21.15.61:3300/0 conn(0x7f7838161090 0x7f78380ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78380ce590 con 0x7f78380caeb0 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f78300019f0 con 0x7f78380caeb0 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 >> v1:172.21.15.6:6789/0 conn(0x7f78380caeb0 legacy=0x7f78380cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7820003a00 0x7f7820003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f783815c990 0x7f7820004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7820004d90 0x7f7820005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78380cebe0 con 0x7f7820004d90 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78380cb3f0 con 0x7f7820003a00 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782000ba90 con 0x7f783815c990 2023-11-28T13:00:33.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f783759e700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7820004d90 0x7f7820005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f7836d9d700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7820003a00 0x7f7820003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.206+0000 7f783659c700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f783815c990 0x7f7820004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783759e700 1 -- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f783815c990 msgr2=0x7f7820004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783759e700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f783815c990 0x7f7820004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783759e700 1 -- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7820003a00 msgr2=0x7f7820003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783759e700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7820003a00 0x7f7820003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783759e700 1 -- 172.21.15.6:0/2800394206 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200035b0 con 0x7f7820004d90 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783759e700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7820004d90 0x7f7820005200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f783000a860 tx=0x7f7820005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6255d5a1d923124b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830012600 con 0x7f7820004d90 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7830012780 con 0x7f7820004d90 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f7835d9b700 1 -- 172.21.15.6:0/2800394206 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830012b30 con 0x7f7820004d90 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783d249700 1 -- 172.21.15.6:0/2800394206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7820004d90 msgr2=0x7f7820005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783d249700 1 --2- 172.21.15.6:0/2800394206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7820004d90 0x7f7820005200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f783000a860 tx=0x7f7820005880 comp rx=0 tx=0).stop 2023-11-28T13:00:33.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783759e700 1 -- 172.21.15.6:0/2800394206 reap_dead start 2023-11-28T13:00:33.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783d249700 1 -- 172.21.15.6:0/2800394206 shutdown_connections 2023-11-28T13:00:33.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783d249700 1 -- 172.21.15.6:0/2800394206 >> 172.21.15.6:0/2800394206 conn(0x7f78380b1650 msgr2=0x7f78380b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:33.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.207+0000 7f783d249700 1 -- 172.21.15.6:0/2800394206 shutdown_connections 2023-11-28T13:00:33.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.208+0000 7f783d249700 1 -- 172.21.15.6:0/2800394206 wait complete. 2023-11-28T13:00:33.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.209+0000 7f783d249700 1 Processor -- start 2023-11-28T13:00:33.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.209+0000 7f783d249700 1 -- start start 2023-11-28T13:00:33.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783d249700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78380b39a0 0x7f78380c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783d249700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f783815f920 0x7f78380c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783d249700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78380b3d70 0x7f78380b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783d249700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78380c5180 con 0x7f78380b3d70 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783d249700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78380c53d0 con 0x7f783815f920 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783d249700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78380c56b0 con 0x7f78380b39a0 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783759e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78380b3d70 0x7f78380b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783759e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78380b3d70 0x7f78380b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57852/0 (socket says 172.21.15.6:57852) 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783759e700 1 -- 172.21.15.6:0/4216915756 learned_addr learned my addr 172.21.15.6:0/4216915756 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f7836d9d700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78380b39a0 0x7f78380c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783659c700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f783815f920 0x7f78380c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783759e700 1 -- 172.21.15.6:0/4216915756 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78380b39a0 msgr2=0x7f78380c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783759e700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78380b39a0 0x7f78380c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783759e700 1 -- 172.21.15.6:0/4216915756 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f783815f920 msgr2=0x7f78380c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783759e700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f783815f920 0x7f78380c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783759e700 1 -- 172.21.15.6:0/4216915756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78380cca20 con 0x7f78380b3d70 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f783759e700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78380b3d70 0x7f78380b4210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7830015040 tx=0x7f7830001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.210+0000 7f781f7fe700 1 -- 172.21.15.6:0/4216915756 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78300019b0 con 0x7f78380b3d70 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.211+0000 7f781f7fe700 1 -- 172.21.15.6:0/4216915756 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7830012610 con 0x7f78380b3d70 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.211+0000 7f783d249700 1 -- 172.21.15.6:0/4216915756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78380ce870 con 0x7f78380b3d70 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.211+0000 7f781f7fe700 1 -- 172.21.15.6:0/4216915756 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783001d610 con 0x7f78380b3d70 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.211+0000 7f783d249700 1 -- 172.21.15.6:0/4216915756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78380ccbb0 con 0x7f78380b3d70 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.212+0000 7f781f7fe700 1 -- 172.21.15.6:0/4216915756 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f783002e070 con 0x7f78380b3d70 2023-11-28T13:00:33.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.212+0000 7f783d249700 1 -- 172.21.15.6:0/4216915756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78380675a0 con 0x7f78380b3d70 2023-11-28T13:00:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.212+0000 7f781f7fe700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78240682f0 0x7f782406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.212+0000 7f781f7fe700 1 -- 172.21.15.6:0/4216915756 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7830087630 con 0x7f78380b3d70 2023-11-28T13:00:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.212+0000 7f7836d9d700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78240682f0 0x7f782406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.213+0000 7f7836d9d700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78240682f0 0x7f782406a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f782c009c60 tx=0x7f782c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.217+0000 7f781f7fe700 1 -- 172.21.15.6:0/4216915756 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7830053c60 con 0x7f78380b3d70 2023-11-28T13:00:33.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.414+0000 7f783d249700 1 -- 172.21.15.6:0/4216915756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78380cdc50 con 0x7f78380b3d70 2023-11-28T13:00:33.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.416+0000 7f781f7fe700 1 -- 172.21.15.6:0/4216915756 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f7830017090 con 0x7f78380b3d70 2023-11-28T13:00:33.418 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:33.418 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:33.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 -- 172.21.15.6:0/4216915756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78240682f0 msgr2=0x7f782406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78240682f0 0x7f782406a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f782c009c60 tx=0x7f782c009210 comp rx=0 tx=0).stop 2023-11-28T13:00:33.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 -- 172.21.15.6:0/4216915756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78380b3d70 msgr2=0x7f78380b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78380b3d70 0x7f78380b4210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7830015040 tx=0x7f7830001d80 comp rx=0 tx=0).stop 2023-11-28T13:00:33.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 -- 172.21.15.6:0/4216915756 shutdown_connections 2023-11-28T13:00:33.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78380b3d70 0x7f78380b4210 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78240682f0 0x7f782406a7a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f78380b39a0 0x7f78380c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 --2- 172.21.15.6:0/4216915756 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f783815f920 0x7f78380c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 -- 172.21.15.6:0/4216915756 >> 172.21.15.6:0/4216915756 conn(0x7f78380b1650 msgr2=0x7f78380b0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:33.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 -- 172.21.15.6:0/4216915756 shutdown_connections 2023-11-28T13:00:33.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.418+0000 7f783d249700 1 -- 172.21.15.6:0/4216915756 wait complete. 2023-11-28T13:00:33.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:00:33.432 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:33.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.561+0000 7f9e2c818700 1 Processor -- start 2023-11-28T13:00:33.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.561+0000 7f9e2c818700 1 -- start start 2023-11-28T13:00:33.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e2c818700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9e280b1630 0x7f9e280cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e2c818700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9e2806afc0 0x7f9e2806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e2c818700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e280ce550 con 0x7f9e280b1a00 2023-11-28T13:00:33.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e2c818700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e280ce830 con 0x7f9e2806afc0 2023-11-28T13:00:33.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e2c818700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e280ceb10 con 0x7f9e280b1630 2023-11-28T13:00:33.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e25d9b700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f9e280b1a00 0x7f9e280caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45180/0 (socket says 172.21.15.6:45180) 2023-11-28T13:00:33.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e25d9b700 1 -- 172.21.15.6:0/3450789212 learned_addr learned my addr 172.21.15.6:0/3450789212 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:33.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e26d9d700 1 --2- 172.21.15.6:0/3450789212 >> v2:172.21.15.61:3300/0 conn(0x7f9e2806afc0 0x7f9e2806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e2659c700 1 --2- 172.21.15.6:0/3450789212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9e280b1630 0x7f9e280cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e2559a700 1 -- 172.21.15.6:0/3450789212 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504180032 0 0) 0x7f9e280ce550 con 0x7f9e280b1a00 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e2559a700 1 -- 172.21.15.6:0/3450789212 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e100035b0 con 0x7f9e280b1a00 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e26d9d700 1 -- 172.21.15.6:0/3450789212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9e280b1630 msgr2=0x7f9e280cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e26d9d700 1 --2- 172.21.15.6:0/3450789212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9e280b1630 0x7f9e280cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e26d9d700 1 -- 172.21.15.6:0/3450789212 >> v1:172.21.15.6:6789/0 conn(0x7f9e280b1a00 legacy=0x7f9e280caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.562+0000 7f9e26d9d700 1 -- 172.21.15.6:0/3450789212 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e280cedf0 con 0x7f9e2806afc0 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e26d9d700 1 --2- 172.21.15.6:0/3450789212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e2806afc0 0x7f9e2806b440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009d90 tx=0x7f9e2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2f7f74e7f40706 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2559a700 1 -- 172.21.15.6:0/3450789212 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000e070 con 0x7f9e2806afc0 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2559a700 1 -- 172.21.15.6:0/3450789212 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2000bb20 con 0x7f9e2806afc0 2023-11-28T13:00:33.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2559a700 1 -- 172.21.15.6:0/3450789212 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20012430 con 0x7f9e2806afc0 2023-11-28T13:00:33.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2c818700 1 -- 172.21.15.6:0/3450789212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e2806afc0 msgr2=0x7f9e2806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2c818700 1 --2- 172.21.15.6:0/3450789212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e2806afc0 0x7f9e2806b440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9e20009d90 tx=0x7f9e2000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:33.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2c818700 1 -- 172.21.15.6:0/3450789212 shutdown_connections 2023-11-28T13:00:33.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2c818700 1 --2- 172.21.15.6:0/3450789212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e2806afc0 0x7f9e2806b440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2c818700 1 --2- 172.21.15.6:0/3450789212 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9e280b1630 0x7f9e280cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2c818700 1 -- 172.21.15.6:0/3450789212 >> 172.21.15.6:0/3450789212 conn(0x7f9e280cc630 msgr2=0x7f9e280cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:33.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2c818700 1 -- 172.21.15.6:0/3450789212 shutdown_connections 2023-11-28T13:00:33.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.563+0000 7f9e2c818700 1 -- 172.21.15.6:0/3450789212 wait complete. 2023-11-28T13:00:33.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.565+0000 7f9e2c818700 1 Processor -- start 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.565+0000 7f9e2c818700 1 -- start start 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.565+0000 7f9e2c818700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e2806afc0 0x7f9e2815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.565+0000 7f9e2c818700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9e280b1630 0x7f9e281f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.565+0000 7f9e2c818700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e280b1a00 0x7f9e280c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.565+0000 7f9e2c818700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e281f2d30 con 0x7f9e280b1a00 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.565+0000 7f9e2c818700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e281f3010 con 0x7f9e2806afc0 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.565+0000 7f9e2c818700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e281f32f0 con 0x7f9e280b1630 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e26d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e280b1a00 0x7f9e280c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e26d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e280b1a00 0x7f9e280c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57868/0 (socket says 172.21.15.6:57868) 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e26d9d700 1 -- 172.21.15.6:0/104681837 learned_addr learned my addr 172.21.15.6:0/104681837 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e2659c700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e2806afc0 0x7f9e2815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e25d9b700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9e280b1630 0x7f9e281f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e26d9d700 1 -- 172.21.15.6:0/104681837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9e280b1630 msgr2=0x7f9e281f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e26d9d700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9e280b1630 0x7f9e281f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e26d9d700 1 -- 172.21.15.6:0/104681837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e2806afc0 msgr2=0x7f9e2815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e26d9d700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e2806afc0 0x7f9e2815b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e26d9d700 1 -- 172.21.15.6:0/104681837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e20009a70 con 0x7f9e280b1a00 2023-11-28T13:00:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e26d9d700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e280b1a00 0x7f9e280c8e50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000b370 tx=0x7f9e20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e16ffd700 1 -- 172.21.15.6:0/104681837 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20012430 con 0x7f9e280b1a00 2023-11-28T13:00:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e16ffd700 1 -- 172.21.15.6:0/104681837 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e20020070 con 0x7f9e280b1a00 2023-11-28T13:00:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e2c818700 1 -- 172.21.15.6:0/104681837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e280b1e70 con 0x7f9e280b1a00 2023-11-28T13:00:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e16ffd700 1 -- 172.21.15.6:0/104681837 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e200098f0 con 0x7f9e280b1a00 2023-11-28T13:00:33.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.566+0000 7f9e2c818700 1 -- 172.21.15.6:0/104681837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e280cf0d0 con 0x7f9e280b1a00 2023-11-28T13:00:33.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.567+0000 7f9e2c818700 1 -- 172.21.15.6:0/104681837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e280cb5b0 con 0x7f9e280b1a00 2023-11-28T13:00:33.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.568+0000 7f9e16ffd700 1 -- 172.21.15.6:0/104681837 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9e2002e070 con 0x7f9e280b1a00 2023-11-28T13:00:33.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.568+0000 7f9e16ffd700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9e080682f0 0x7f9e0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.568+0000 7f9e16ffd700 1 -- 172.21.15.6:0/104681837 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9e20022070 con 0x7f9e280b1a00 2023-11-28T13:00:33.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.568+0000 7f9e2659c700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9e080682f0 0x7f9e0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.569+0000 7f9e2659c700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9e080682f0 0x7f9e0806a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e280c9b30 tx=0x7f9e18009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.573+0000 7f9e16ffd700 1 -- 172.21.15.6:0/104681837 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9e20054080 con 0x7f9e280b1a00 2023-11-28T13:00:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.763+0000 7f9e2c818700 1 -- 172.21.15.6:0/104681837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9e280739f0 con 0x7f9e280b1a00 2023-11-28T13:00:33.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.764+0000 7f9e16ffd700 1 -- 172.21.15.6:0/104681837 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9e2002c020 con 0x7f9e280b1a00 2023-11-28T13:00:33.765 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:00:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 -- 172.21.15.6:0/104681837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9e080682f0 msgr2=0x7f9e0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9e080682f0 0x7f9e0806a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e280c9b30 tx=0x7f9e18009210 comp rx=0 tx=0).stop 2023-11-28T13:00:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 -- 172.21.15.6:0/104681837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e280b1a00 msgr2=0x7f9e280c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e280b1a00 0x7f9e280c8e50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000b370 tx=0x7f9e20004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 -- 172.21.15.6:0/104681837 shutdown_connections 2023-11-28T13:00:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9e280b1a00 0x7f9e280c8e50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e2806afc0 0x7f9e2815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9e080682f0 0x7f9e0806a7a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 --2- 172.21.15.6:0/104681837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9e280b1630 0x7f9e281f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 -- 172.21.15.6:0/104681837 >> 172.21.15.6:0/104681837 conn(0x7f9e280cc630 msgr2=0x7f9e280caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 -- 172.21.15.6:0/104681837 shutdown_connections 2023-11-28T13:00:33.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.766+0000 7f9e2c818700 1 -- 172.21.15.6:0/104681837 wait complete. 2023-11-28T13:00:33.779 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.905+0000 7f7fecf57700 1 Processor -- start 2023-11-28T13:00:33.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.905+0000 7f7fecf57700 1 -- start start 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fecf57700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7fe80b2680 0x7f7fe80b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fecf57700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe806f940 con 0x7f7fe815fc00 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fecf57700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe806fc20 con 0x7f7fe80cbbf0 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fecf57700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe80b3030 con 0x7f7fe80b2680 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fe5d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7fe80b2680 0x7f7fe80b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fe659c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f7fe815fc00 0x7f7fe806eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45192/0 (socket says 172.21.15.6:45192) 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fe659c700 1 -- 172.21.15.6:0/32205632 learned_addr learned my addr 172.21.15.6:0/32205632 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fe5d9b700 1 -- 172.21.15.6:0/32205632 >> v1:172.21.15.61:6789/0 conn(0x7f7fe80cbbf0 legacy=0x7f7fe80cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fe5d9b700 1 -- 172.21.15.6:0/32205632 >> v1:172.21.15.6:6789/0 conn(0x7f7fe815fc00 legacy=0x7f7fe806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fe5d9b700 1 -- 172.21.15.6:0/32205632 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe80cb3f0 con 0x7f7fe80b2680 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.906+0000 7f7fe5d9b700 1 --2- 172.21.15.6:0/32205632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe80b2680 0x7f7fe80b2af0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80d03c0 tx=0x7f7fdc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6213fe9b151296a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.907+0000 7f7fe559a700 1 -- 172.21.15.6:0/32205632 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc011830 con 0x7f7fe80b2680 2023-11-28T13:00:33.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.907+0000 7f7fe559a700 1 -- 172.21.15.6:0/32205632 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc011990 con 0x7f7fe80b2680 2023-11-28T13:00:33.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.907+0000 7f7fe559a700 1 -- 172.21.15.6:0/32205632 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc011d20 con 0x7f7fe80b2680 2023-11-28T13:00:33.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.907+0000 7f7fecf57700 1 -- 172.21.15.6:0/32205632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe80b2680 msgr2=0x7f7fe80b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.907+0000 7f7fecf57700 1 --2- 172.21.15.6:0/32205632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe80b2680 0x7f7fe80b2af0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80d03c0 tx=0x7f7fdc009b20 comp rx=0 tx=0).stop 2023-11-28T13:00:33.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.907+0000 7f7fecf57700 1 -- 172.21.15.6:0/32205632 shutdown_connections 2023-11-28T13:00:33.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.907+0000 7f7fecf57700 1 --2- 172.21.15.6:0/32205632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe80b2680 0x7f7fe80b2af0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.907+0000 7f7fecf57700 1 -- 172.21.15.6:0/32205632 >> 172.21.15.6:0/32205632 conn(0x7f7fe80cd680 msgr2=0x7f7fe80cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:33.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.907+0000 7f7fecf57700 1 -- 172.21.15.6:0/32205632 shutdown_connections 2023-11-28T13:00:33.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.907+0000 7f7fecf57700 1 -- 172.21.15.6:0/32205632 wait complete. 2023-11-28T13:00:33.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.909+0000 7f7fecf57700 1 Processor -- start 2023-11-28T13:00:33.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.909+0000 7f7fecf57700 1 -- start start 2023-11-28T13:00:33.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.909+0000 7f7fecf57700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7fe80b2680 0x7f7fe8158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.909+0000 7f7fecf57700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7fe80cbbf0 0x7f7fe80c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.909+0000 7f7fecf57700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe815fc00 0x7f7fe815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.909+0000 7f7fecf57700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80c7510 con 0x7f7fe815fc00 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.909+0000 7f7fecf57700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81f2270 con 0x7f7fe80cbbf0 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.909+0000 7f7fecf57700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81f2530 con 0x7f7fe80b2680 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.909+0000 7f7fe6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe815fc00 0x7f7fe815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.909+0000 7f7fe6d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe815fc00 0x7f7fe815e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57882/0 (socket says 172.21.15.6:57882) 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fe6d9d700 1 -- 172.21.15.6:0/2455447352 learned_addr learned my addr 172.21.15.6:0/2455447352 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fe5d9b700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7fe80cbbf0 0x7f7fe80c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fe659c700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7fe80b2680 0x7f7fe8158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fe6d9d700 1 -- 172.21.15.6:0/2455447352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7fe80b2680 msgr2=0x7f7fe8158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fe6d9d700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7fe80b2680 0x7f7fe8158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fe6d9d700 1 -- 172.21.15.6:0/2455447352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7fe80cbbf0 msgr2=0x7f7fe80c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fe6d9d700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7fe80cbbf0 0x7f7fe80c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:33.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fe6d9d700 1 -- 172.21.15.6:0/2455447352 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc0094a0 con 0x7f7fe815fc00 2023-11-28T13:00:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fe5d9b700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7fe80cbbf0 0x7f7fe80c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fe6d9d700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe815fc00 0x7f7fe815e240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000cc10 tx=0x7f7fe000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fd6ffd700 1 -- 172.21.15.6:0/2455447352 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000abb0 con 0x7f7fe815fc00 2023-11-28T13:00:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fd6ffd700 1 -- 172.21.15.6:0/2455447352 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe000ad10 con 0x7f7fe815fc00 2023-11-28T13:00:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fecf57700 1 -- 172.21.15.6:0/2455447352 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe815ffd0 con 0x7f7fe815fc00 2023-11-28T13:00:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fd6ffd700 1 -- 172.21.15.6:0/2455447352 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0012660 con 0x7f7fe815fc00 2023-11-28T13:00:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.910+0000 7f7fecf57700 1 -- 172.21.15.6:0/2455447352 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe80cb6d0 con 0x7f7fe815fc00 2023-11-28T13:00:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.911+0000 7f7fecf57700 1 -- 172.21.15.6:0/2455447352 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe806f5b0 con 0x7f7fe815fc00 2023-11-28T13:00:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.911+0000 7f7fd6ffd700 1 -- 172.21.15.6:0/2455447352 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7fe0022070 con 0x7f7fe815fc00 2023-11-28T13:00:33.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.912+0000 7f7fd6ffd700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7fc80682f0 0x7f7fc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:33.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.912+0000 7f7fd6ffd700 1 -- 172.21.15.6:0/2455447352 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7fe0086650 con 0x7f7fe815fc00 2023-11-28T13:00:33.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.912+0000 7f7fe659c700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7fc80682f0 0x7f7fc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:33.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.912+0000 7f7fe659c700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7fc80682f0 0x7f7fc806a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8005910 tx=0x7f7fd8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:33.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:33.916+0000 7f7fd6ffd700 1 -- 172.21.15.6:0/2455447352 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7fe0052b30 con 0x7f7fe815fc00 2023-11-28T13:00:34.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.110+0000 7f7fecf57700 1 -- 172.21.15.6:0/2455447352 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fe80739f0 con 0x7f7fe815fc00 2023-11-28T13:00:34.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.111+0000 7f7fd6ffd700 1 -- 172.21.15.6:0/2455447352 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fe0016070 con 0x7f7fe815fc00 2023-11-28T13:00:34.112 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:00:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.112+0000 7f7fecf57700 1 -- 172.21.15.6:0/2455447352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7fc80682f0 msgr2=0x7f7fc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.112+0000 7f7fecf57700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7fc80682f0 0x7f7fc806a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8005910 tx=0x7f7fd8009040 comp rx=0 tx=0).stop 2023-11-28T13:00:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.112+0000 7f7fecf57700 1 -- 172.21.15.6:0/2455447352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe815fc00 msgr2=0x7f7fe815e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.112+0000 7f7fecf57700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe815fc00 0x7f7fe815e240 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000cc10 tx=0x7f7fe000a060 comp rx=0 tx=0).stop 2023-11-28T13:00:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.112+0000 7f7fecf57700 1 -- 172.21.15.6:0/2455447352 shutdown_connections 2023-11-28T13:00:34.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.112+0000 7f7fecf57700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7fe815fc00 0x7f7fe815e240 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.113+0000 7f7fecf57700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7fc80682f0 0x7f7fc806a7a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.113+0000 7f7fecf57700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7fe80b2680 0x7f7fe8158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.113+0000 7f7fecf57700 1 --2- 172.21.15.6:0/2455447352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7fe80cbbf0 0x7f7fe80c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.113+0000 7f7fecf57700 1 -- 172.21.15.6:0/2455447352 >> 172.21.15.6:0/2455447352 conn(0x7f7fe80cd680 msgr2=0x7f7fe80b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:34.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.113+0000 7f7fecf57700 1 -- 172.21.15.6:0/2455447352 shutdown_connections 2023-11-28T13:00:34.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.113+0000 7f7fecf57700 1 -- 172.21.15.6:0/2455447352 wait complete. 2023-11-28T13:00:34.126 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:34.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.252+0000 7f789b872700 1 Processor -- start 2023-11-28T13:00:34.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.253+0000 7f789b872700 1 -- start start 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.253+0000 7f789b872700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f78940b1630 0x7f78940cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.253+0000 7f789b872700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789406afc0 0x7f789406b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.253+0000 7f789b872700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78940ce550 con 0x7f78940b1a00 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.253+0000 7f789b872700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78940ce830 con 0x7f78940b1630 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.253+0000 7f789b872700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78940ceb10 con 0x7f789406afc0 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7898e0d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f78940b1a00 0x7f78940caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45206/0 (socket says 172.21.15.6:45206) 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7898e0d700 1 -- 172.21.15.6:0/2093526647 learned_addr learned my addr 172.21.15.6:0/2093526647 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7899e0f700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789406afc0 0x7f789406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f789960e700 1 --2- 172.21.15.6:0/2093526647 >> v2:172.21.15.61:3300/0 conn(0x7f78940b1630 0x7f78940cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637830426 0 0) 0x7f78940ce550 con 0x7f78940b1a00 2023-11-28T13:00:34.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78800035b0 con 0x7f78940b1a00 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1750254956 0 0) 0x7f78800035b0 con 0x7f78940b1a00 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789406afc0 msgr2=0x7f789406b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789406afc0 0x7f789406b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 >> v2:172.21.15.61:3300/0 conn(0x7f78940b1630 msgr2=0x7f78940cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 --2- 172.21.15.6:0/2093526647 >> v2:172.21.15.61:3300/0 conn(0x7f78940b1630 0x7f78940cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940cedf0 con 0x7f78940b1a00 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f7888001670 con 0x7f78940b1a00 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 >> v1:172.21.15.6:6789/0 conn(0x7f78940b1a00 legacy=0x7f78940caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789406afc0 0x7f7880003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 0x7f7880004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7880004d90 0x7f7880005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940ce830 con 0x7f7880003f80 2023-11-28T13:00:34.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940ceb10 con 0x7f7880004d90 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788000ba10 con 0x7f789406afc0 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7898e0d700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 0x7f7880004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f789960e700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789406afc0 0x7f7880003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.254+0000 7f7899e0f700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7880004d90 0x7f7880005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f7898e0d700 1 -- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789406afc0 msgr2=0x7f7880003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f7898e0d700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789406afc0 0x7f7880003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f7898e0d700 1 -- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7880004d90 msgr2=0x7f7880005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f7898e0d700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7880004d90 0x7f7880005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f7898e0d700 1 -- 172.21.15.6:0/2093526647 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78800035b0 con 0x7f7880003f80 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f7898e0d700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 0x7f7880004850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7888001050 tx=0x7f7880005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b805a719962723d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788800cd80 con 0x7f7880003f80 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7888005960 con 0x7f7880003f80 2023-11-28T13:00:34.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f7887fff700 1 -- 172.21.15.6:0/2093526647 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7888005d10 con 0x7f7880003f80 2023-11-28T13:00:34.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f789b872700 1 -- 172.21.15.6:0/2093526647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 msgr2=0x7f7880004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f789b872700 1 --2- 172.21.15.6:0/2093526647 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 0x7f7880004850 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7888001050 tx=0x7f7880005880 comp rx=0 tx=0).stop 2023-11-28T13:00:34.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f7899e0f700 1 -- 172.21.15.6:0/2093526647 reap_dead start 2023-11-28T13:00:34.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f789b872700 1 -- 172.21.15.6:0/2093526647 shutdown_connections 2023-11-28T13:00:34.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f789b872700 1 -- 172.21.15.6:0/2093526647 >> 172.21.15.6:0/2093526647 conn(0x7f78940cc630 msgr2=0x7f78940cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:34.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f789b872700 1 -- 172.21.15.6:0/2093526647 shutdown_connections 2023-11-28T13:00:34.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.255+0000 7f789b872700 1 -- 172.21.15.6:0/2093526647 wait complete. 2023-11-28T13:00:34.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.257+0000 7f789b872700 1 Processor -- start 2023-11-28T13:00:34.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.257+0000 7f789b872700 1 -- start start 2023-11-28T13:00:34.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.257+0000 7f789b872700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 0x7f78940c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.257+0000 7f789b872700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789415d4b0 0x7f7894154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.257+0000 7f789b872700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7894156a90 0x7f78940b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.257+0000 7f789b872700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789415f9a0 con 0x7f7880003f80 2023-11-28T13:00:34.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.257+0000 7f789b872700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789415f3e0 con 0x7f789415d4b0 2023-11-28T13:00:34.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.257+0000 7f789b872700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789415f6c0 con 0x7f7894156a90 2023-11-28T13:00:34.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.257+0000 7f789960e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 0x7f78940c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f789960e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 0x7f78940c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57898/0 (socket says 172.21.15.6:57898) 2023-11-28T13:00:34.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f789960e700 1 -- 172.21.15.6:0/207641614 learned_addr learned my addr 172.21.15.6:0/207641614 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f7899e0f700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7894156a90 0x7f78940b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f7898e0d700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789415d4b0 0x7f7894154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f789960e700 1 -- 172.21.15.6:0/207641614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7894156a90 msgr2=0x7f78940b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f789960e700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7894156a90 0x7f78940b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f789960e700 1 -- 172.21.15.6:0/207641614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789415d4b0 msgr2=0x7f7894154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f789960e700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789415d4b0 0x7f7894154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f789960e700 1 -- 172.21.15.6:0/207641614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940b1ea0 con 0x7f7880003f80 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f7899e0f700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7894156a90 0x7f78940b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f7898e0d700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789415d4b0 0x7f7894154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f789960e700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 0x7f78940c63e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7890011aa0 tx=0x7f7890011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f7885ffb700 1 -- 172.21.15.6:0/207641614 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000f980 con 0x7f7880003f80 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f7885ffb700 1 -- 172.21.15.6:0/207641614 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789000fb00 con 0x7f7880003f80 2023-11-28T13:00:34.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f789b872700 1 -- 172.21.15.6:0/207641614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78940cf0d0 con 0x7f7880003f80 2023-11-28T13:00:34.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f7885ffb700 1 -- 172.21.15.6:0/207641614 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789000feb0 con 0x7f7880003f80 2023-11-28T13:00:34.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.258+0000 7f789b872700 1 -- 172.21.15.6:0/207641614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78940cb5b0 con 0x7f7880003f80 2023-11-28T13:00:34.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.259+0000 7f789b872700 1 -- 172.21.15.6:0/207641614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78940739f0 con 0x7f7880003f80 2023-11-28T13:00:34.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.260+0000 7f7885ffb700 1 -- 172.21.15.6:0/207641614 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7890014440 con 0x7f7880003f80 2023-11-28T13:00:34.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.260+0000 7f7885ffb700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78780682f0 0x7f787806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.260+0000 7f7885ffb700 1 -- 172.21.15.6:0/207641614 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f78900868b0 con 0x7f7880003f80 2023-11-28T13:00:34.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.260+0000 7f7898e0d700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78780682f0 0x7f787806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.261+0000 7f7898e0d700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78780682f0 0x7f787806a7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f78940b4b30 tx=0x7f788800a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:34.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.264+0000 7f7885ffb700 1 -- 172.21.15.6:0/207641614 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7890052e80 con 0x7f7880003f80 2023-11-28T13:00:34.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.456+0000 7f789b872700 1 -- 172.21.15.6:0/207641614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78940cac80 con 0x7f7880003f80 2023-11-28T13:00:34.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.456+0000 7f7885ffb700 1 -- 172.21.15.6:0/207641614 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78900564a0 con 0x7f7880003f80 2023-11-28T13:00:34.458 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:00:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.458+0000 7f789b872700 1 -- 172.21.15.6:0/207641614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78780682f0 msgr2=0x7f787806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.458+0000 7f789b872700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78780682f0 0x7f787806a7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f78940b4b30 tx=0x7f788800a000 comp rx=0 tx=0).stop 2023-11-28T13:00:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.458+0000 7f789b872700 1 -- 172.21.15.6:0/207641614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 msgr2=0x7f78940c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.458+0000 7f789b872700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 0x7f78940c63e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7890011aa0 tx=0x7f7890011e60 comp rx=0 tx=0).stop 2023-11-28T13:00:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.459+0000 7f789b872700 1 -- 172.21.15.6:0/207641614 shutdown_connections 2023-11-28T13:00:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.459+0000 7f789b872700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78780682f0 0x7f787806a7a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.459+0000 7f789b872700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7894156a90 0x7f78940b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.459+0000 7f789b872700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7880003f80 0x7f78940c63e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.459+0000 7f789b872700 1 --2- 172.21.15.6:0/207641614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789415d4b0 0x7f7894154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.459+0000 7f789b872700 1 -- 172.21.15.6:0/207641614 >> 172.21.15.6:0/207641614 conn(0x7f78940cc630 msgr2=0x7f78940caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:34.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.459+0000 7f789b872700 1 -- 172.21.15.6:0/207641614 shutdown_connections 2023-11-28T13:00:34.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.459+0000 7f789b872700 1 -- 172.21.15.6:0/207641614 wait complete. 2023-11-28T13:00:34.472 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:34.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.598+0000 7f2566ba6700 1 Processor -- start 2023-11-28T13:00:34.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.598+0000 7f2566ba6700 1 -- start start 2023-11-28T13:00:34.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.598+0000 7f2566ba6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f25600b1630 0x7f25600cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2566ba6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f25600b1a00 0x7f25600caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2566ba6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25600cb3e0 0x7f2560160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2566ba6700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25600ce550 con 0x7f25600b1630 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2566ba6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25600ce830 con 0x7f25600b1a00 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2566ba6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25600ceb10 con 0x7f25600cb3e0 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2564942700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f25600b1630 0x7f25600cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2564942700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f25600b1630 0x7f25600cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57906/0 (socket says 172.21.15.6:57906) 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2564942700 1 -- 172.21.15.6:0/1534703816 learned_addr learned my addr 172.21.15.6:0/1534703816 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2564942700 1 -- 172.21.15.6:0/1534703816 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25600cb3e0 msgr2=0x7f2560160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2564942700 1 --2- 172.21.15.6:0/1534703816 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25600cb3e0 0x7f2560160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2564942700 1 -- 172.21.15.6:0/1534703816 >> v2:172.21.15.61:3300/0 conn(0x7f25600b1a00 msgr2=0x7f25600caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:34.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f255ffff700 1 --2- 172.21.15.6:0/1534703816 >> v2:172.21.15.61:3300/0 conn(0x7f25600b1a00 0x7f25600caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2564942700 1 --2- 172.21.15.6:0/1534703816 >> v2:172.21.15.61:3300/0 conn(0x7f25600b1a00 0x7f25600caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2564942700 1 -- 172.21.15.6:0/1534703816 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25600cedf0 con 0x7f25600b1630 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f255ffff700 1 --2- 172.21.15.6:0/1534703816 >> v2:172.21.15.61:3300/0 conn(0x7f25600b1a00 0x7f25600caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f2564942700 1 --2- 172.21.15.6:0/1534703816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 0x7f25600cf5f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e50 tx=0x7f255800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e10a0f7b3435ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f255f7fe700 1 -- 172.21.15.6:0/1534703816 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800e070 con 0x7f25600b1630 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f255f7fe700 1 -- 172.21.15.6:0/1534703816 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255800b980 con 0x7f25600b1630 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.599+0000 7f255f7fe700 1 -- 172.21.15.6:0/1534703816 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800bd30 con 0x7f25600b1630 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.600+0000 7f2566ba6700 1 -- 172.21.15.6:0/1534703816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 msgr2=0x7f25600cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.600+0000 7f2566ba6700 1 --2- 172.21.15.6:0/1534703816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 0x7f25600cf5f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e50 tx=0x7f255800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.600+0000 7f2566ba6700 1 -- 172.21.15.6:0/1534703816 shutdown_connections 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.600+0000 7f2566ba6700 1 --2- 172.21.15.6:0/1534703816 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25600cb3e0 0x7f2560160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.600+0000 7f2566ba6700 1 --2- 172.21.15.6:0/1534703816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 0x7f25600cf5f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.600+0000 7f2566ba6700 1 --2- 172.21.15.6:0/1534703816 >> v2:172.21.15.61:3300/0 conn(0x7f25600b1a00 0x7f25600caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.600+0000 7f2566ba6700 1 -- 172.21.15.6:0/1534703816 >> 172.21.15.6:0/1534703816 conn(0x7f25600cc630 msgr2=0x7f25600cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:34.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.600+0000 7f2566ba6700 1 -- 172.21.15.6:0/1534703816 shutdown_connections 2023-11-28T13:00:34.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.600+0000 7f2566ba6700 1 -- 172.21.15.6:0/1534703816 wait complete. 2023-11-28T13:00:34.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.601+0000 7f2566ba6700 1 Processor -- start 2023-11-28T13:00:34.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.601+0000 7f2566ba6700 1 -- start start 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2566ba6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 0x7f25600c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2566ba6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600b1a00 0x7f256015f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2566ba6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25600cb3e0 0x7f25600b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2566ba6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256015fbf0 con 0x7f25600b1630 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2566ba6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b4280 con 0x7f25600b1a00 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2566ba6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b4560 con 0x7f25600cb3e0 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2564942700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 0x7f25600c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2564942700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 0x7f25600c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57912/0 (socket says 172.21.15.6:57912) 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2564942700 1 -- 172.21.15.6:0/1303569120 learned_addr learned my addr 172.21.15.6:0/1303569120 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2565143700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25600cb3e0 0x7f25600b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f255ffff700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600b1a00 0x7f256015f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2564942700 1 -- 172.21.15.6:0/1303569120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25600cb3e0 msgr2=0x7f25600b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2564942700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25600cb3e0 0x7f25600b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2564942700 1 -- 172.21.15.6:0/1303569120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600b1a00 msgr2=0x7f256015f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2564942700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600b1a00 0x7f256015f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f2564942700 1 -- 172.21.15.6:0/1303569120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558009b30 con 0x7f25600b1630 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.602+0000 7f255ffff700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600b1a00 0x7f256015f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.603+0000 7f2564942700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 0x7f25600c62d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f255800b370 tx=0x7f2558004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.603+0000 7f255d7fa700 1 -- 172.21.15.6:0/1303569120 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558004250 con 0x7f25600b1630 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.603+0000 7f255d7fa700 1 -- 172.21.15.6:0/1303569120 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25580043b0 con 0x7f25600b1630 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.603+0000 7f2566ba6700 1 -- 172.21.15.6:0/1303569120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25600b1ea0 con 0x7f25600b1630 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.603+0000 7f255d7fa700 1 -- 172.21.15.6:0/1303569120 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2558004760 con 0x7f25600b1630 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.603+0000 7f2566ba6700 1 -- 172.21.15.6:0/1303569120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25600cf0d0 con 0x7f25600b1630 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.604+0000 7f2566ba6700 1 -- 172.21.15.6:0/1303569120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25600ce140 con 0x7f25600b1630 2023-11-28T13:00:34.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.604+0000 7f255d7fa700 1 -- 172.21.15.6:0/1303569120 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f255801c440 con 0x7f25600b1630 2023-11-28T13:00:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.605+0000 7f255d7fa700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f25400682f0 0x7f254006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.605+0000 7f255ffff700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f25400682f0 0x7f254006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.605+0000 7f255d7fa700 1 -- 172.21.15.6:0/1303569120 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2558087770 con 0x7f25600b1630 2023-11-28T13:00:34.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.605+0000 7f255ffff700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f25400682f0 0x7f254006a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f25600b5a70 tx=0x7f2548008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:34.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.609+0000 7f255d7fa700 1 -- 172.21.15.6:0/1303569120 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2558053c00 con 0x7f25600b1630 2023-11-28T13:00:34.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.802+0000 7f2566ba6700 1 -- 172.21.15.6:0/1303569120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f25600739f0 con 0x7f25600b1630 2023-11-28T13:00:34.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.803+0000 7f255d7fa700 1 -- 172.21.15.6:0/1303569120 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2558020020 con 0x7f25600b1630 2023-11-28T13:00:34.804 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:00:34.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 -- 172.21.15.6:0/1303569120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f25400682f0 msgr2=0x7f254006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f25400682f0 0x7f254006a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f25600b5a70 tx=0x7f2548008040 comp rx=0 tx=0).stop 2023-11-28T13:00:34.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 -- 172.21.15.6:0/1303569120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 msgr2=0x7f25600c62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 0x7f25600c62d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f255800b370 tx=0x7f2558004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 -- 172.21.15.6:0/1303569120 shutdown_connections 2023-11-28T13:00:34.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f25400682f0 0x7f254006a7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f25600cb3e0 0x7f25600b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f25600b1630 0x7f25600c62d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 --2- 172.21.15.6:0/1303569120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f25600b1a00 0x7f256015f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 -- 172.21.15.6:0/1303569120 >> 172.21.15.6:0/1303569120 conn(0x7f25600cc630 msgr2=0x7f256015af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:34.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 -- 172.21.15.6:0/1303569120 shutdown_connections 2023-11-28T13:00:34.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.805+0000 7f2566ba6700 1 -- 172.21.15.6:0/1303569120 wait complete. 2023-11-28T13:00:34.818 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:00:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.945+0000 7fc5c9ec8700 1 Processor -- start 2023-11-28T13:00:34.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.945+0000 7fc5c9ec8700 1 -- start start 2023-11-28T13:00:34.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c9ec8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc5c40cf5f0 0x7fc5c40caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c9ec8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc5c40cb3e0 0x7fc5c406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c9ec8700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c40ce550 con 0x7fc5c40b1630 2023-11-28T13:00:34.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c9ec8700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c40ce830 con 0x7fc5c40cf5f0 2023-11-28T13:00:34.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c9ec8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c40ceb10 con 0x7fc5c40cb3e0 2023-11-28T13:00:34.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c2ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc5c40cf5f0 0x7fc5c40caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c37fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fc5c40b1630 0x7fc5c40b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45240/0 (socket says 172.21.15.6:45240) 2023-11-28T13:00:34.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c37fe700 1 -- 172.21.15.6:0/2899178627 learned_addr learned my addr 172.21.15.6:0/2899178627 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:34.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c3fff700 1 --2- 172.21.15.6:0/2899178627 >> v2:172.21.15.61:3300/0 conn(0x7fc5c40cb3e0 0x7fc5c406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c2ffd700 1 -- 172.21.15.6:0/2899178627 >> v2:172.21.15.61:3300/0 conn(0x7fc5c40cb3e0 msgr2=0x7fc5c406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c2ffd700 1 --2- 172.21.15.6:0/2899178627 >> v2:172.21.15.61:3300/0 conn(0x7fc5c40cb3e0 0x7fc5c406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c2ffd700 1 -- 172.21.15.6:0/2899178627 >> v1:172.21.15.6:6789/0 conn(0x7fc5c40b1630 legacy=0x7fc5c40b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c27fc700 1 -- 172.21.15.6:0/2899178627 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30785478 0 0) 0x7fc5c40ce550 con 0x7fc5c40b1630 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.946+0000 7fc5c2ffd700 1 -- 172.21.15.6:0/2899178627 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c40cedf0 con 0x7fc5c40cf5f0 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.947+0000 7fc5c2ffd700 1 --2- 172.21.15.6:0/2899178627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40cf5f0 0x7fc5c40caea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009840 tx=0x7fc5b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a308025e399e018 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.947+0000 7fc5c27fc700 1 -- 172.21.15.6:0/2899178627 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8011830 con 0x7fc5c40cf5f0 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.947+0000 7fc5c27fc700 1 -- 172.21.15.6:0/2899178627 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b8011990 con 0x7fc5c40cf5f0 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.947+0000 7fc5c27fc700 1 -- 172.21.15.6:0/2899178627 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8011d20 con 0x7fc5c40cf5f0 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.947+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/2899178627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40cf5f0 msgr2=0x7fc5c40caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.947+0000 7fc5c9ec8700 1 --2- 172.21.15.6:0/2899178627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40cf5f0 0x7fc5c40caea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009840 tx=0x7fc5b8009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:34.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.947+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/2899178627 shutdown_connections 2023-11-28T13:00:34.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.947+0000 7fc5c9ec8700 1 --2- 172.21.15.6:0/2899178627 >> v2:172.21.15.61:3300/0 conn(0x7fc5c40cb3e0 0x7fc5c406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.947+0000 7fc5c9ec8700 1 --2- 172.21.15.6:0/2899178627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40cf5f0 0x7fc5c40caea0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.947+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/2899178627 >> 172.21.15.6:0/2899178627 conn(0x7fc5c40cc630 msgr2=0x7fc5c40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:34.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.948+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/2899178627 shutdown_connections 2023-11-28T13:00:34.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.948+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/2899178627 wait complete. 2023-11-28T13:00:34.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.949+0000 7fc5c9ec8700 1 Processor -- start 2023-11-28T13:00:34.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.949+0000 7fc5c9ec8700 1 -- start start 2023-11-28T13:00:34.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c9ec8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40b1630 0x7fc5c40c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c9ec8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5c40cb3e0 0x7fc5c40c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c9ec8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5c40cf5f0 0x7fc5c40b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c9ec8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c40c8860 con 0x7fc5c40b1630 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c9ec8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c41f2490 con 0x7fc5c40cb3e0 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c9ec8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c40c9190 con 0x7fc5c40cf5f0 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40b1630 0x7fc5c40c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40b1630 0x7fc5c40c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57926/0 (socket says 172.21.15.6:57926) 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c37fe700 1 -- 172.21.15.6:0/4293326279 learned_addr learned my addr 172.21.15.6:0/4293326279 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c3fff700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5c40cf5f0 0x7fc5c40b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c2ffd700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5c40cb3e0 0x7fc5c40c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c37fe700 1 -- 172.21.15.6:0/4293326279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5c40cf5f0 msgr2=0x7fc5c40b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c37fe700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5c40cf5f0 0x7fc5c40b4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c37fe700 1 -- 172.21.15.6:0/4293326279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5c40cb3e0 msgr2=0x7fc5c40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c37fe700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5c40cb3e0 0x7fc5c40c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c37fe700 1 -- 172.21.15.6:0/4293326279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b80094a0 con 0x7fc5c40b1630 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c37fe700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40b1630 0x7fc5c40c7690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40011e0 tx=0x7fc5b400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5abfff700 1 -- 172.21.15.6:0/4293326279 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4004c80 con 0x7fc5c40b1630 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5abfff700 1 -- 172.21.15.6:0/4293326279 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b4004e00 con 0x7fc5c40b1630 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.950+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/4293326279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c40ce140 con 0x7fc5c40b1630 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.951+0000 7fc5abfff700 1 -- 172.21.15.6:0/4293326279 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4007310 con 0x7fc5c40b1630 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.951+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/4293326279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c40cf0d0 con 0x7fc5c40b1630 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.951+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/4293326279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c40ce2d0 con 0x7fc5c40b1630 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.952+0000 7fc5abfff700 1 -- 172.21.15.6:0/4293326279 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc5b401a020 con 0x7fc5c40b1630 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.952+0000 7fc5abfff700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5b00682f0 0x7fc5b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.952+0000 7fc5abfff700 1 -- 172.21.15.6:0/4293326279 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc5b4085aa0 con 0x7fc5c40b1630 2023-11-28T13:00:34.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.952+0000 7fc5c2ffd700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5b00682f0 0x7fc5b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:34.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.953+0000 7fc5c2ffd700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5b00682f0 0x7fc5b006a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80090b0 tx=0x7fc5b8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:34.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:34.957+0000 7fc5abfff700 1 -- 172.21.15.6:0/4293326279 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc5b4051f90 con 0x7fc5c40b1630 2023-11-28T13:00:35.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.149+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/4293326279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc5c41f2600 con 0x7fc5c40b1630 2023-11-28T13:00:35.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.150+0000 7fc5abfff700 1 -- 172.21.15.6:0/4293326279 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc5b40555b0 con 0x7fc5c40b1630 2023-11-28T13:00:35.151 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:00:35.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/4293326279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5b00682f0 msgr2=0x7fc5b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5b00682f0 0x7fc5b006a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80090b0 tx=0x7fc5b8008030 comp rx=0 tx=0).stop 2023-11-28T13:00:35.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/4293326279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40b1630 msgr2=0x7fc5c40c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40b1630 0x7fc5c40c7690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40011e0 tx=0x7fc5b400ca30 comp rx=0 tx=0).stop 2023-11-28T13:00:35.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/4293326279 shutdown_connections 2023-11-28T13:00:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5b00682f0 0x7fc5b006a7a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5c40cf5f0 0x7fc5c40b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5c40b1630 0x7fc5c40c7690 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 --2- 172.21.15.6:0/4293326279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5c40cb3e0 0x7fc5c40c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/4293326279 >> 172.21.15.6:0/4293326279 conn(0x7fc5c40cc630 msgr2=0x7fc5c406b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/4293326279 shutdown_connections 2023-11-28T13:00:35.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.152+0000 7fc5c9ec8700 1 -- 172.21.15.6:0/4293326279 wait complete. 2023-11-28T13:00:35.165 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:00:35.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.290+0000 7f8637968700 1 Processor -- start 2023-11-28T13:00:35.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.290+0000 7f8637968700 1 -- start start 2023-11-28T13:00:35.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8637968700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86300cefa0 0x7f863015db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8637968700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863015e0b0 con 0x7f863006e400 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8637968700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863015d120 con 0x7f8630159590 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8637968700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863015d6e0 con 0x7f86300cefa0 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8634f03700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f863006e400 0x7f86300ce890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45256/0 (socket says 172.21.15.6:45256) 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8634f03700 1 -- 172.21.15.6:0/1485442053 learned_addr learned my addr 172.21.15.6:0/1485442053 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8635f05700 1 --2- 172.21.15.6:0/1485442053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86300cefa0 0x7f863015db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8627fff700 1 -- 172.21.15.6:0/1485442053 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590167802 0 0) 0x7f8620001110 con 0x7f863006e400 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8627fff700 1 -- 172.21.15.6:0/1485442053 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f861c0035b0 con 0x7f863006e400 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8627fff700 1 -- 172.21.15.6:0/1485442053 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193068156 0 0) 0x7f863015d120 con 0x7f8630159590 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8627fff700 1 -- 172.21.15.6:0/1485442053 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8620001110 con 0x7f8630159590 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8627fff700 1 -- 172.21.15.6:0/1485442053 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749077647 0 0) 0x7f861c0035b0 con 0x7f863006e400 2023-11-28T13:00:35.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8635f05700 1 -- 172.21.15.6:0/1485442053 >> v1:172.21.15.61:6789/0 conn(0x7f8630159590 legacy=0x7f86300ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8635f05700 1 -- 172.21.15.6:0/1485442053 >> v1:172.21.15.6:6789/0 conn(0x7f863006e400 legacy=0x7f86300ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.291+0000 7f8635f05700 1 -- 172.21.15.6:0/1485442053 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86300aef60 con 0x7f86300cefa0 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8635f05700 1 --2- 172.21.15.6:0/1485442053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86300cefa0 0x7f863015db70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8628009dc0 tx=0x7f862800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff8450e091a65c46 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8627fff700 1 -- 172.21.15.6:0/1485442053 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800e070 con 0x7f86300cefa0 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8627fff700 1 -- 172.21.15.6:0/1485442053 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8628003a20 con 0x7f86300cefa0 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8627fff700 1 -- 172.21.15.6:0/1485442053 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8620001800 con 0x7f86300cefa0 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8637968700 1 -- 172.21.15.6:0/1485442053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86300cefa0 msgr2=0x7f863015db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8637968700 1 --2- 172.21.15.6:0/1485442053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86300cefa0 0x7f863015db70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8628009dc0 tx=0x7f862800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8637968700 1 -- 172.21.15.6:0/1485442053 shutdown_connections 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8637968700 1 --2- 172.21.15.6:0/1485442053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f86300cefa0 0x7f863015db70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8637968700 1 -- 172.21.15.6:0/1485442053 >> 172.21.15.6:0/1485442053 conn(0x7f86300b1650 msgr2=0x7f86300b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8637968700 1 -- 172.21.15.6:0/1485442053 shutdown_connections 2023-11-28T13:00:35.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.292+0000 7f8637968700 1 -- 172.21.15.6:0/1485442053 wait complete. 2023-11-28T13:00:35.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.293+0000 7f8637968700 1 Processor -- start 2023-11-28T13:00:35.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.294+0000 7f8637968700 1 -- start start 2023-11-28T13:00:35.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.294+0000 7f8637968700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f863006e400 0x7f8630154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.294+0000 7f8637968700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86300cefa0 0x7f86300b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.294+0000 7f8637968700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8630159590 0x7f86300b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.294+0000 7f8637968700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86301f0610 con 0x7f8630159590 2023-11-28T13:00:35.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.294+0000 7f8637968700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86301f08f0 con 0x7f86300cefa0 2023-11-28T13:00:35.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.294+0000 7f8637968700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86301f0bd0 con 0x7f863006e400 2023-11-28T13:00:35.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.294+0000 7f8635f05700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8630159590 0x7f86300b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.294+0000 7f8635f05700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8630159590 0x7f86300b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57944/0 (socket says 172.21.15.6:57944) 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.294+0000 7f8635f05700 1 -- 172.21.15.6:0/3179654740 learned_addr learned my addr 172.21.15.6:0/3179654740 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8635704700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f863006e400 0x7f8630154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8634f03700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86300cefa0 0x7f86300b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8635f05700 1 -- 172.21.15.6:0/3179654740 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f863006e400 msgr2=0x7f8630154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8635f05700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f863006e400 0x7f8630154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8635f05700 1 -- 172.21.15.6:0/3179654740 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86300cefa0 msgr2=0x7f86300b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8635f05700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86300cefa0 0x7f86300b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8635f05700 1 -- 172.21.15.6:0/3179654740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628009a70 con 0x7f8630159590 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8635f05700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8630159590 0x7f86300b4750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8628009d90 tx=0x7f8628004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8625ffb700 1 -- 172.21.15.6:0/3179654740 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86280046f0 con 0x7f8630159590 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8625ffb700 1 -- 172.21.15.6:0/3179654740 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8628004850 con 0x7f8630159590 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8637968700 1 -- 172.21.15.6:0/3179654740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863015d370 con 0x7f8630159590 2023-11-28T13:00:35.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8625ffb700 1 -- 172.21.15.6:0/3179654740 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862801d670 con 0x7f8630159590 2023-11-28T13:00:35.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.295+0000 7f8637968700 1 -- 172.21.15.6:0/3179654740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86300af240 con 0x7f8630159590 2023-11-28T13:00:35.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.296+0000 7f8637968700 1 -- 172.21.15.6:0/3179654740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8630159960 con 0x7f8630159590 2023-11-28T13:00:35.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.297+0000 7f8625ffb700 1 -- 172.21.15.6:0/3179654740 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8628023030 con 0x7f8630159590 2023-11-28T13:00:35.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.297+0000 7f8625ffb700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86140682f0 0x7f861406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.297+0000 7f8625ffb700 1 -- 172.21.15.6:0/3179654740 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8628087b90 con 0x7f8630159590 2023-11-28T13:00:35.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.297+0000 7f8635704700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86140682f0 0x7f861406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.298+0000 7f8635704700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86140682f0 0x7f861406a7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f862c001220 tx=0x7f862c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:35.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.302+0000 7f8625ffb700 1 -- 172.21.15.6:0/3179654740 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f86280540a0 con 0x7f8630159590 2023-11-28T13:00:35.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.513+0000 7f8637968700 1 -- 172.21.15.6:0/3179654740 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f86300675a0 con 0x7f8630159590 2023-11-28T13:00:35.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.514+0000 7f8625ffb700 1 -- 172.21.15.6:0/3179654740 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f86280576c0 con 0x7f8630159590 2023-11-28T13:00:35.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 -- 172.21.15.6:0/3179654740 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86140682f0 msgr2=0x7f861406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86140682f0 0x7f861406a7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f862c001220 tx=0x7f862c009040 comp rx=0 tx=0).stop 2023-11-28T13:00:35.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 -- 172.21.15.6:0/3179654740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8630159590 msgr2=0x7f86300b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8630159590 0x7f86300b4750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8628009d90 tx=0x7f8628004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 -- 172.21.15.6:0/3179654740 shutdown_connections 2023-11-28T13:00:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8630159590 0x7f86300b4750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f86140682f0 0x7f861406a7a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f863006e400 0x7f8630154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 --2- 172.21.15.6:0/3179654740 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86300cefa0 0x7f86300b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 -- 172.21.15.6:0/3179654740 >> 172.21.15.6:0/3179654740 conn(0x7f86300b1650 msgr2=0x7f86300ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 -- 172.21.15.6:0/3179654740 shutdown_connections 2023-11-28T13:00:35.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.516+0000 7f8637968700 1 -- 172.21.15.6:0/3179654740 wait complete. 2023-11-28T13:00:35.519 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:00:35.529 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2023-11-28T13:00:35.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.530 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-28T13:00:35.565 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi006.stderr:ceph-fuse[111395]: fuse finished with error 0 and tester_r 0 2023-11-28T13:00:35.566 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:00:35.638 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:00:35.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:35.638 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-28T13:00:35.662 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:35.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.825+0000 7f542330a700 1 Processor -- start 2023-11-28T13:00:35.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.825+0000 7f542330a700 1 -- start start 2023-11-28T13:00:35.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f542330a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f541c0cb470 0x7f541c0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f542330a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f541c06bb80 0x7f541c14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f542330a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c06c140 con 0x7f541c0cb0a0 2023-11-28T13:00:35.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f542330a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541c0caad0 con 0x7f541c06bb80 2023-11-28T13:00:35.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f542330a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0c9ab0 con 0x7f541c0cb470 2023-11-28T13:00:35.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f54210a6700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f541c0cb0a0 0x7f541c15a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45268/0 (socket says 172.21.15.6:45268) 2023-11-28T13:00:35.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f54210a6700 1 -- 172.21.15.6:0/2016311246 learned_addr learned my addr 172.21.15.6:0/2016311246 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:35.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f54218a7700 1 --2- 172.21.15.6:0/2016311246 >> v2:172.21.15.61:3300/0 conn(0x7f541c06bb80 0x7f541c14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f54208a5700 1 --2- 172.21.15.6:0/2016311246 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f541c0cb470 0x7f541c0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f5413fff700 1 -- 172.21.15.6:0/2016311246 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765877788 0 0) 0x7f541c06c140 con 0x7f541c0cb0a0 2023-11-28T13:00:35.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.826+0000 7f5413fff700 1 -- 172.21.15.6:0/2016311246 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54080035b0 con 0x7f541c0cb0a0 2023-11-28T13:00:35.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f54218a7700 1 -- 172.21.15.6:0/2016311246 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f541c0cb470 msgr2=0x7f541c0ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:35.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f54218a7700 1 --2- 172.21.15.6:0/2016311246 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f541c0cb470 0x7f541c0ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f54218a7700 1 -- 172.21.15.6:0/2016311246 >> v1:172.21.15.6:6789/0 conn(0x7f541c0cb0a0 legacy=0x7f541c15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f5413fff700 1 -- 172.21.15.6:0/2016311246 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1401489239 0 0) 0x7f54080035b0 con 0x7f541c0cb0a0 2023-11-28T13:00:35.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f54218a7700 1 -- 172.21.15.6:0/2016311246 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c0c9d90 con 0x7f541c06bb80 2023-11-28T13:00:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f54218a7700 1 --2- 172.21.15.6:0/2016311246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f541c06bb80 0x7f541c14fea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5414009d90 tx=0x7f541400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcef08556e5ef834 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f5413fff700 1 -- 172.21.15.6:0/2016311246 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400e070 con 0x7f541c06bb80 2023-11-28T13:00:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f5413fff700 1 -- 172.21.15.6:0/2016311246 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5414003a20 con 0x7f541c06bb80 2023-11-28T13:00:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f5413fff700 1 -- 172.21.15.6:0/2016311246 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c001670 con 0x7f541c06bb80 2023-11-28T13:00:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f542330a700 1 -- 172.21.15.6:0/2016311246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f541c06bb80 msgr2=0x7f541c14fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f542330a700 1 --2- 172.21.15.6:0/2016311246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f541c06bb80 0x7f541c14fea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5414009d90 tx=0x7f541400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.827+0000 7f542330a700 1 -- 172.21.15.6:0/2016311246 shutdown_connections 2023-11-28T13:00:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.828+0000 7f542330a700 1 --2- 172.21.15.6:0/2016311246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f541c06bb80 0x7f541c14fea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.828+0000 7f542330a700 1 --2- 172.21.15.6:0/2016311246 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f541c0cb470 0x7f541c0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.828+0000 7f542330a700 1 -- 172.21.15.6:0/2016311246 >> 172.21.15.6:0/2016311246 conn(0x7f541c0b0dd0 msgr2=0x7f541c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.828+0000 7f542330a700 1 -- 172.21.15.6:0/2016311246 shutdown_connections 2023-11-28T13:00:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.828+0000 7f542330a700 1 -- 172.21.15.6:0/2016311246 wait complete. 2023-11-28T13:00:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.829+0000 7f542330a700 1 Processor -- start 2023-11-28T13:00:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.829+0000 7f542330a700 1 -- start start 2023-11-28T13:00:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.829+0000 7f542330a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f541c06bb80 0x7f541c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f542330a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f541c0cb0a0 0x7f541c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f542330a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f541c0cb470 0x7f541c0ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f542330a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c149280 con 0x7f541c06bb80 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f542330a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0b4160 con 0x7f541c0cb470 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f542330a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c14a6e0 con 0x7f541c0cb0a0 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54210a6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f541c06bb80 0x7f541c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54210a6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f541c06bb80 0x7f541c0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57954/0 (socket says 172.21.15.6:57954) 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54210a6700 1 -- 172.21.15.6:0/2508450436 learned_addr learned my addr 172.21.15.6:0/2508450436 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54208a5700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f541c0cb0a0 0x7f541c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54210a6700 1 -- 172.21.15.6:0/2508450436 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f541c0cb0a0 msgr2=0x7f541c0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54210a6700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f541c0cb0a0 0x7f541c0aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54218a7700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f541c0cb470 0x7f541c0ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54210a6700 1 -- 172.21.15.6:0/2508450436 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f541c0cb470 msgr2=0x7f541c0ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54210a6700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f541c0cb470 0x7f541c0ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54210a6700 1 -- 172.21.15.6:0/2508450436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414009a70 con 0x7f541c06bb80 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f54210a6700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f541c06bb80 0x7f541c0b4bb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f540c0011e0 tx=0x7f540c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f5411ffb700 1 -- 172.21.15.6:0/2508450436 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c007560 con 0x7f541c06bb80 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f5411ffb700 1 -- 172.21.15.6:0/2508450436 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c0076c0 con 0x7f541c06bb80 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f542330a700 1 -- 172.21.15.6:0/2508450436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c1503e0 con 0x7f541c06bb80 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.830+0000 7f5411ffb700 1 -- 172.21.15.6:0/2508450436 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c004a90 con 0x7f541c06bb80 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.831+0000 7f542330a700 1 -- 172.21.15.6:0/2508450436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c0ca070 con 0x7f541c06bb80 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.831+0000 7f542330a700 1 -- 172.21.15.6:0/2508450436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c150570 con 0x7f541c06bb80 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.832+0000 7f5411ffb700 1 -- 172.21.15.6:0/2508450436 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f540c010070 con 0x7f541c06bb80 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.832+0000 7f5411ffb700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f54000682f0 0x7f540006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.832+0000 7f5411ffb700 1 -- 172.21.15.6:0/2508450436 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f540c0850e0 con 0x7f541c06bb80 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.832+0000 7f54208a5700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f54000682f0 0x7f540006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.833+0000 7f54208a5700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f54000682f0 0x7f540006a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f541c1f3310 tx=0x7f5418008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:35.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:35.836+0000 7f5411ffb700 1 -- 172.21.15.6:0/2508450436 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f540c0515d0 con 0x7f541c06bb80 2023-11-28T13:00:36.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.034+0000 7f542330a700 1 -- 172.21.15.6:0/2508450436 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f541c0675a0 con 0x7f541c06bb80 2023-11-28T13:00:36.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.035+0000 7f5411ffb700 1 -- 172.21.15.6:0/2508450436 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f540c018090 con 0x7f541c06bb80 2023-11-28T13:00:36.037 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:36.037 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:36.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 -- 172.21.15.6:0/2508450436 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f54000682f0 msgr2=0x7f540006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f54000682f0 0x7f540006a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f541c1f3310 tx=0x7f5418008040 comp rx=0 tx=0).stop 2023-11-28T13:00:36.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 -- 172.21.15.6:0/2508450436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f541c06bb80 msgr2=0x7f541c0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f541c06bb80 0x7f541c0b4bb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f540c0011e0 tx=0x7f540c00cb00 comp rx=0 tx=0).stop 2023-11-28T13:00:36.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 -- 172.21.15.6:0/2508450436 shutdown_connections 2023-11-28T13:00:36.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f541c0cb470 0x7f541c0ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f541c06bb80 0x7f541c0b4bb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f54000682f0 0x7f540006a7a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 --2- 172.21.15.6:0/2508450436 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f541c0cb0a0 0x7f541c0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 -- 172.21.15.6:0/2508450436 >> 172.21.15.6:0/2508450436 conn(0x7f541c0b0dd0 msgr2=0x7f541c06a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:36.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 -- 172.21.15.6:0/2508450436 shutdown_connections 2023-11-28T13:00:36.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.038+0000 7f542330a700 1 -- 172.21.15.6:0/2508450436 wait complete. 2023-11-28T13:00:36.052 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:36.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.174+0000 7fce91ede700 1 Processor -- start 2023-11-28T13:00:36.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.174+0000 7fce91ede700 1 -- start start 2023-11-28T13:00:36.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.175+0000 7fce91ede700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fce8c06e400 0x7fce8c15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.175+0000 7fce91ede700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8c15e0b0 con 0x7fce8c159590 2023-11-28T13:00:36.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.175+0000 7fce91ede700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8c15d120 con 0x7fce8c0ce890 2023-11-28T13:00:36.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.175+0000 7fce91ede700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c15d6e0 con 0x7fce8c06e400 2023-11-28T13:00:36.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.175+0000 7fce8b7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fce8c159590 0x7fce8c0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45282/0 (socket says 172.21.15.6:45282) 2023-11-28T13:00:36.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.175+0000 7fce8b7fe700 1 -- 172.21.15.6:0/2843669531 learned_addr learned my addr 172.21.15.6:0/2843669531 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:36.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8affd700 1 --2- 172.21.15.6:0/2843669531 >> v2:172.21.15.61:3300/0 conn(0x7fce8c06e400 0x7fce8c15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729488263 0 0) 0x7fce7c001110 con 0x7fce8c159590 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce780035b0 con 0x7fce8c159590 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2430358803 0 0) 0x7fce780035b0 con 0x7fce8c159590 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 >> v2:172.21.15.61:3300/0 conn(0x7fce8c06e400 msgr2=0x7fce8c15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 --2- 172.21.15.6:0/2843669531 >> v2:172.21.15.61:3300/0 conn(0x7fce8c06e400 0x7fce8c15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 >> v1:172.21.15.61:6789/0 conn(0x7fce8c0ce890 legacy=0x7fce8c0caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c0aef60 con 0x7fce8c159590 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fce7c001800 con 0x7fce8c159590 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 >> v1:172.21.15.6:6789/0 conn(0x7fce8c159590 legacy=0x7fce8c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce78003a00 0x7fce78003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce78004850 0x7fce78004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce78005200 0x7fce7800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c15d6e0 con 0x7fce78003a00 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7800c080 con 0x7fce78004850 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7800c240 con 0x7fce78005200 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8b7fe700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce78003a00 0x7fce78003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8affd700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce78004850 0x7fce78004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.176+0000 7fce8bfff700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce78005200 0x7fce7800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce8b7fe700 1 -- 172.21.15.6:0/2843669531 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce78005200 msgr2=0x7fce7800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce8b7fe700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce78005200 0x7fce7800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce8b7fe700 1 -- 172.21.15.6:0/2843669531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce78004850 msgr2=0x7fce78004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce8b7fe700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce78004850 0x7fce78004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce8b7fe700 1 -- 172.21.15.6:0/2843669531 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780035b0 con 0x7fce78003a00 2023-11-28T13:00:36.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce8bfff700 1 -- 172.21.15.6:0/2843669531 reap_dead start 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce8b7fe700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce78003a00 0x7fce78003e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00c210 tx=0x7fce78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8357a611d354ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c007d90 con 0x7fce78003a00 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c00cdb0 con 0x7fce78003a00 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce8a7fc700 1 -- 172.21.15.6:0/2843669531 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c005b80 con 0x7fce78003a00 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce91ede700 1 -- 172.21.15.6:0/2843669531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce78003a00 msgr2=0x7fce78003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce91ede700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce78003a00 0x7fce78003e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00c210 tx=0x7fce78005880 comp rx=0 tx=0).stop 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce91ede700 1 -- 172.21.15.6:0/2843669531 shutdown_connections 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce91ede700 1 --2- 172.21.15.6:0/2843669531 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce78003a00 0x7fce78003e10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce91ede700 1 -- 172.21.15.6:0/2843669531 >> 172.21.15.6:0/2843669531 conn(0x7fce8c0b1650 msgr2=0x7fce8c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce91ede700 1 -- 172.21.15.6:0/2843669531 shutdown_connections 2023-11-28T13:00:36.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.177+0000 7fce91ede700 1 -- 172.21.15.6:0/2843669531 wait complete. 2023-11-28T13:00:36.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.179+0000 7fce91ede700 1 Processor -- start 2023-11-28T13:00:36.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.179+0000 7fce91ede700 1 -- start start 2023-11-28T13:00:36.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.180+0000 7fce91ede700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce78003a00 0x7fce8c1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.180+0000 7fce91ede700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce8c0c61b0 0x7fce8c0c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.180+0000 7fce91ede700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce8c0c8340 0x7fce8c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.180+0000 7fce91ede700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c0b4f10 con 0x7fce8c0c61b0 2023-11-28T13:00:36.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.180+0000 7fce91ede700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c0c98f0 con 0x7fce8c0c8340 2023-11-28T13:00:36.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.180+0000 7fce91ede700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c0c9610 con 0x7fce78003a00 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.180+0000 7fce8affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce8c0c61b0 0x7fce8c0c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.180+0000 7fce8affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce8c0c61b0 0x7fce8c0c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57978/0 (socket says 172.21.15.6:57978) 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.180+0000 7fce8affd700 1 -- 172.21.15.6:0/1452221614 learned_addr learned my addr 172.21.15.6:0/1452221614 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce8b7fe700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce78003a00 0x7fce8c1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce8bfff700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce8c0c8340 0x7fce8c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce8affd700 1 -- 172.21.15.6:0/1452221614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce78003a00 msgr2=0x7fce8c1543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce8affd700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce78003a00 0x7fce8c1543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce8affd700 1 -- 172.21.15.6:0/1452221614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce8c0c8340 msgr2=0x7fce8c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce8affd700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce8c0c8340 0x7fce8c0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce8affd700 1 -- 172.21.15.6:0/1452221614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c15d370 con 0x7fce8c0c61b0 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce8affd700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce8c0c61b0 0x7fce8c0c8890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fce74011a00 tx=0x7fce74011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce83fff700 1 -- 172.21.15.6:0/1452221614 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74004240 con 0x7fce8c0c61b0 2023-11-28T13:00:36.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce83fff700 1 -- 172.21.15.6:0/1452221614 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce740043a0 con 0x7fce8c0c61b0 2023-11-28T13:00:36.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce91ede700 1 -- 172.21.15.6:0/1452221614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c0af240 con 0x7fce8c0c61b0 2023-11-28T13:00:36.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce83fff700 1 -- 172.21.15.6:0/1452221614 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce740036a0 con 0x7fce8c0c61b0 2023-11-28T13:00:36.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.181+0000 7fce91ede700 1 -- 172.21.15.6:0/1452221614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c159960 con 0x7fce8c0c61b0 2023-11-28T13:00:36.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.182+0000 7fce91ede700 1 -- 172.21.15.6:0/1452221614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c0675a0 con 0x7fce8c0c61b0 2023-11-28T13:00:36.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.182+0000 7fce83fff700 1 -- 172.21.15.6:0/1452221614 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fce74025070 con 0x7fce8c0c61b0 2023-11-28T13:00:36.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.183+0000 7fce83fff700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce6c0682f0 0x7fce6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.183+0000 7fce83fff700 1 -- 172.21.15.6:0/1452221614 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fce740861f0 con 0x7fce8c0c61b0 2023-11-28T13:00:36.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.183+0000 7fce8b7fe700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce6c0682f0 0x7fce6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.183+0000 7fce8b7fe700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce6c0682f0 0x7fce6c06a7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fce7c000c00 tx=0x7fce7c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:36.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.187+0000 7fce83fff700 1 -- 172.21.15.6:0/1452221614 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fce74052760 con 0x7fce8c0c61b0 2023-11-28T13:00:36.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.383+0000 7fce91ede700 1 -- 172.21.15.6:0/1452221614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce8c06c1b0 con 0x7fce8c0c61b0 2023-11-28T13:00:36.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.385+0000 7fce83fff700 1 -- 172.21.15.6:0/1452221614 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7fce74019090 con 0x7fce8c0c61b0 2023-11-28T13:00:36.386 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:36.386 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:36.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 -- 172.21.15.6:0/1452221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce6c0682f0 msgr2=0x7fce6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce6c0682f0 0x7fce6c06a7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fce7c000c00 tx=0x7fce7c007000 comp rx=0 tx=0).stop 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 -- 172.21.15.6:0/1452221614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce8c0c61b0 msgr2=0x7fce8c0c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce8c0c61b0 0x7fce8c0c8890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fce74011a00 tx=0x7fce74011dc0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 -- 172.21.15.6:0/1452221614 shutdown_connections 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce8c0c8340 0x7fce8c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce6c0682f0 0x7fce6c06a7a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce78003a00 0x7fce8c1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 --2- 172.21.15.6:0/1452221614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce8c0c61b0 0x7fce8c0c8890 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 -- 172.21.15.6:0/1452221614 >> 172.21.15.6:0/1452221614 conn(0x7fce8c0b1650 msgr2=0x7fce8c0b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 -- 172.21.15.6:0/1452221614 shutdown_connections 2023-11-28T13:00:36.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.387+0000 7fce91ede700 1 -- 172.21.15.6:0/1452221614 wait complete. 2023-11-28T13:00:36.401 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:00:36.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.527+0000 7f12a023c700 1 Processor -- start 2023-11-28T13:00:36.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.527+0000 7f12a023c700 1 -- start start 2023-11-28T13:00:36.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f12a023c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1298159500 0x7f12980cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f12a023c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f129815dbf0 0x7f12980ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f12a023c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129815e040 con 0x7f12980b1630 2023-11-28T13:00:36.536 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f12a023c700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12980cede0 con 0x7f129815dbf0 2023-11-28T13:00:36.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f12a023c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12980ceff0 con 0x7f1298159500 2023-11-28T13:00:36.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f129d7d7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f129815dbf0 0x7f12980ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f129d7d7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f129815dbf0 0x7f12980ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57984/0 (socket says 172.21.15.6:57984) 2023-11-28T13:00:36.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f129d7d7700 1 -- 172.21.15.6:0/3591021417 learned_addr learned my addr 172.21.15.6:0/3591021417 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:36.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f129dfd8700 1 --2- 172.21.15.6:0/3591021417 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1298159500 0x7f12980cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f129d7d7700 1 -- 172.21.15.6:0/3591021417 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1298159500 msgr2=0x7f12980cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f129d7d7700 1 --2- 172.21.15.6:0/3591021417 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1298159500 0x7f12980cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f129d7d7700 1 -- 172.21.15.6:0/3591021417 >> v1:172.21.15.61:6789/0 conn(0x7f12980b1630 legacy=0x7f129806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f129d7d7700 1 -- 172.21.15.6:0/3591021417 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12980cf2d0 con 0x7f129815dbf0 2023-11-28T13:00:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.528+0000 7f129d7d7700 1 --2- 172.21.15.6:0/3591021417 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 0x7f12980ce8a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f129400bb90 tx=0x7f129400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0ef3cd4d166a6d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f129cfd6700 1 -- 172.21.15.6:0/3591021417 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129400dd10 con 0x7f129815dbf0 2023-11-28T13:00:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f129cfd6700 1 -- 172.21.15.6:0/3591021417 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129400de70 con 0x7f129815dbf0 2023-11-28T13:00:36.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f129cfd6700 1 -- 172.21.15.6:0/3591021417 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294004230 con 0x7f129815dbf0 2023-11-28T13:00:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f12a023c700 1 -- 172.21.15.6:0/3591021417 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 msgr2=0x7f12980ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f12a023c700 1 --2- 172.21.15.6:0/3591021417 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 0x7f12980ce8a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f129400bb90 tx=0x7f129400bea0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f12a023c700 1 -- 172.21.15.6:0/3591021417 shutdown_connections 2023-11-28T13:00:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f12a023c700 1 --2- 172.21.15.6:0/3591021417 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1298159500 0x7f12980cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f12a023c700 1 --2- 172.21.15.6:0/3591021417 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 0x7f12980ce8a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f12a023c700 1 -- 172.21.15.6:0/3591021417 >> 172.21.15.6:0/3591021417 conn(0x7f12980cc630 msgr2=0x7f12980cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:36.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f12a023c700 1 -- 172.21.15.6:0/3591021417 shutdown_connections 2023-11-28T13:00:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.529+0000 7f12a023c700 1 -- 172.21.15.6:0/3591021417 wait complete. 2023-11-28T13:00:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f12a023c700 1 Processor -- start 2023-11-28T13:00:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f12a023c700 1 -- start start 2023-11-28T13:00:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f12a023c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12980b1630 0x7f12980b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f12a023c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1298159500 0x7f12980b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f12a023c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 0x7f129815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f12a023c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12980c32c0 con 0x7f129815dbf0 2023-11-28T13:00:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f12a023c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129815a480 con 0x7f1298159500 2023-11-28T13:00:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f12a023c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129806aa70 con 0x7f12980b1630 2023-11-28T13:00:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f129e7d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 0x7f129815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f129e7d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 0x7f129815ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:57992/0 (socket says 172.21.15.6:57992) 2023-11-28T13:00:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f129dfd8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12980b1630 0x7f12980b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f129e7d9700 1 -- 172.21.15.6:0/1528717633 learned_addr learned my addr 172.21.15.6:0/1528717633 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:36.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.531+0000 7f129d7d7700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1298159500 0x7f12980b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f129e7d9700 1 -- 172.21.15.6:0/1528717633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12980b1630 msgr2=0x7f12980b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f129e7d9700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12980b1630 0x7f12980b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f129e7d9700 1 -- 172.21.15.6:0/1528717633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1298159500 msgr2=0x7f12980b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f129e7d9700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1298159500 0x7f12980b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f129e7d9700 1 -- 172.21.15.6:0/1528717633 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129400b7f0 con 0x7f129815dbf0 2023-11-28T13:00:36.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f129e7d9700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 0x7f129815ab30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f129000b970 tx=0x7f129000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:36.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f128e7fc700 1 -- 172.21.15.6:0/1528717633 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129000d860 con 0x7f129815dbf0 2023-11-28T13:00:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f128e7fc700 1 -- 172.21.15.6:0/1528717633 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129000d9c0 con 0x7f129815dbf0 2023-11-28T13:00:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f128e7fc700 1 -- 172.21.15.6:0/1528717633 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129000dd70 con 0x7f129815dbf0 2023-11-28T13:00:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f12a023c700 1 -- 172.21.15.6:0/1528717633 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129815d3f0 con 0x7f129815dbf0 2023-11-28T13:00:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.532+0000 7f12a023c700 1 -- 172.21.15.6:0/1528717633 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129815d110 con 0x7f129815dbf0 2023-11-28T13:00:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.533+0000 7f12a023c700 1 -- 172.21.15.6:0/1528717633 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129815d6d0 con 0x7f129815dbf0 2023-11-28T13:00:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.534+0000 7f128e7fc700 1 -- 172.21.15.6:0/1528717633 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f129000e430 con 0x7f129815dbf0 2023-11-28T13:00:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.534+0000 7f128e7fc700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f127c0682f0 0x7f127c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.534+0000 7f128e7fc700 1 -- 172.21.15.6:0/1528717633 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f12900861c0 con 0x7f129815dbf0 2023-11-28T13:00:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.534+0000 7f129dfd8700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f127c0682f0 0x7f127c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.535+0000 7f129dfd8700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f127c0682f0 0x7f127c06a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1288006b20 tx=0x7f1288008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.538+0000 7f128e7fc700 1 -- 172.21.15.6:0/1528717633 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f12900526d0 con 0x7f129815dbf0 2023-11-28T13:00:36.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.779+0000 7f12a023c700 1 -- 172.21.15.6:0/1528717633 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f12980739f0 con 0x7f129815dbf0 2023-11-28T13:00:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.945+0000 7f128e7fc700 1 -- 172.21.15.6:0/1528717633 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1290055cf0 con 0x7f129815dbf0 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 -- 172.21.15.6:0/1528717633 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f127c0682f0 msgr2=0x7f127c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f127c0682f0 0x7f127c06a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1288006b20 tx=0x7f1288008040 comp rx=0 tx=0).stop 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 -- 172.21.15.6:0/1528717633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 msgr2=0x7f129815ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 0x7f129815ab30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f129000b970 tx=0x7f129000bc80 comp rx=0 tx=0).stop 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 -- 172.21.15.6:0/1528717633 shutdown_connections 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f129815dbf0 0x7f129815ab30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f127c0682f0 0x7f127c06a7a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f12980b1630 0x7f12980b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 --2- 172.21.15.6:0/1528717633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1298159500 0x7f12980b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 -- 172.21.15.6:0/1528717633 >> 172.21.15.6:0/1528717633 conn(0x7f12980cc630 msgr2=0x7f12980af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 -- 172.21.15.6:0/1528717633 shutdown_connections 2023-11-28T13:00:36.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:36.947+0000 7f12a023c700 1 -- 172.21.15.6:0/1528717633 wait complete. 2023-11-28T13:00:36.961 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-11-28T13:00:36.961 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:00:37.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.089+0000 7f97e61b3700 1 Processor -- start 2023-11-28T13:00:37.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.089+0000 7f97e61b3700 1 -- start start 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.090+0000 7f97e61b3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cf5f0 0x7f97e00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.090+0000 7f97e61b3700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e00ce550 con 0x7f97e00b1630 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.090+0000 7f97e61b3700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e00ce830 con 0x7f97e00cb3e0 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.090+0000 7f97e61b3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00cfa60 con 0x7f97e00cf5f0 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.090+0000 7f97df7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f97e00b1630 0x7f97e00b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45316/0 (socket says 172.21.15.6:45316) 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.090+0000 7f97df7fe700 1 -- 172.21.15.6:0/1502796713 learned_addr learned my addr 172.21.15.6:0/1502796713 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.091+0000 7f97deffd700 1 --2- 172.21.15.6:0/1502796713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cf5f0 0x7f97e00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.091+0000 7f97de7fc700 1 -- 172.21.15.6:0/1502796713 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2387491763 0 0) 0x7f97e00ce550 con 0x7f97e00b1630 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.091+0000 7f97de7fc700 1 -- 172.21.15.6:0/1502796713 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97c80035b0 con 0x7f97e00b1630 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.091+0000 7f97de7fc700 1 -- 172.21.15.6:0/1502796713 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373254089 0 0) 0x7f97e00ce830 con 0x7f97e00cb3e0 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.091+0000 7f97de7fc700 1 -- 172.21.15.6:0/1502796713 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e00ce550 con 0x7f97e00cb3e0 2023-11-28T13:00:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.091+0000 7f97deffd700 1 -- 172.21.15.6:0/1502796713 >> v1:172.21.15.61:6789/0 conn(0x7f97e00cb3e0 legacy=0x7f97e006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.091+0000 7f97deffd700 1 -- 172.21.15.6:0/1502796713 >> v1:172.21.15.6:6789/0 conn(0x7f97e00b1630 legacy=0x7f97e00b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.091+0000 7f97deffd700 1 -- 172.21.15.6:0/1502796713 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e00ceb10 con 0x7f97e00cf5f0 2023-11-28T13:00:37.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.091+0000 7f97deffd700 1 --2- 172.21.15.6:0/1502796713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cf5f0 0x7f97e00caea0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009870 tx=0x7f97d0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9df297b8cbc5f409 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:37.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.092+0000 7f97de7fc700 1 -- 172.21.15.6:0/1502796713 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0011830 con 0x7f97e00cf5f0 2023-11-28T13:00:37.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.092+0000 7f97de7fc700 1 -- 172.21.15.6:0/1502796713 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d0011990 con 0x7f97e00cf5f0 2023-11-28T13:00:37.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.092+0000 7f97de7fc700 1 -- 172.21.15.6:0/1502796713 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0011d00 con 0x7f97e00cf5f0 2023-11-28T13:00:37.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.092+0000 7f97e61b3700 1 -- 172.21.15.6:0/1502796713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cf5f0 msgr2=0x7f97e00caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.092+0000 7f97e61b3700 1 --2- 172.21.15.6:0/1502796713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cf5f0 0x7f97e00caea0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009870 tx=0x7f97d0009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:37.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.092+0000 7f97e61b3700 1 -- 172.21.15.6:0/1502796713 shutdown_connections 2023-11-28T13:00:37.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.092+0000 7f97e61b3700 1 --2- 172.21.15.6:0/1502796713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cf5f0 0x7f97e00caea0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.092+0000 7f97e61b3700 1 -- 172.21.15.6:0/1502796713 >> 172.21.15.6:0/1502796713 conn(0x7f97e00cc630 msgr2=0x7f97e00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:37.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.092+0000 7f97e61b3700 1 -- 172.21.15.6:0/1502796713 shutdown_connections 2023-11-28T13:00:37.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.092+0000 7f97e61b3700 1 -- 172.21.15.6:0/1502796713 wait complete. 2023-11-28T13:00:37.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.093+0000 7f97e61b3700 1 Processor -- start 2023-11-28T13:00:37.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.094+0000 7f97e61b3700 1 -- start start 2023-11-28T13:00:37.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.094+0000 7f97e61b3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e00b1630 0x7f97e00c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:37.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.094+0000 7f97e61b3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cb3e0 0x7f97e00c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:37.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.094+0000 7f97e61b3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e00cf5f0 0x7f97e00c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.094+0000 7f97e61b3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00b3800 con 0x7f97e00b1630 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.094+0000 7f97e61b3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00b31d0 con 0x7f97e00cf5f0 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.094+0000 7f97e61b3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00c5bc0 con 0x7f97e00cb3e0 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.094+0000 7f97df7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e00b1630 0x7f97e00c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97df7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e00b1630 0x7f97e00c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58004/0 (socket says 172.21.15.6:58004) 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97df7fe700 1 -- 172.21.15.6:0/2691196260 learned_addr learned my addr 172.21.15.6:0/2691196260 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97deffd700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cb3e0 0x7f97e00c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97dffff700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e00cf5f0 0x7f97e00c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97df7fe700 1 -- 172.21.15.6:0/2691196260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cb3e0 msgr2=0x7f97e00c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97df7fe700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cb3e0 0x7f97e00c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97df7fe700 1 -- 172.21.15.6:0/2691196260 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e00cf5f0 msgr2=0x7f97e00c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97df7fe700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e00cf5f0 0x7f97e00c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97df7fe700 1 -- 172.21.15.6:0/2691196260 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d00094a0 con 0x7f97e00b1630 2023-11-28T13:00:37.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97deffd700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cb3e0 0x7f97e00c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:37.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97dffff700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e00cf5f0 0x7f97e00c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:37.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97df7fe700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e00b1630 0x7f97e00c5140 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97d400c700 tx=0x7f97d400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:37.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97c7fff700 1 -- 172.21.15.6:0/2691196260 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4005d70 con 0x7f97e00b1630 2023-11-28T13:00:37.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97c7fff700 1 -- 172.21.15.6:0/2691196260 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d40070d0 con 0x7f97e00b1630 2023-11-28T13:00:37.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97e61b3700 1 -- 172.21.15.6:0/2691196260 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e00cf0d0 con 0x7f97e00b1630 2023-11-28T13:00:37.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97c7fff700 1 -- 172.21.15.6:0/2691196260 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4007480 con 0x7f97e00b1630 2023-11-28T13:00:37.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.095+0000 7f97e61b3700 1 -- 172.21.15.6:0/2691196260 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e00cedf0 con 0x7f97e00b1630 2023-11-28T13:00:37.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.096+0000 7f97e61b3700 1 -- 172.21.15.6:0/2691196260 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e00c60e0 con 0x7f97e00b1630 2023-11-28T13:00:37.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.096+0000 7f97c7fff700 1 -- 172.21.15.6:0/2691196260 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97d4019030 con 0x7f97e00b1630 2023-11-28T13:00:37.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.097+0000 7f97c7fff700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 0x7f97cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:37.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.097+0000 7f97c7fff700 1 -- 172.21.15.6:0/2691196260 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f97d40854e0 con 0x7f97e00b1630 2023-11-28T13:00:37.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.097+0000 7f97deffd700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 0x7f97cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:37.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.097+0000 7f97deffd700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 0x7f97cc06a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f97e00b1a70 tx=0x7f97d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:37.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.101+0000 7f97c7fff700 1 -- 172.21.15.6:0/2691196260 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97d40519d0 con 0x7f97e00b1630 2023-11-28T13:00:37.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.343+0000 7f97e61b3700 1 -- 172.21.15.6:0/2691196260 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f97e00b3a70 con 0x7f97e00b1630 2023-11-28T13:00:37.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.951+0000 7f97c7fff700 1 -- 172.21.15.6:0/2691196260 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f97d4054ff0 con 0x7f97e00b1630 2023-11-28T13:00:37.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 -- 172.21.15.6:0/2691196260 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 msgr2=0x7f97cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 0x7f97cc06a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f97e00b1a70 tx=0x7f97d0008030 comp rx=0 tx=0).stop 2023-11-28T13:00:37.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 -- 172.21.15.6:0/2691196260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e00b1630 msgr2=0x7f97e00c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:37.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e00b1630 0x7f97e00c5140 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97d400c700 tx=0x7f97d400cac0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 -- 172.21.15.6:0/2691196260 shutdown_connections 2023-11-28T13:00:37.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e00cf5f0 0x7f97e00c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e00b1630 0x7f97e00c5140 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 0x7f97cc06a7a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 --2- 172.21.15.6:0/2691196260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cb3e0 0x7f97e00c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:37.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 -- 172.21.15.6:0/2691196260 >> 172.21.15.6:0/2691196260 conn(0x7f97e00cc630 msgr2=0x7f97e00d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:37.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 -- 172.21.15.6:0/2691196260 shutdown_connections 2023-11-28T13:00:37.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:37.954+0000 7f97e61b3700 1 -- 172.21.15.6:0/2691196260 wait complete. 2023-11-28T13:00:37.970 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:00:37.971 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:00:38.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.101+0000 7f7a7b347700 1 Processor -- start 2023-11-28T13:00:38.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.101+0000 7f7a7b347700 1 -- start start 2023-11-28T13:00:38.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a7b347700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a74159590 0x7f7a7406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a7b347700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7a7415db70 0x7f7a740ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a7b347700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7a740ce890 0x7f7a740ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a7b347700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a740cf250 con 0x7f7a740ce890 2023-11-28T13:00:38.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a7b347700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7415d120 con 0x7f7a7415db70 2023-11-28T13:00:38.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a7b347700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7415d6e0 con 0x7f7a74159590 2023-11-28T13:00:38.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a798e4700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7a740ce890 0x7f7a740ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a798e4700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7a740ce890 0x7f7a740ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58014/0 (socket says 172.21.15.6:58014) 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a798e4700 1 -- 172.21.15.6:0/2614009671 learned_addr learned my addr 172.21.15.6:0/2614009671 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a790e3700 1 --2- 172.21.15.6:0/2614009671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a74159590 0x7f7a7406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.102+0000 7f7a788e2700 1 --2- 172.21.15.6:0/2614009671 >> v2:172.21.15.61:3300/0 conn(0x7f7a7415db70 0x7f7a740ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.103+0000 7f7a798e4700 1 -- 172.21.15.6:0/2614009671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a74159590 msgr2=0x7f7a7406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.103+0000 7f7a798e4700 1 --2- 172.21.15.6:0/2614009671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a74159590 0x7f7a7406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.103+0000 7f7a798e4700 1 -- 172.21.15.6:0/2614009671 >> v2:172.21.15.61:3300/0 conn(0x7f7a7415db70 msgr2=0x7f7a740ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.103+0000 7f7a798e4700 1 --2- 172.21.15.6:0/2614009671 >> v2:172.21.15.61:3300/0 conn(0x7f7a7415db70 0x7f7a740ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.103+0000 7f7a798e4700 1 -- 172.21.15.6:0/2614009671 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a740aef60 con 0x7f7a740ce890 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.103+0000 7f7a798e4700 1 --2- 172.21.15.6:0/2614009671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a740ce890 0x7f7a740ced10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7a7406f840 tx=0x7f7a6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ea15a14cbf5c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.103+0000 7f7a6bfff700 1 -- 172.21.15.6:0/2614009671 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00e070 con 0x7f7a740ce890 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.103+0000 7f7a6bfff700 1 -- 172.21.15.6:0/2614009671 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6c00b910 con 0x7f7a740ce890 2023-11-28T13:00:38.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.103+0000 7f7a6bfff700 1 -- 172.21.15.6:0/2614009671 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c00bcc0 con 0x7f7a740ce890 2023-11-28T13:00:38.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.104+0000 7f7a7b347700 1 -- 172.21.15.6:0/2614009671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a740ce890 msgr2=0x7f7a740ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.104+0000 7f7a7b347700 1 --2- 172.21.15.6:0/2614009671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a740ce890 0x7f7a740ced10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7a7406f840 tx=0x7f7a6c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:38.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.104+0000 7f7a7b347700 1 -- 172.21.15.6:0/2614009671 shutdown_connections 2023-11-28T13:00:38.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.104+0000 7f7a7b347700 1 --2- 172.21.15.6:0/2614009671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a740ce890 0x7f7a740ced10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.104+0000 7f7a7b347700 1 --2- 172.21.15.6:0/2614009671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a74159590 0x7f7a7406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.104+0000 7f7a7b347700 1 --2- 172.21.15.6:0/2614009671 >> v2:172.21.15.61:3300/0 conn(0x7f7a7415db70 0x7f7a740ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.104+0000 7f7a7b347700 1 -- 172.21.15.6:0/2614009671 >> 172.21.15.6:0/2614009671 conn(0x7f7a740b1650 msgr2=0x7f7a740b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:38.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.104+0000 7f7a7b347700 1 -- 172.21.15.6:0/2614009671 shutdown_connections 2023-11-28T13:00:38.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.104+0000 7f7a7b347700 1 -- 172.21.15.6:0/2614009671 wait complete. 2023-11-28T13:00:38.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.105+0000 7f7a7b347700 1 Processor -- start 2023-11-28T13:00:38.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.105+0000 7f7a7b347700 1 -- start start 2023-11-28T13:00:38.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a7b347700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a740ce890 0x7f7a740c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a7b347700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a74159590 0x7f7a740c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a7b347700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a7415db70 0x7f7a741f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a7b347700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a740c9a60 con 0x7f7a74159590 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a7b347700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7415aae0 con 0x7f7a740ce890 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a7b347700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7415ad80 con 0x7f7a7415db70 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a788e2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a74159590 0x7f7a740c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a788e2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a74159590 0x7f7a740c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58018/0 (socket says 172.21.15.6:58018) 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a788e2700 1 -- 172.21.15.6:0/2358824588 learned_addr learned my addr 172.21.15.6:0/2358824588 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a790e3700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a740ce890 0x7f7a740c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a798e4700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a7415db70 0x7f7a741f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a788e2700 1 -- 172.21.15.6:0/2358824588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a7415db70 msgr2=0x7f7a741f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a788e2700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a7415db70 0x7f7a741f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a788e2700 1 -- 172.21.15.6:0/2358824588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a740ce890 msgr2=0x7f7a740c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a788e2700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a740ce890 0x7f7a740c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a788e2700 1 -- 172.21.15.6:0/2358824588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c009b30 con 0x7f7a74159590 2023-11-28T13:00:38.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a788e2700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a74159590 0x7f7a740c9230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000e870 tx=0x7f7a7000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:38.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.106+0000 7f7a69ffb700 1 -- 172.21.15.6:0/2358824588 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70009590 con 0x7f7a74159590 2023-11-28T13:00:38.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.107+0000 7f7a69ffb700 1 -- 172.21.15.6:0/2358824588 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a70013680 con 0x7f7a74159590 2023-11-28T13:00:38.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.107+0000 7f7a7b347700 1 -- 172.21.15.6:0/2358824588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7415d370 con 0x7f7a74159590 2023-11-28T13:00:38.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.107+0000 7f7a69ffb700 1 -- 172.21.15.6:0/2358824588 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a700139f0 con 0x7f7a74159590 2023-11-28T13:00:38.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.107+0000 7f7a7b347700 1 -- 172.21.15.6:0/2358824588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a740af240 con 0x7f7a74159590 2023-11-28T13:00:38.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.108+0000 7f7a7b347700 1 -- 172.21.15.6:0/2358824588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a740aec70 con 0x7f7a74159590 2023-11-28T13:00:38.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.108+0000 7f7a69ffb700 1 -- 172.21.15.6:0/2358824588 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a70019030 con 0x7f7a74159590 2023-11-28T13:00:38.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.109+0000 7f7a69ffb700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a580682f0 0x7f7a5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.109+0000 7f7a790e3700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a580682f0 0x7f7a5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.109+0000 7f7a69ffb700 1 -- 172.21.15.6:0/2358824588 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7a70085bc0 con 0x7f7a74159590 2023-11-28T13:00:38.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.109+0000 7f7a790e3700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a580682f0 0x7f7a5806a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009be0 tx=0x7f7a64009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:38.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.113+0000 7f7a69ffb700 1 -- 172.21.15.6:0/2358824588 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a70052130 con 0x7f7a74159590 2023-11-28T13:00:38.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.352+0000 7f7a7b347700 1 -- 172.21.15.6:0/2358824588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a740675a0 con 0x7f7a74159590 2023-11-28T13:00:38.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.353+0000 7f7a69ffb700 1 -- 172.21.15.6:0/2358824588 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4638 (secure 0 0 0) 0x7f7a70055750 con 0x7f7a74159590 2023-11-28T13:00:38.355 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:38.355 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6834/1568686201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":1568686201},{"type":"v1","addr":"172.21.15.61:6834","nonce":1568686201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/3333487635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6833","nonce":3333487635},{"type":"v1","addr":"172.21.15.61:6835","nonce":3333487635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3917075730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3917075730},{"type":"v1","addr":"172.21.15.6:6835","nonce":3917075730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:21.404170+0000","modified":"2023-11-28T13:00:37.945716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/1668558371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1668558371},{"type":"v1","addr":"172.21.15.6:6837","nonce":1668558371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:00:38.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 -- 172.21.15.6:0/2358824588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a580682f0 msgr2=0x7f7a5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a580682f0 0x7f7a5806a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009be0 tx=0x7f7a64009210 comp rx=0 tx=0).stop 2023-11-28T13:00:38.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 -- 172.21.15.6:0/2358824588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a74159590 msgr2=0x7f7a740c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a74159590 0x7f7a740c9230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000e870 tx=0x7f7a7000eb80 comp rx=0 tx=0).stop 2023-11-28T13:00:38.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 -- 172.21.15.6:0/2358824588 shutdown_connections 2023-11-28T13:00:38.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a580682f0 0x7f7a5806a7a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7a7415db70 0x7f7a741f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a740ce890 0x7f7a740c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 --2- 172.21.15.6:0/2358824588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a74159590 0x7f7a740c9230 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 -- 172.21.15.6:0/2358824588 >> 172.21.15.6:0/2358824588 conn(0x7f7a740b1650 msgr2=0x7f7a7406d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:38.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 -- 172.21.15.6:0/2358824588 shutdown_connections 2023-11-28T13:00:38.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.355+0000 7f7a7b347700 1 -- 172.21.15.6:0/2358824588 wait complete. 2023-11-28T13:00:38.360 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 9 2023-11-28T13:00:38.368 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:38.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.493+0000 7f399ba9c700 1 Processor -- start 2023-11-28T13:00:38.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.493+0000 7f399ba9c700 1 -- start start 2023-11-28T13:00:38.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.494+0000 7f399ba9c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f399415c990 0x7f39940cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.494+0000 7f399ba9c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f39940cc310 0x7f39940cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.494+0000 7f399ba9c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39940ce340 con 0x7f3994161090 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.494+0000 7f399ba9c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39940cebe0 con 0x7f39940cc310 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.494+0000 7f399ba9c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940cccf0 con 0x7f399415c990 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.494+0000 7f399a039700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f39940cc310 0x7f39940cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.494+0000 7f3999037700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f3994161090 0x7f39940ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:35374/0 (socket says 172.21.15.6:35374) 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.494+0000 7f399a039700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f39940cc310 0x7f39940cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:33514/0 (socket says 172.21.15.6:33514) 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.494+0000 7f3999037700 1 -- 172.21.15.6:0/3517219339 learned_addr learned my addr 172.21.15.6:0/3517219339 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.494+0000 7f3999838700 1 --2- 172.21.15.6:0/3517219339 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f399415c990 0x7f39940cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.495+0000 7f3999838700 1 -- 172.21.15.6:0/3517219339 >> v2:172.21.15.61:3300/0 conn(0x7f39940cc310 msgr2=0x7f39940cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.495+0000 7f3999838700 1 --2- 172.21.15.6:0/3517219339 >> v2:172.21.15.61:3300/0 conn(0x7f39940cc310 0x7f39940cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.495+0000 7f3999838700 1 -- 172.21.15.6:0/3517219339 >> v1:172.21.15.61:6789/0 conn(0x7f3994161090 legacy=0x7f39940ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.495+0000 7f3999838700 1 -- 172.21.15.6:0/3517219339 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39940cb3f0 con 0x7f399415c990 2023-11-28T13:00:38.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.495+0000 7f3999838700 1 --2- 172.21.15.6:0/3517219339 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f399415c990 0x7f39940cf070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3984009840 tx=0x7f3984009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57f5569078385817 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:38.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.495+0000 7f3998836700 1 -- 172.21.15.6:0/3517219339 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984011830 con 0x7f399415c990 2023-11-28T13:00:38.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.495+0000 7f3998836700 1 -- 172.21.15.6:0/3517219339 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3984011990 con 0x7f399415c990 2023-11-28T13:00:38.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.495+0000 7f3998836700 1 -- 172.21.15.6:0/3517219339 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984011d20 con 0x7f399415c990 2023-11-28T13:00:38.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.496+0000 7f399ba9c700 1 -- 172.21.15.6:0/3517219339 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f399415c990 msgr2=0x7f39940cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.496+0000 7f399ba9c700 1 --2- 172.21.15.6:0/3517219339 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f399415c990 0x7f39940cf070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3984009840 tx=0x7f3984009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:38.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.496+0000 7f399ba9c700 1 -- 172.21.15.6:0/3517219339 shutdown_connections 2023-11-28T13:00:38.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.496+0000 7f399ba9c700 1 --2- 172.21.15.6:0/3517219339 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f399415c990 0x7f39940cf070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.496+0000 7f399ba9c700 1 --2- 172.21.15.6:0/3517219339 >> v2:172.21.15.61:3300/0 conn(0x7f39940cc310 0x7f39940cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.496+0000 7f399ba9c700 1 -- 172.21.15.6:0/3517219339 >> 172.21.15.6:0/3517219339 conn(0x7f39940b1650 msgr2=0x7f39940b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:38.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.496+0000 7f399ba9c700 1 -- 172.21.15.6:0/3517219339 shutdown_connections 2023-11-28T13:00:38.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.496+0000 7f399ba9c700 1 -- 172.21.15.6:0/3517219339 wait complete. 2023-11-28T13:00:38.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.497+0000 7f399ba9c700 1 Processor -- start 2023-11-28T13:00:38.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.497+0000 7f399ba9c700 1 -- start start 2023-11-28T13:00:38.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f399ba9c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39940cc310 0x7f39940c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f399ba9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f399415c990 0x7f39941f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f399ba9c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3994161090 0x7f39941f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f399ba9c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c95f0 con 0x7f39940cc310 2023-11-28T13:00:38.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f399ba9c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c7bc0 con 0x7f399415c990 2023-11-28T13:00:38.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f399ba9c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b3560 con 0x7f3994161090 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f3999838700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39940cc310 0x7f39940c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f3999838700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39940cc310 0x7f39940c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58030/0 (socket says 172.21.15.6:58030) 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f3999838700 1 -- 172.21.15.6:0/28742322 learned_addr learned my addr 172.21.15.6:0/28742322 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f3999037700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f399415c990 0x7f39941f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f399a039700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3994161090 0x7f39941f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f3999838700 1 -- 172.21.15.6:0/28742322 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3994161090 msgr2=0x7f39941f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f3999838700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3994161090 0x7f39941f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f3999838700 1 -- 172.21.15.6:0/28742322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f399415c990 msgr2=0x7f39941f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f3999838700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f399415c990 0x7f39941f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f3999838700 1 -- 172.21.15.6:0/28742322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39840094a0 con 0x7f39940cc310 2023-11-28T13:00:38.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.498+0000 7f3999838700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39940cc310 0x7f39940c8610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3984000c00 tx=0x7f3984004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:38.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.499+0000 7f398a7fc700 1 -- 172.21.15.6:0/28742322 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39840036a0 con 0x7f39940cc310 2023-11-28T13:00:38.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.499+0000 7f398a7fc700 1 -- 172.21.15.6:0/28742322 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398401f070 con 0x7f39940cc310 2023-11-28T13:00:38.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.499+0000 7f399ba9c700 1 -- 172.21.15.6:0/28742322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39940ce870 con 0x7f39940cc310 2023-11-28T13:00:38.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.499+0000 7f398a7fc700 1 -- 172.21.15.6:0/28742322 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984003a40 con 0x7f39940cc310 2023-11-28T13:00:38.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.499+0000 7f399ba9c700 1 -- 172.21.15.6:0/28742322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39940ce590 con 0x7f39940cc310 2023-11-28T13:00:38.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.500+0000 7f399ba9c700 1 -- 172.21.15.6:0/28742322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39940cb080 con 0x7f39940cc310 2023-11-28T13:00:38.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.500+0000 7f398a7fc700 1 -- 172.21.15.6:0/28742322 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f398400f050 con 0x7f39940cc310 2023-11-28T13:00:38.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.500+0000 7f398a7fc700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39780682f0 0x7f397806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.501+0000 7f398a7fc700 1 -- 172.21.15.6:0/28742322 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3984021070 con 0x7f39940cc310 2023-11-28T13:00:38.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.501+0000 7f3999037700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39780682f0 0x7f397806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.501+0000 7f3999037700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39780682f0 0x7f397806a7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3990005910 tx=0x7f3990009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:38.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.505+0000 7f398a7fc700 1 -- 172.21.15.6:0/28742322 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3984053d60 con 0x7f39940cc310 2023-11-28T13:00:38.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.701+0000 7f399ba9c700 1 -- 172.21.15.6:0/28742322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39940675a0 con 0x7f39940cc310 2023-11-28T13:00:38.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.702+0000 7f398a7fc700 1 -- 172.21.15.6:0/28742322 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f3984057380 con 0x7f39940cc310 2023-11-28T13:00:38.704 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:38.704 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:38.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.704+0000 7f399ba9c700 1 -- 172.21.15.6:0/28742322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39780682f0 msgr2=0x7f397806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.704+0000 7f399ba9c700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39780682f0 0x7f397806a7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3990005910 tx=0x7f3990009040 comp rx=0 tx=0).stop 2023-11-28T13:00:38.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.704+0000 7f399ba9c700 1 -- 172.21.15.6:0/28742322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39940cc310 msgr2=0x7f39940c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.704+0000 7f399ba9c700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39940cc310 0x7f39940c8610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3984000c00 tx=0x7f3984004750 comp rx=0 tx=0).stop 2023-11-28T13:00:38.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.705+0000 7f399ba9c700 1 -- 172.21.15.6:0/28742322 shutdown_connections 2023-11-28T13:00:38.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.705+0000 7f399ba9c700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39780682f0 0x7f397806a7a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.705+0000 7f399ba9c700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3994161090 0x7f39941f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.705+0000 7f399ba9c700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39940cc310 0x7f39940c8610 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.705+0000 7f399ba9c700 1 --2- 172.21.15.6:0/28742322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f399415c990 0x7f39941f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.705+0000 7f399ba9c700 1 -- 172.21.15.6:0/28742322 >> 172.21.15.6:0/28742322 conn(0x7f39940b1650 msgr2=0x7f39940c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.705+0000 7f399ba9c700 1 -- 172.21.15.6:0/28742322 shutdown_connections 2023-11-28T13:00:38.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.705+0000 7f399ba9c700 1 -- 172.21.15.6:0/28742322 wait complete. 2023-11-28T13:00:38.718 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:00:38.719 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:38.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.843+0000 7ff43b541700 1 Processor -- start 2023-11-28T13:00:38.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.843+0000 7ff43b541700 1 -- start start 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.844+0000 7ff43b541700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff434159590 0x7ff43406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.844+0000 7ff43b541700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43415d120 con 0x7ff43415db70 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.844+0000 7ff43b541700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43415d6e0 con 0x7ff4340ce890 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.844+0000 7ff43b541700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4340aef60 con 0x7ff434159590 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.844+0000 7ff438adc700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff43415db70 0x7ff4340ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45358/0 (socket says 172.21.15.6:45358) 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.844+0000 7ff438adc700 1 -- 172.21.15.6:0/3445278078 learned_addr learned my addr 172.21.15.6:0/3445278078 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.844+0000 7ff4392dd700 1 --2- 172.21.15.6:0/3445278078 >> v2:172.21.15.61:3300/0 conn(0x7ff434159590 0x7ff43406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.844+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632169397 0 0) 0x7ff43415d120 con 0x7ff43415db70 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.844+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4200057c0 con 0x7ff43415db70 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4101008563 0 0) 0x7ff4200057c0 con 0x7ff43415db70 2023-11-28T13:00:38.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 >> v2:172.21.15.61:3300/0 conn(0x7ff434159590 msgr2=0x7ff43406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 --2- 172.21.15.6:0/3445278078 >> v2:172.21.15.61:3300/0 conn(0x7ff434159590 0x7ff43406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 >> v1:172.21.15.61:6789/0 conn(0x7ff4340ce890 legacy=0x7ff4340caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4340af240 con 0x7ff43415db70 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7ff430001670 con 0x7ff43415db70 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 >> v1:172.21.15.6:6789/0 conn(0x7ff43415db70 legacy=0x7ff4340ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff420005c10 0x7ff420006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff420006a60 0x7ff420006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff420007410 0x7ff42000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340aef60 con 0x7ff420007410 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42000e290 con 0x7ff420006a60 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42000e450 con 0x7ff420005c10 2023-11-28T13:00:38.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff439ade700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff420007410 0x7ff42000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff438adc700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff420006a60 0x7ff420006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff4392dd700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff420005c10 0x7ff420006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff439ade700 1 -- 172.21.15.6:0/3445278078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff420005c10 msgr2=0x7ff420006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:38.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff439ade700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff420005c10 0x7ff420006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff439ade700 1 -- 172.21.15.6:0/3445278078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff420006a60 msgr2=0x7ff420006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:38.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff439ade700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff420006a60 0x7ff420006ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff439ade700 1 -- 172.21.15.6:0/3445278078 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200057c0 con 0x7ff420007410 2023-11-28T13:00:38.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff439ade700 1 -- 172.21.15.6:0/3445278078 reap_dead start 2023-11-28T13:00:38.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff439ade700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff420007410 0x7ff42000dc20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c190 tx=0x7ff420007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420d07344274fb61 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c001db0 con 0x7ff420007410 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c00ccf0 con 0x7ff420007410 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.845+0000 7ff42bfff700 1 -- 172.21.15.6:0/3445278078 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c003c40 con 0x7ff420007410 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.846+0000 7ff43b541700 1 -- 172.21.15.6:0/3445278078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff420007410 msgr2=0x7ff42000dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.846+0000 7ff43b541700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff420007410 0x7ff42000dc20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c190 tx=0x7ff420007a90 comp rx=0 tx=0).stop 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.846+0000 7ff43b541700 1 -- 172.21.15.6:0/3445278078 shutdown_connections 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.846+0000 7ff43b541700 1 --2- 172.21.15.6:0/3445278078 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff420007410 0x7ff42000dc20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.846+0000 7ff43b541700 1 -- 172.21.15.6:0/3445278078 >> 172.21.15.6:0/3445278078 conn(0x7ff4340b1650 msgr2=0x7ff4340b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.846+0000 7ff43b541700 1 -- 172.21.15.6:0/3445278078 shutdown_connections 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.846+0000 7ff43b541700 1 -- 172.21.15.6:0/3445278078 wait complete. 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.847+0000 7ff43b541700 1 Processor -- start 2023-11-28T13:00:38.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.847+0000 7ff43b541700 1 -- start start 2023-11-28T13:00:38.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff43b541700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff420007410 0x7ff434151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff43b541700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff4340b52e0 0x7ff4340c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff43b541700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4340c83e0 0x7ff4340c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff43b541700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340b57c0 con 0x7ff4340c83e0 2023-11-28T13:00:38.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff43b541700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43415acf0 con 0x7ff4340b52e0 2023-11-28T13:00:38.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff43b541700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43415afd0 con 0x7ff420007410 2023-11-28T13:00:38.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff439ade700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4340c83e0 0x7ff4340c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff4392dd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff420007410 0x7ff434151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff439ade700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4340c83e0 0x7ff4340c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58054/0 (socket says 172.21.15.6:58054) 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff438adc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff4340b52e0 0x7ff4340c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff439ade700 1 -- 172.21.15.6:0/1861903273 learned_addr learned my addr 172.21.15.6:0/1861903273 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff439ade700 1 -- 172.21.15.6:0/1861903273 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff420007410 msgr2=0x7ff434151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff439ade700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff420007410 0x7ff434151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff439ade700 1 -- 172.21.15.6:0/1861903273 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff4340b52e0 msgr2=0x7ff4340c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff439ade700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff4340b52e0 0x7ff4340c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.848+0000 7ff439ade700 1 -- 172.21.15.6:0/1861903273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43415e010 con 0x7ff4340c83e0 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.849+0000 7ff439ade700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4340c83e0 0x7ff4340c8850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff4340ca180 tx=0x7ff42c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.849+0000 7ff429ffb700 1 -- 172.21.15.6:0/1861903273 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c005070 con 0x7ff4340c83e0 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.849+0000 7ff429ffb700 1 -- 172.21.15.6:0/1861903273 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c0051f0 con 0x7ff4340c83e0 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.849+0000 7ff43b541700 1 -- 172.21.15.6:0/1861903273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43415d370 con 0x7ff4340c83e0 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.849+0000 7ff429ffb700 1 -- 172.21.15.6:0/1861903273 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c0055a0 con 0x7ff4340c83e0 2023-11-28T13:00:38.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.849+0000 7ff43b541700 1 -- 172.21.15.6:0/1861903273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff434159960 con 0x7ff4340c83e0 2023-11-28T13:00:38.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.850+0000 7ff43b541700 1 -- 172.21.15.6:0/1861903273 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4340675a0 con 0x7ff4340c83e0 2023-11-28T13:00:38.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.850+0000 7ff429ffb700 1 -- 172.21.15.6:0/1861903273 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff42c010050 con 0x7ff4340c83e0 2023-11-28T13:00:38.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.851+0000 7ff429ffb700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4180682f0 0x7ff41806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:38.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.851+0000 7ff429ffb700 1 -- 172.21.15.6:0/1861903273 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff42c087090 con 0x7ff4340c83e0 2023-11-28T13:00:38.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.851+0000 7ff4392dd700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4180682f0 0x7ff41806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:38.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.851+0000 7ff4392dd700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4180682f0 0x7ff41806a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4340c6b20 tx=0x7ff424009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:38.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:38.855+0000 7ff429ffb700 1 -- 172.21.15.6:0/1861903273 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff42c053660 con 0x7ff4340c83e0 2023-11-28T13:00:39.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.047+0000 7ff43b541700 1 -- 172.21.15.6:0/1861903273 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff43406c1b0 con 0x7ff4180682f0 2023-11-28T13:00:39.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.058+0000 7ff429ffb700 1 -- 172.21.15.6:0/1861903273 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367539 (secure 0 0 0) 0x7ff43406c1b0 con 0x7ff4180682f0 2023-11-28T13:00:39.064 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:39.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.061+0000 7ff43b541700 1 -- 172.21.15.6:0/1861903273 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4180682f0 msgr2=0x7ff41806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.061+0000 7ff43b541700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4180682f0 0x7ff41806a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4340c6b20 tx=0x7ff424009210 comp rx=0 tx=0).stop 2023-11-28T13:00:39.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.061+0000 7ff43b541700 1 -- 172.21.15.6:0/1861903273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4340c83e0 msgr2=0x7ff4340c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.061+0000 7ff43b541700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4340c83e0 0x7ff4340c8850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff4340ca180 tx=0x7ff42c0014a0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.061+0000 7ff43b541700 1 -- 172.21.15.6:0/1861903273 shutdown_connections 2023-11-28T13:00:39.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.062+0000 7ff43b541700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff4340c83e0 0x7ff4340c8850 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.062+0000 7ff43b541700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4180682f0 0x7ff41806a7a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.062+0000 7ff43b541700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff420007410 0x7ff434151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.062+0000 7ff43b541700 1 --2- 172.21.15.6:0/1861903273 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff4340b52e0 0x7ff4340c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.062+0000 7ff43b541700 1 -- 172.21.15.6:0/1861903273 >> 172.21.15.6:0/1861903273 conn(0x7ff4340b1650 msgr2=0x7ff4340b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:39.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.062+0000 7ff43b541700 1 -- 172.21.15.6:0/1861903273 shutdown_connections 2023-11-28T13:00:39.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.062+0000 7ff43b541700 1 -- 172.21.15.6:0/1861903273 wait complete. 2023-11-28T13:00:39.068 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:00:39.075 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-11-28T13:00:38.946157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594755,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67232,"kb_used_data":3296,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907488,"statfs":{"total":773094113280,"available":773025267712,"internally_reserved":0,"allocated":3375104,"data_stored":2065615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2057,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002619"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.456044+0000","last_change":"2023-11-28T13:00:21.430642+0000","last_active":"2023-11-28T13:00:25.456044+0000","last_peered":"2023-11-28T13:00:25.456044+0000","last_clean":"2023-11-28T13:00:25.456044+0000","last_became_active":"2023-11-28T13:00:21.430357+0000","last_became_peered":"2023-11-28T13:00:21.430357+0000","last_unstale":"2023-11-28T13:00:25.456044+0000","last_undegraded":"2023-11-28T13:00:25.456044+0000","last_fullsized":"2023-11-28T13:00:25.456044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:43.433683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452930+0000","last_change":"2023-11-28T13:00:21.432299+0000","last_active":"2023-11-28T13:00:25.452930+0000","last_peered":"2023-11-28T13:00:25.452930+0000","last_clean":"2023-11-28T13:00:25.452930+0000","last_became_active":"2023-11-28T13:00:21.431566+0000","last_became_peered":"2023-11-28T13:00:21.431566+0000","last_unstale":"2023-11-28T13:00:25.452930+0000","last_undegraded":"2023-11-28T13:00:25.452930+0000","last_fullsized":"2023-11-28T13:00:25.452930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:24.765270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.461121+0000","last_change":"2023-11-28T13:00:21.430850+0000","last_active":"2023-11-28T13:00:25.461121+0000","last_peered":"2023-11-28T13:00:25.461121+0000","last_clean":"2023-11-28T13:00:25.461121+0000","last_became_active":"2023-11-28T13:00:21.430557+0000","last_became_peered":"2023-11-28T13:00:21.430557+0000","last_unstale":"2023-11-28T13:00:25.461121+0000","last_undegraded":"2023-11-28T13:00:25.461121+0000","last_fullsized":"2023-11-28T13:00:25.461121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:00.080768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442074+0000","last_change":"2023-11-28T13:00:21.437362+0000","last_active":"2023-11-28T13:00:25.442074+0000","last_peered":"2023-11-28T13:00:25.442074+0000","last_clean":"2023-11-28T13:00:25.442074+0000","last_became_active":"2023-11-28T13:00:21.435046+0000","last_became_peered":"2023-11-28T13:00:21.435046+0000","last_unstale":"2023-11-28T13:00:25.442074+0000","last_undegraded":"2023-11-28T13:00:25.442074+0000","last_fullsized":"2023-11-28T13:00:25.442074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:39.731416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448772+0000","last_change":"2023-11-28T13:00:21.436404+0000","last_active":"2023-11-28T13:00:25.448772+0000","last_peered":"2023-11-28T13:00:25.448772+0000","last_clean":"2023-11-28T13:00:25.448772+0000","last_became_active":"2023-11-28T13:00:21.436158+0000","last_became_peered":"2023-11-28T13:00:21.436158+0000","last_unstale":"2023-11-28T13:00:25.448772+0000","last_undegraded":"2023-11-28T13:00:25.448772+0000","last_fullsized":"2023-11-28T13:00:25.448772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:42.706331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460337+0000","last_change":"2023-11-28T13:00:21.431231+0000","last_active":"2023-11-28T13:00:25.460337+0000","last_peered":"2023-11-28T13:00:25.460337+0000","last_clean":"2023-11-28T13:00:25.460337+0000","last_became_active":"2023-11-28T13:00:21.430602+0000","last_became_peered":"2023-11-28T13:00:21.430602+0000","last_unstale":"2023-11-28T13:00:25.460337+0000","last_undegraded":"2023-11-28T13:00:25.460337+0000","last_fullsized":"2023-11-28T13:00:25.460337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:13.072237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443249+0000","last_change":"2023-11-28T13:00:21.419513+0000","last_active":"2023-11-28T13:00:25.443249+0000","last_peered":"2023-11-28T13:00:25.443249+0000","last_clean":"2023-11-28T13:00:25.443249+0000","last_became_active":"2023-11-28T13:00:21.418668+0000","last_became_peered":"2023-11-28T13:00:21.418668+0000","last_unstale":"2023-11-28T13:00:25.443249+0000","last_undegraded":"2023-11-28T13:00:25.443249+0000","last_fullsized":"2023-11-28T13:00:25.443249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448590+0000","last_change":"2023-11-28T13:00:21.432362+0000","last_active":"2023-11-28T13:00:25.448590+0000","last_peered":"2023-11-28T13:00:25.448590+0000","last_clean":"2023-11-28T13:00:25.448590+0000","last_became_active":"2023-11-28T13:00:21.432056+0000","last_became_peered":"2023-11-28T13:00:21.432056+0000","last_unstale":"2023-11-28T13:00:25.448590+0000","last_undegraded":"2023-11-28T13:00:25.448590+0000","last_fullsized":"2023-11-28T13:00:25.448590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:13.039552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448588+0000","last_change":"2023-11-28T13:00:21.434750+0000","last_active":"2023-11-28T13:00:25.448588+0000","last_peered":"2023-11-28T13:00:25.448588+0000","last_clean":"2023-11-28T13:00:25.448588+0000","last_became_active":"2023-11-28T13:00:21.434060+0000","last_became_peered":"2023-11-28T13:00:21.434060+0000","last_unstale":"2023-11-28T13:00:25.448588+0000","last_undegraded":"2023-11-28T13:00:25.448588+0000","last_fullsized":"2023-11-28T13:00:25.448588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:34.468496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.453105+0000","last_change":"2023-11-28T13:00:21.433386+0000","last_active":"2023-11-28T13:00:25.453105+0000","last_peered":"2023-11-28T13:00:25.453105+0000","last_clean":"2023-11-28T13:00:25.453105+0000","last_became_active":"2023-11-28T13:00:21.432005+0000","last_became_peered":"2023-11-28T13:00:21.432005+0000","last_unstale":"2023-11-28T13:00:25.453105+0000","last_undegraded":"2023-11-28T13:00:25.453105+0000","last_fullsized":"2023-11-28T13:00:25.453105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:11.589800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.461025+0000","last_change":"2023-11-28T13:00:21.427275+0000","last_active":"2023-11-28T13:00:25.461025+0000","last_peered":"2023-11-28T13:00:25.461025+0000","last_clean":"2023-11-28T13:00:25.461025+0000","last_became_active":"2023-11-28T13:00:21.426893+0000","last_became_peered":"2023-11-28T13:00:21.426893+0000","last_unstale":"2023-11-28T13:00:25.461025+0000","last_undegraded":"2023-11-28T13:00:25.461025+0000","last_fullsized":"2023-11-28T13:00:25.461025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:24.765270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455341+0000","last_change":"2023-11-28T13:00:21.436650+0000","last_active":"2023-11-28T13:00:25.455341+0000","last_peered":"2023-11-28T13:00:25.455341+0000","last_clean":"2023-11-28T13:00:25.455341+0000","last_became_active":"2023-11-28T13:00:21.435700+0000","last_became_peered":"2023-11-28T13:00:21.435700+0000","last_unstale":"2023-11-28T13:00:25.455341+0000","last_undegraded":"2023-11-28T13:00:25.455341+0000","last_fullsized":"2023-11-28T13:00:25.455341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:02.740267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.454713+0000","last_change":"2023-11-28T13:00:21.434352+0000","last_active":"2023-11-28T13:00:25.454713+0000","last_peered":"2023-11-28T13:00:25.454713+0000","last_clean":"2023-11-28T13:00:25.454713+0000","last_became_active":"2023-11-28T13:00:21.434067+0000","last_became_peered":"2023-11-28T13:00:21.434067+0000","last_unstale":"2023-11-28T13:00:25.454713+0000","last_undegraded":"2023-11-28T13:00:25.454713+0000","last_fullsized":"2023-11-28T13:00:25.454713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:30.629627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449953+0000","last_change":"2023-11-28T13:00:21.431774+0000","last_active":"2023-11-28T13:00:25.449953+0000","last_peered":"2023-11-28T13:00:25.449953+0000","last_clean":"2023-11-28T13:00:25.449953+0000","last_became_active":"2023-11-28T13:00:21.431445+0000","last_became_peered":"2023-11-28T13:00:21.431445+0000","last_unstale":"2023-11-28T13:00:25.449953+0000","last_undegraded":"2023-11-28T13:00:25.449953+0000","last_fullsized":"2023-11-28T13:00:25.449953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:24.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443018+0000","last_change":"2023-11-28T13:00:21.420404+0000","last_active":"2023-11-28T13:00:25.443018+0000","last_peered":"2023-11-28T13:00:25.443018+0000","last_clean":"2023-11-28T13:00:25.443018+0000","last_became_active":"2023-11-28T13:00:21.419228+0000","last_became_peered":"2023-11-28T13:00:21.419228+0000","last_unstale":"2023-11-28T13:00:25.443018+0000","last_undegraded":"2023-11-28T13:00:25.443018+0000","last_fullsized":"2023-11-28T13:00:25.443018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:40:24.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458890+0000","last_change":"2023-11-28T13:00:21.430943+0000","last_active":"2023-11-28T13:00:25.458890+0000","last_peered":"2023-11-28T13:00:25.458890+0000","last_clean":"2023-11-28T13:00:25.458890+0000","last_became_active":"2023-11-28T13:00:21.430664+0000","last_became_peered":"2023-11-28T13:00:21.430664+0000","last_unstale":"2023-11-28T13:00:25.458890+0000","last_undegraded":"2023-11-28T13:00:25.458890+0000","last_fullsized":"2023-11-28T13:00:25.458890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:12:14.186445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448670+0000","last_change":"2023-11-28T13:00:21.425335+0000","last_active":"2023-11-28T13:00:25.448670+0000","last_peered":"2023-11-28T13:00:25.448670+0000","last_clean":"2023-11-28T13:00:25.448670+0000","last_became_active":"2023-11-28T13:00:21.425073+0000","last_became_peered":"2023-11-28T13:00:21.425073+0000","last_unstale":"2023-11-28T13:00:25.448670+0000","last_undegraded":"2023-11-28T13:00:25.448670+0000","last_fullsized":"2023-11-28T13:00:25.448670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:45.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460831+0000","last_change":"2023-11-28T13:00:21.426161+0000","last_active":"2023-11-28T13:00:25.460831+0000","last_peered":"2023-11-28T13:00:25.460831+0000","last_clean":"2023-11-28T13:00:25.460831+0000","last_became_active":"2023-11-28T13:00:21.425796+0000","last_became_peered":"2023-11-28T13:00:21.425796+0000","last_unstale":"2023-11-28T13:00:25.460831+0000","last_undegraded":"2023-11-28T13:00:25.460831+0000","last_fullsized":"2023-11-28T13:00:25.460831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:17.422401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455990+0000","last_change":"2023-11-28T13:00:21.423565+0000","last_active":"2023-11-28T13:00:25.455990+0000","last_peered":"2023-11-28T13:00:25.455990+0000","last_clean":"2023-11-28T13:00:25.455990+0000","last_became_active":"2023-11-28T13:00:21.423270+0000","last_became_peered":"2023-11-28T13:00:21.423270+0000","last_unstale":"2023-11-28T13:00:25.455990+0000","last_undegraded":"2023-11-28T13:00:25.455990+0000","last_fullsized":"2023-11-28T13:00:25.455990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:31.139532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460193+0000","last_change":"2023-11-28T13:00:21.426452+0000","last_active":"2023-11-28T13:00:25.460193+0000","last_peered":"2023-11-28T13:00:25.460193+0000","last_clean":"2023-11-28T13:00:25.460193+0000","last_became_active":"2023-11-28T13:00:21.426096+0000","last_became_peered":"2023-11-28T13:00:21.426096+0000","last_unstale":"2023-11-28T13:00:25.460193+0000","last_undegraded":"2023-11-28T13:00:25.460193+0000","last_fullsized":"2023-11-28T13:00:25.460193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:33.439707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443668+0000","last_change":"2023-11-28T13:00:21.435483+0000","last_active":"2023-11-28T13:00:25.443668+0000","last_peered":"2023-11-28T13:00:25.443668+0000","last_clean":"2023-11-28T13:00:25.443668+0000","last_became_active":"2023-11-28T13:00:21.435162+0000","last_became_peered":"2023-11-28T13:00:21.435162+0000","last_unstale":"2023-11-28T13:00:25.443668+0000","last_undegraded":"2023-11-28T13:00:25.443668+0000","last_fullsized":"2023-11-28T13:00:25.443668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:13.039552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448596+0000","last_change":"2023-11-28T13:00:21.417958+0000","last_active":"2023-11-28T13:00:25.448596+0000","last_peered":"2023-11-28T13:00:25.448596+0000","last_clean":"2023-11-28T13:00:25.448596+0000","last_became_active":"2023-11-28T13:00:21.417664+0000","last_became_peered":"2023-11-28T13:00:21.417664+0000","last_unstale":"2023-11-28T13:00:25.448596+0000","last_undegraded":"2023-11-28T13:00:25.448596+0000","last_fullsized":"2023-11-28T13:00:25.448596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:35.062024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441395+0000","last_change":"2023-11-28T13:00:21.432315+0000","last_active":"2023-11-28T13:00:25.441395+0000","last_peered":"2023-11-28T13:00:25.441395+0000","last_clean":"2023-11-28T13:00:25.441395+0000","last_became_active":"2023-11-28T13:00:21.431878+0000","last_became_peered":"2023-11-28T13:00:21.431878+0000","last_unstale":"2023-11-28T13:00:25.441395+0000","last_undegraded":"2023-11-28T13:00:25.441395+0000","last_fullsized":"2023-11-28T13:00:25.441395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:26:43.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446220+0000","last_change":"2023-11-28T13:00:21.429436+0000","last_active":"2023-11-28T13:00:25.446220+0000","last_peered":"2023-11-28T13:00:25.446220+0000","last_clean":"2023-11-28T13:00:25.446220+0000","last_became_active":"2023-11-28T13:00:21.429051+0000","last_became_peered":"2023-11-28T13:00:21.429051+0000","last_unstale":"2023-11-28T13:00:25.446220+0000","last_undegraded":"2023-11-28T13:00:25.446220+0000","last_fullsized":"2023-11-28T13:00:25.446220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:11.589800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458565+0000","last_change":"2023-11-28T13:00:21.423512+0000","last_active":"2023-11-28T13:00:25.458565+0000","last_peered":"2023-11-28T13:00:25.458565+0000","last_clean":"2023-11-28T13:00:25.458565+0000","last_became_active":"2023-11-28T13:00:21.423223+0000","last_became_peered":"2023-11-28T13:00:21.423223+0000","last_unstale":"2023-11-28T13:00:25.458565+0000","last_undegraded":"2023-11-28T13:00:25.458565+0000","last_fullsized":"2023-11-28T13:00:25.458565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:35.062024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457987+0000","last_change":"2023-11-28T13:00:21.426702+0000","last_active":"2023-11-28T13:00:25.457987+0000","last_peered":"2023-11-28T13:00:25.457987+0000","last_clean":"2023-11-28T13:00:25.457987+0000","last_became_active":"2023-11-28T13:00:21.425807+0000","last_became_peered":"2023-11-28T13:00:21.425807+0000","last_unstale":"2023-11-28T13:00:25.457987+0000","last_undegraded":"2023-11-28T13:00:25.457987+0000","last_fullsized":"2023-11-28T13:00:25.457987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:04.077060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452451+0000","last_change":"2023-11-28T13:00:21.434848+0000","last_active":"2023-11-28T13:00:25.452451+0000","last_peered":"2023-11-28T13:00:25.452451+0000","last_clean":"2023-11-28T13:00:25.452451+0000","last_became_active":"2023-11-28T13:00:21.432193+0000","last_became_peered":"2023-11-28T13:00:21.432193+0000","last_unstale":"2023-11-28T13:00:25.452451+0000","last_undegraded":"2023-11-28T13:00:25.452451+0000","last_fullsized":"2023-11-28T13:00:25.452451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:43.321870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871441+0000","last_change":"2023-11-28T13:00:25.470736+0000","last_active":"2023-11-28T13:00:25.871441+0000","last_peered":"2023-11-28T13:00:25.871441+0000","last_clean":"2023-11-28T13:00:25.871441+0000","last_became_active":"2023-11-28T13:00:25.470009+0000","last_became_peered":"2023-11-28T13:00:25.470009+0000","last_unstale":"2023-11-28T13:00:25.871441+0000","last_undegraded":"2023-11-28T13:00:25.871441+0000","last_fullsized":"2023-11-28T13:00:25.871441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:05.706754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459220+0000","last_change":"2023-11-28T13:00:21.006957+0000","last_active":"2023-11-28T13:00:25.459220+0000","last_peered":"2023-11-28T13:00:25.459220+0000","last_clean":"2023-11-28T13:00:25.459220+0000","last_became_active":"2023-11-28T13:00:21.006690+0000","last_became_peered":"2023-11-28T13:00:21.006690+0000","last_unstale":"2023-11-28T13:00:25.459220+0000","last_undegraded":"2023-11-28T13:00:25.459220+0000","last_fullsized":"2023-11-28T13:00:25.459220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:30.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441650+0000","last_change":"2023-11-28T13:00:21.430560+0000","last_active":"2023-11-28T13:00:25.441650+0000","last_peered":"2023-11-28T13:00:25.441650+0000","last_clean":"2023-11-28T13:00:25.441650+0000","last_became_active":"2023-11-28T13:00:21.430312+0000","last_became_peered":"2023-11-28T13:00:21.430312+0000","last_unstale":"2023-11-28T13:00:25.441650+0000","last_undegraded":"2023-11-28T13:00:25.441650+0000","last_fullsized":"2023-11-28T13:00:25.441650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347196+0000","last_change":"2023-11-28T13:00:25.460012+0000","last_active":"2023-11-28T13:00:26.347196+0000","last_peered":"2023-11-28T13:00:26.347196+0000","last_clean":"2023-11-28T13:00:26.347196+0000","last_became_active":"2023-11-28T13:00:25.459532+0000","last_became_peered":"2023-11-28T13:00:25.459532+0000","last_unstale":"2023-11-28T13:00:26.347196+0000","last_undegraded":"2023-11-28T13:00:26.347196+0000","last_fullsized":"2023-11-28T13:00:26.347196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:36.890229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448628+0000","last_change":"2023-11-28T13:00:21.005058+0000","last_active":"2023-11-28T13:00:25.448628+0000","last_peered":"2023-11-28T13:00:25.448628+0000","last_clean":"2023-11-28T13:00:25.448628+0000","last_became_active":"2023-11-28T13:00:21.004749+0000","last_became_peered":"2023-11-28T13:00:21.004749+0000","last_unstale":"2023-11-28T13:00:25.448628+0000","last_undegraded":"2023-11-28T13:00:25.448628+0000","last_fullsized":"2023-11-28T13:00:25.448628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:22.962275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448569+0000","last_change":"2023-11-28T13:00:21.416645+0000","last_active":"2023-11-28T13:00:25.448569+0000","last_peered":"2023-11-28T13:00:25.448569+0000","last_clean":"2023-11-28T13:00:25.448569+0000","last_became_active":"2023-11-28T13:00:21.416365+0000","last_became_peered":"2023-11-28T13:00:21.416365+0000","last_unstale":"2023-11-28T13:00:25.448569+0000","last_undegraded":"2023-11-28T13:00:25.448569+0000","last_fullsized":"2023-11-28T13:00:25.448569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:14.429821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073812+0000","last_change":"2023-11-28T13:00:25.468574+0000","last_active":"2023-11-28T13:00:26.073812+0000","last_peered":"2023-11-28T13:00:26.073812+0000","last_clean":"2023-11-28T13:00:26.073812+0000","last_became_active":"2023-11-28T13:00:25.468177+0000","last_became_peered":"2023-11-28T13:00:25.468177+0000","last_unstale":"2023-11-28T13:00:26.073812+0000","last_undegraded":"2023-11-28T13:00:26.073812+0000","last_fullsized":"2023-11-28T13:00:26.073812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:13:55.892226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441558+0000","last_change":"2023-11-28T13:00:21.005438+0000","last_active":"2023-11-28T13:00:25.441558+0000","last_peered":"2023-11-28T13:00:25.441558+0000","last_clean":"2023-11-28T13:00:25.441558+0000","last_became_active":"2023-11-28T13:00:21.004885+0000","last_became_peered":"2023-11-28T13:00:21.004885+0000","last_unstale":"2023-11-28T13:00:25.441558+0000","last_undegraded":"2023-11-28T13:00:25.441558+0000","last_fullsized":"2023-11-28T13:00:25.441558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:47.845351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442760+0000","last_change":"2023-11-28T13:00:21.437422+0000","last_active":"2023-11-28T13:00:25.442760+0000","last_peered":"2023-11-28T13:00:25.442760+0000","last_clean":"2023-11-28T13:00:25.442760+0000","last_became_active":"2023-11-28T13:00:21.437095+0000","last_became_peered":"2023-11-28T13:00:21.437095+0000","last_unstale":"2023-11-28T13:00:25.442760+0000","last_undegraded":"2023-11-28T13:00:25.442760+0000","last_fullsized":"2023-11-28T13:00:25.442760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:52.972220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346252+0000","last_change":"2023-11-28T13:00:25.468548+0000","last_active":"2023-11-28T13:00:26.346252+0000","last_peered":"2023-11-28T13:00:26.346252+0000","last_clean":"2023-11-28T13:00:26.346252+0000","last_became_active":"2023-11-28T13:00:25.468071+0000","last_became_peered":"2023-11-28T13:00:25.468071+0000","last_unstale":"2023-11-28T13:00:26.346252+0000","last_undegraded":"2023-11-28T13:00:26.346252+0000","last_fullsized":"2023-11-28T13:00:26.346252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:20.872923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448774+0000","last_change":"2023-11-28T13:00:21.003619+0000","last_active":"2023-11-28T13:00:25.448774+0000","last_peered":"2023-11-28T13:00:25.448774+0000","last_clean":"2023-11-28T13:00:25.448774+0000","last_became_active":"2023-11-28T13:00:21.003189+0000","last_became_peered":"2023-11-28T13:00:21.003189+0000","last_unstale":"2023-11-28T13:00:25.448774+0000","last_undegraded":"2023-11-28T13:00:25.448774+0000","last_fullsized":"2023-11-28T13:00:25.448774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:53.009716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441443+0000","last_change":"2023-11-28T13:00:21.428000+0000","last_active":"2023-11-28T13:00:25.441443+0000","last_peered":"2023-11-28T13:00:25.441443+0000","last_clean":"2023-11-28T13:00:25.441443+0000","last_became_active":"2023-11-28T13:00:21.427738+0000","last_became_peered":"2023-11-28T13:00:21.427738+0000","last_unstale":"2023-11-28T13:00:25.441443+0000","last_undegraded":"2023-11-28T13:00:25.441443+0000","last_fullsized":"2023-11-28T13:00:25.441443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:24.529486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871318+0000","last_change":"2023-11-28T13:00:25.463375+0000","last_active":"2023-11-28T13:00:25.871318+0000","last_peered":"2023-11-28T13:00:25.871318+0000","last_clean":"2023-11-28T13:00:25.871318+0000","last_became_active":"2023-11-28T13:00:25.462962+0000","last_became_peered":"2023-11-28T13:00:25.462962+0000","last_unstale":"2023-11-28T13:00:25.871318+0000","last_undegraded":"2023-11-28T13:00:25.871318+0000","last_fullsized":"2023-11-28T13:00:25.871318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:16.522760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452987+0000","last_change":"2023-11-28T13:00:21.011149+0000","last_active":"2023-11-28T13:00:25.452987+0000","last_peered":"2023-11-28T13:00:25.452987+0000","last_clean":"2023-11-28T13:00:25.452987+0000","last_became_active":"2023-11-28T13:00:21.010813+0000","last_became_peered":"2023-11-28T13:00:21.010813+0000","last_unstale":"2023-11-28T13:00:25.452987+0000","last_undegraded":"2023-11-28T13:00:25.452987+0000","last_fullsized":"2023-11-28T13:00:25.452987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441165+0000","last_change":"2023-11-28T13:00:21.428947+0000","last_active":"2023-11-28T13:00:25.441165+0000","last_peered":"2023-11-28T13:00:25.441165+0000","last_clean":"2023-11-28T13:00:25.441165+0000","last_became_active":"2023-11-28T13:00:21.428690+0000","last_became_peered":"2023-11-28T13:00:21.428690+0000","last_unstale":"2023-11-28T13:00:25.441165+0000","last_undegraded":"2023-11-28T13:00:25.441165+0000","last_fullsized":"2023-11-28T13:00:25.441165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:55:41.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869911+0000","last_change":"2023-11-28T13:00:25.468634+0000","last_active":"2023-11-28T13:00:25.869911+0000","last_peered":"2023-11-28T13:00:25.869911+0000","last_clean":"2023-11-28T13:00:25.869911+0000","last_became_active":"2023-11-28T13:00:25.468224+0000","last_became_peered":"2023-11-28T13:00:25.468224+0000","last_unstale":"2023-11-28T13:00:25.869911+0000","last_undegraded":"2023-11-28T13:00:25.869911+0000","last_fullsized":"2023-11-28T13:00:25.869911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:21.591804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446244+0000","last_change":"2023-11-28T13:00:21.006053+0000","last_active":"2023-11-28T13:00:25.446244+0000","last_peered":"2023-11-28T13:00:25.446244+0000","last_clean":"2023-11-28T13:00:25.446244+0000","last_became_active":"2023-11-28T13:00:21.005693+0000","last_became_peered":"2023-11-28T13:00:21.005693+0000","last_unstale":"2023-11-28T13:00:25.446244+0000","last_undegraded":"2023-11-28T13:00:25.446244+0000","last_fullsized":"2023-11-28T13:00:25.446244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:38.720815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457494+0000","last_change":"2023-11-28T13:00:21.426158+0000","last_active":"2023-11-28T13:00:25.457494+0000","last_peered":"2023-11-28T13:00:25.457494+0000","last_clean":"2023-11-28T13:00:25.457494+0000","last_became_active":"2023-11-28T13:00:21.425771+0000","last_became_peered":"2023-11-28T13:00:21.425771+0000","last_unstale":"2023-11-28T13:00:25.457494+0000","last_undegraded":"2023-11-28T13:00:25.457494+0000","last_fullsized":"2023-11-28T13:00:25.457494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:48.996841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076004+0000","last_change":"2023-11-28T13:00:25.467575+0000","last_active":"2023-11-28T13:00:26.076004+0000","last_peered":"2023-11-28T13:00:26.076004+0000","last_clean":"2023-11-28T13:00:26.076004+0000","last_became_active":"2023-11-28T13:00:25.466169+0000","last_became_peered":"2023-11-28T13:00:25.466169+0000","last_unstale":"2023-11-28T13:00:26.076004+0000","last_undegraded":"2023-11-28T13:00:26.076004+0000","last_fullsized":"2023-11-28T13:00:26.076004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:57.102759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:29.849985+0000","last_change":"2023-11-28T13:00:21.000813+0000","last_active":"2023-11-28T13:00:29.849985+0000","last_peered":"2023-11-28T13:00:29.849985+0000","last_clean":"2023-11-28T13:00:29.849985+0000","last_became_active":"2023-11-28T13:00:20.999079+0000","last_became_peered":"2023-11-28T13:00:20.999079+0000","last_unstale":"2023-11-28T13:00:29.849985+0000","last_undegraded":"2023-11-28T13:00:29.849985+0000","last_fullsized":"2023-11-28T13:00:29.849985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:04.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3605,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460800+0000","last_change":"2023-11-28T13:00:21.422098+0000","last_active":"2023-11-28T13:00:25.460800+0000","last_peered":"2023-11-28T13:00:25.460800+0000","last_clean":"2023-11-28T13:00:25.460800+0000","last_became_active":"2023-11-28T13:00:21.421774+0000","last_became_peered":"2023-11-28T13:00:21.421774+0000","last_unstale":"2023-11-28T13:00:25.460800+0000","last_undegraded":"2023-11-28T13:00:25.460800+0000","last_fullsized":"2023-11-28T13:00:25.460800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:08.151957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347474+0000","last_change":"2023-11-28T13:00:25.468792+0000","last_active":"2023-11-28T13:00:26.347474+0000","last_peered":"2023-11-28T13:00:26.347474+0000","last_clean":"2023-11-28T13:00:26.347474+0000","last_became_active":"2023-11-28T13:00:25.468376+0000","last_became_peered":"2023-11-28T13:00:25.468376+0000","last_unstale":"2023-11-28T13:00:26.347474+0000","last_undegraded":"2023-11-28T13:00:26.347474+0000","last_fullsized":"2023-11-28T13:00:26.347474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:51.024234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443541+0000","last_change":"2023-11-28T13:00:21.003466+0000","last_active":"2023-11-28T13:00:25.443541+0000","last_peered":"2023-11-28T13:00:25.443541+0000","last_clean":"2023-11-28T13:00:25.443541+0000","last_became_active":"2023-11-28T13:00:21.003127+0000","last_became_peered":"2023-11-28T13:00:21.003127+0000","last_unstale":"2023-11-28T13:00:25.443541+0000","last_undegraded":"2023-11-28T13:00:25.443541+0000","last_fullsized":"2023-11-28T13:00:25.443541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441650+0000","last_change":"2023-11-28T13:00:21.429933+0000","last_active":"2023-11-28T13:00:25.441650+0000","last_peered":"2023-11-28T13:00:25.441650+0000","last_clean":"2023-11-28T13:00:25.441650+0000","last_became_active":"2023-11-28T13:00:21.429681+0000","last_became_peered":"2023-11-28T13:00:21.429681+0000","last_unstale":"2023-11-28T13:00:25.441650+0000","last_undegraded":"2023-11-28T13:00:25.441650+0000","last_fullsized":"2023-11-28T13:00:25.441650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:45.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869412+0000","last_change":"2023-11-28T13:00:25.468518+0000","last_active":"2023-11-28T13:00:25.869412+0000","last_peered":"2023-11-28T13:00:25.869412+0000","last_clean":"2023-11-28T13:00:25.869412+0000","last_became_active":"2023-11-28T13:00:25.468083+0000","last_became_peered":"2023-11-28T13:00:25.468083+0000","last_unstale":"2023-11-28T13:00:25.869412+0000","last_undegraded":"2023-11-28T13:00:25.869412+0000","last_fullsized":"2023-11-28T13:00:25.869412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:53:56.156971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448763+0000","last_change":"2023-11-28T13:00:20.997868+0000","last_active":"2023-11-28T13:00:25.448763+0000","last_peered":"2023-11-28T13:00:25.448763+0000","last_clean":"2023-11-28T13:00:25.448763+0000","last_became_active":"2023-11-28T13:00:20.997565+0000","last_became_peered":"2023-11-28T13:00:20.997565+0000","last_unstale":"2023-11-28T13:00:25.448763+0000","last_undegraded":"2023-11-28T13:00:25.448763+0000","last_fullsized":"2023-11-28T13:00:25.448763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:37.465748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.451059+0000","last_change":"2023-11-28T13:00:21.431844+0000","last_active":"2023-11-28T13:00:25.451059+0000","last_peered":"2023-11-28T13:00:25.451059+0000","last_clean":"2023-11-28T13:00:25.451059+0000","last_became_active":"2023-11-28T13:00:21.431496+0000","last_became_peered":"2023-11-28T13:00:21.431496+0000","last_unstale":"2023-11-28T13:00:25.451059+0000","last_undegraded":"2023-11-28T13:00:25.451059+0000","last_fullsized":"2023-11-28T13:00:25.451059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.154908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869150+0000","last_change":"2023-11-28T13:00:25.469324+0000","last_active":"2023-11-28T13:00:25.869150+0000","last_peered":"2023-11-28T13:00:25.869150+0000","last_clean":"2023-11-28T13:00:25.869150+0000","last_became_active":"2023-11-28T13:00:25.468890+0000","last_became_peered":"2023-11-28T13:00:25.468890+0000","last_unstale":"2023-11-28T13:00:25.869150+0000","last_undegraded":"2023-11-28T13:00:25.869150+0000","last_fullsized":"2023-11-28T13:00:25.869150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:41.487196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443130+0000","last_change":"2023-11-28T13:00:21.002295+0000","last_active":"2023-11-28T13:00:25.443130+0000","last_peered":"2023-11-28T13:00:25.443130+0000","last_clean":"2023-11-28T13:00:25.443130+0000","last_became_active":"2023-11-28T13:00:21.001997+0000","last_became_peered":"2023-11-28T13:00:21.001997+0000","last_unstale":"2023-11-28T13:00:25.443130+0000","last_undegraded":"2023-11-28T13:00:25.443130+0000","last_fullsized":"2023-11-28T13:00:25.443130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449818+0000","last_change":"2023-11-28T13:00:21.431066+0000","last_active":"2023-11-28T13:00:25.449818+0000","last_peered":"2023-11-28T13:00:25.449818+0000","last_clean":"2023-11-28T13:00:25.449818+0000","last_became_active":"2023-11-28T13:00:21.430741+0000","last_became_peered":"2023-11-28T13:00:21.430741+0000","last_unstale":"2023-11-28T13:00:25.449818+0000","last_undegraded":"2023-11-28T13:00:25.449818+0000","last_fullsized":"2023-11-28T13:00:25.449818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:55.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347017+0000","last_change":"2023-11-28T13:00:25.468987+0000","last_active":"2023-11-28T13:00:26.347017+0000","last_peered":"2023-11-28T13:00:26.347017+0000","last_clean":"2023-11-28T13:00:26.347017+0000","last_became_active":"2023-11-28T13:00:25.468345+0000","last_became_peered":"2023-11-28T13:00:25.468345+0000","last_unstale":"2023-11-28T13:00:26.347017+0000","last_undegraded":"2023-11-28T13:00:26.347017+0000","last_fullsized":"2023-11-28T13:00:26.347017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:16.201562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441951+0000","last_change":"2023-11-28T13:00:20.993647+0000","last_active":"2023-11-28T13:00:25.441951+0000","last_peered":"2023-11-28T13:00:25.441951+0000","last_clean":"2023-11-28T13:00:25.441951+0000","last_became_active":"2023-11-28T13:00:20.993298+0000","last_became_peered":"2023-11-28T13:00:20.993298+0000","last_unstale":"2023-11-28T13:00:25.441951+0000","last_undegraded":"2023-11-28T13:00:25.441951+0000","last_fullsized":"2023-11-28T13:00:25.441951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:14:48.024917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442632+0000","last_change":"2023-11-28T13:00:21.429127+0000","last_active":"2023-11-28T13:00:25.442632+0000","last_peered":"2023-11-28T13:00:25.442632+0000","last_clean":"2023-11-28T13:00:25.442632+0000","last_became_active":"2023-11-28T13:00:21.428830+0000","last_became_peered":"2023-11-28T13:00:21.428830+0000","last_unstale":"2023-11-28T13:00:25.442632+0000","last_undegraded":"2023-11-28T13:00:25.442632+0000","last_fullsized":"2023-11-28T13:00:25.442632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:55.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871898+0000","last_change":"2023-11-28T13:00:25.464255+0000","last_active":"2023-11-28T13:00:25.871898+0000","last_peered":"2023-11-28T13:00:25.871898+0000","last_clean":"2023-11-28T13:00:25.871898+0000","last_became_active":"2023-11-28T13:00:25.463731+0000","last_became_peered":"2023-11-28T13:00:25.463731+0000","last_unstale":"2023-11-28T13:00:25.871898+0000","last_undegraded":"2023-11-28T13:00:25.871898+0000","last_fullsized":"2023-11-28T13:00:25.871898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:23:35.278662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.447931+0000","last_change":"2023-11-28T13:00:21.002097+0000","last_active":"2023-11-28T13:00:25.447931+0000","last_peered":"2023-11-28T13:00:25.447931+0000","last_clean":"2023-11-28T13:00:25.447931+0000","last_became_active":"2023-11-28T13:00:21.001830+0000","last_became_peered":"2023-11-28T13:00:21.001830+0000","last_unstale":"2023-11-28T13:00:25.447931+0000","last_undegraded":"2023-11-28T13:00:25.447931+0000","last_fullsized":"2023-11-28T13:00:25.447931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:18.850685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448366+0000","last_change":"2023-11-28T13:00:21.426617+0000","last_active":"2023-11-28T13:00:25.448366+0000","last_peered":"2023-11-28T13:00:25.448366+0000","last_clean":"2023-11-28T13:00:25.448366+0000","last_became_active":"2023-11-28T13:00:21.426326+0000","last_became_peered":"2023-11-28T13:00:21.426326+0000","last_unstale":"2023-11-28T13:00:25.448366+0000","last_undegraded":"2023-11-28T13:00:25.448366+0000","last_fullsized":"2023-11-28T13:00:25.448366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231013+0000","last_change":"2023-11-28T13:00:25.476625+0000","last_active":"2023-11-28T13:00:26.231013+0000","last_peered":"2023-11-28T13:00:26.231013+0000","last_clean":"2023-11-28T13:00:26.231013+0000","last_became_active":"2023-11-28T13:00:25.476137+0000","last_became_peered":"2023-11-28T13:00:25.476137+0000","last_unstale":"2023-11-28T13:00:26.231013+0000","last_undegraded":"2023-11-28T13:00:26.231013+0000","last_fullsized":"2023-11-28T13:00:26.231013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:28.631772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446518+0000","last_change":"2023-11-28T13:00:21.005624+0000","last_active":"2023-11-28T13:00:25.446518+0000","last_peered":"2023-11-28T13:00:25.446518+0000","last_clean":"2023-11-28T13:00:25.446518+0000","last_became_active":"2023-11-28T13:00:21.005092+0000","last_became_peered":"2023-11-28T13:00:21.005092+0000","last_unstale":"2023-11-28T13:00:25.446518+0000","last_undegraded":"2023-11-28T13:00:25.446518+0000","last_fullsized":"2023-11-28T13:00:25.446518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:37.060637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459224+0000","last_change":"2023-11-28T13:00:21.439380+0000","last_active":"2023-11-28T13:00:25.459224+0000","last_peered":"2023-11-28T13:00:25.459224+0000","last_clean":"2023-11-28T13:00:25.459224+0000","last_became_active":"2023-11-28T13:00:21.439094+0000","last_became_peered":"2023-11-28T13:00:21.439094+0000","last_unstale":"2023-11-28T13:00:25.459224+0000","last_undegraded":"2023-11-28T13:00:25.459224+0000","last_fullsized":"2023-11-28T13:00:25.459224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:02.248744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231207+0000","last_change":"2023-11-28T13:00:25.464879+0000","last_active":"2023-11-28T13:00:26.231207+0000","last_peered":"2023-11-28T13:00:26.231207+0000","last_clean":"2023-11-28T13:00:26.231207+0000","last_became_active":"2023-11-28T13:00:25.464486+0000","last_became_peered":"2023-11-28T13:00:25.464486+0000","last_unstale":"2023-11-28T13:00:26.231207+0000","last_undegraded":"2023-11-28T13:00:26.231207+0000","last_fullsized":"2023-11-28T13:00:26.231207+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:39.922810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446121+0000","last_change":"2023-11-28T13:00:21.003453+0000","last_active":"2023-11-28T13:00:25.446121+0000","last_peered":"2023-11-28T13:00:25.446121+0000","last_clean":"2023-11-28T13:00:25.446121+0000","last_became_active":"2023-11-28T13:00:21.003010+0000","last_became_peered":"2023-11-28T13:00:21.003010+0000","last_unstale":"2023-11-28T13:00:25.446121+0000","last_undegraded":"2023-11-28T13:00:25.446121+0000","last_fullsized":"2023-11-28T13:00:25.446121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:50.427721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442176+0000","last_change":"2023-11-28T13:00:21.428448+0000","last_active":"2023-11-28T13:00:25.442176+0000","last_peered":"2023-11-28T13:00:25.442176+0000","last_clean":"2023-11-28T13:00:25.442176+0000","last_became_active":"2023-11-28T13:00:21.427997+0000","last_became_peered":"2023-11-28T13:00:21.427997+0000","last_unstale":"2023-11-28T13:00:25.442176+0000","last_undegraded":"2023-11-28T13:00:25.442176+0000","last_fullsized":"2023-11-28T13:00:25.442176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:41.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231241+0000","last_change":"2023-11-28T13:00:25.465904+0000","last_active":"2023-11-28T13:00:26.231241+0000","last_peered":"2023-11-28T13:00:26.231241+0000","last_clean":"2023-11-28T13:00:26.231241+0000","last_became_active":"2023-11-28T13:00:25.465285+0000","last_became_peered":"2023-11-28T13:00:25.465285+0000","last_unstale":"2023-11-28T13:00:26.231241+0000","last_undegraded":"2023-11-28T13:00:26.231241+0000","last_fullsized":"2023-11-28T13:00:26.231241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:36.890229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450628+0000","last_change":"2023-11-28T13:00:21.010220+0000","last_active":"2023-11-28T13:00:25.450628+0000","last_peered":"2023-11-28T13:00:25.450628+0000","last_clean":"2023-11-28T13:00:25.450628+0000","last_became_active":"2023-11-28T13:00:21.009892+0000","last_became_peered":"2023-11-28T13:00:21.009892+0000","last_unstale":"2023-11-28T13:00:25.450628+0000","last_undegraded":"2023-11-28T13:00:25.450628+0000","last_fullsized":"2023-11-28T13:00:25.450628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448937+0000","last_change":"2023-11-28T13:00:21.421459+0000","last_active":"2023-11-28T13:00:25.448937+0000","last_peered":"2023-11-28T13:00:25.448937+0000","last_clean":"2023-11-28T13:00:25.448937+0000","last_became_active":"2023-11-28T13:00:21.420822+0000","last_became_peered":"2023-11-28T13:00:21.420822+0000","last_unstale":"2023-11-28T13:00:25.448937+0000","last_undegraded":"2023-11-28T13:00:25.448937+0000","last_fullsized":"2023-11-28T13:00:25.448937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:32:01.515518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231366+0000","last_change":"2023-11-28T13:00:25.464941+0000","last_active":"2023-11-28T13:00:26.231366+0000","last_peered":"2023-11-28T13:00:26.231366+0000","last_clean":"2023-11-28T13:00:26.231366+0000","last_became_active":"2023-11-28T13:00:25.464504+0000","last_became_peered":"2023-11-28T13:00:25.464504+0000","last_unstale":"2023-11-28T13:00:26.231366+0000","last_undegraded":"2023-11-28T13:00:26.231366+0000","last_fullsized":"2023-11-28T13:00:26.231366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:11.602479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441429+0000","last_change":"2023-11-28T13:00:21.008391+0000","last_active":"2023-11-28T13:00:25.441429+0000","last_peered":"2023-11-28T13:00:25.441429+0000","last_clean":"2023-11-28T13:00:25.441429+0000","last_became_active":"2023-11-28T13:00:21.006513+0000","last_became_peered":"2023-11-28T13:00:21.006513+0000","last_unstale":"2023-11-28T13:00:25.441429+0000","last_undegraded":"2023-11-28T13:00:25.441429+0000","last_fullsized":"2023-11-28T13:00:25.441429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:58.676237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460081+0000","last_change":"2023-11-28T13:00:21.431780+0000","last_active":"2023-11-28T13:00:25.460081+0000","last_peered":"2023-11-28T13:00:25.460081+0000","last_clean":"2023-11-28T13:00:25.460081+0000","last_became_active":"2023-11-28T13:00:21.430937+0000","last_became_peered":"2023-11-28T13:00:21.430937+0000","last_unstale":"2023-11-28T13:00:25.460081+0000","last_undegraded":"2023-11-28T13:00:25.460081+0000","last_fullsized":"2023-11-28T13:00:25.460081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:12:13.983682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076305+0000","last_change":"2023-11-28T13:00:25.466505+0000","last_active":"2023-11-28T13:00:26.076305+0000","last_peered":"2023-11-28T13:00:26.076305+0000","last_clean":"2023-11-28T13:00:26.076305+0000","last_became_active":"2023-11-28T13:00:25.466059+0000","last_became_peered":"2023-11-28T13:00:25.466059+0000","last_unstale":"2023-11-28T13:00:26.076305+0000","last_undegraded":"2023-11-28T13:00:26.076305+0000","last_fullsized":"2023-11-28T13:00:26.076305+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:12.856136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442306+0000","last_change":"2023-11-28T13:00:21.003064+0000","last_active":"2023-11-28T13:00:25.442306+0000","last_peered":"2023-11-28T13:00:25.442306+0000","last_clean":"2023-11-28T13:00:25.442306+0000","last_became_active":"2023-11-28T13:00:21.002742+0000","last_became_peered":"2023-11-28T13:00:21.002742+0000","last_unstale":"2023-11-28T13:00:25.442306+0000","last_undegraded":"2023-11-28T13:00:25.442306+0000","last_fullsized":"2023-11-28T13:00:25.442306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:04.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460629+0000","last_change":"2023-11-28T13:00:21.427213+0000","last_active":"2023-11-28T13:00:25.460629+0000","last_peered":"2023-11-28T13:00:25.460629+0000","last_clean":"2023-11-28T13:00:25.460629+0000","last_became_active":"2023-11-28T13:00:21.426737+0000","last_became_peered":"2023-11-28T13:00:21.426737+0000","last_unstale":"2023-11-28T13:00:25.460629+0000","last_undegraded":"2023-11-28T13:00:25.460629+0000","last_fullsized":"2023-11-28T13:00:25.460629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:43.321870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347427+0000","last_change":"2023-11-28T13:00:25.466146+0000","last_active":"2023-11-28T13:00:26.347427+0000","last_peered":"2023-11-28T13:00:26.347427+0000","last_clean":"2023-11-28T13:00:26.347427+0000","last_became_active":"2023-11-28T13:00:25.465652+0000","last_became_peered":"2023-11-28T13:00:25.465652+0000","last_unstale":"2023-11-28T13:00:26.347427+0000","last_undegraded":"2023-11-28T13:00:26.347427+0000","last_fullsized":"2023-11-28T13:00:26.347427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:14.273148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459071+0000","last_change":"2023-11-28T13:00:21.001745+0000","last_active":"2023-11-28T13:00:25.459071+0000","last_peered":"2023-11-28T13:00:25.459071+0000","last_clean":"2023-11-28T13:00:25.459071+0000","last_became_active":"2023-11-28T13:00:21.001473+0000","last_became_peered":"2023-11-28T13:00:21.001473+0000","last_unstale":"2023-11-28T13:00:25.459071+0000","last_undegraded":"2023-11-28T13:00:25.459071+0000","last_fullsized":"2023-11-28T13:00:25.459071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.462841+0000","last_change":"2023-11-28T13:00:21.422938+0000","last_active":"2023-11-28T13:00:25.462841+0000","last_peered":"2023-11-28T13:00:25.462841+0000","last_clean":"2023-11-28T13:00:25.462841+0000","last_became_active":"2023-11-28T13:00:21.422654+0000","last_became_peered":"2023-11-28T13:00:21.422654+0000","last_unstale":"2023-11-28T13:00:25.462841+0000","last_undegraded":"2023-11-28T13:00:25.462841+0000","last_fullsized":"2023-11-28T13:00:25.462841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:28:59.158475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.870965+0000","last_change":"2023-11-28T13:00:25.463080+0000","last_active":"2023-11-28T13:00:25.870965+0000","last_peered":"2023-11-28T13:00:25.870965+0000","last_clean":"2023-11-28T13:00:25.870965+0000","last_became_active":"2023-11-28T13:00:25.462460+0000","last_became_peered":"2023-11-28T13:00:25.462460+0000","last_unstale":"2023-11-28T13:00:25.870965+0000","last_undegraded":"2023-11-28T13:00:25.870965+0000","last_fullsized":"2023-11-28T13:00:25.870965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:45.428316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441326+0000","last_change":"2023-11-28T13:00:21.009988+0000","last_active":"2023-11-28T13:00:25.441326+0000","last_peered":"2023-11-28T13:00:25.441326+0000","last_clean":"2023-11-28T13:00:25.441326+0000","last_became_active":"2023-11-28T13:00:21.009637+0000","last_became_peered":"2023-11-28T13:00:21.009637+0000","last_unstale":"2023-11-28T13:00:25.441326+0000","last_undegraded":"2023-11-28T13:00:25.441326+0000","last_fullsized":"2023-11-28T13:00:25.441326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:53.009716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458749+0000","last_change":"2023-11-28T13:00:21.433621+0000","last_active":"2023-11-28T13:00:25.458749+0000","last_peered":"2023-11-28T13:00:25.458749+0000","last_clean":"2023-11-28T13:00:25.458749+0000","last_became_active":"2023-11-28T13:00:21.433293+0000","last_became_peered":"2023-11-28T13:00:21.433293+0000","last_unstale":"2023-11-28T13:00:25.458749+0000","last_undegraded":"2023-11-28T13:00:25.458749+0000","last_fullsized":"2023-11-28T13:00:25.458749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:18.248057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.870341+0000","last_change":"2023-11-28T13:00:25.460697+0000","last_active":"2023-11-28T13:00:25.870341+0000","last_peered":"2023-11-28T13:00:25.870341+0000","last_clean":"2023-11-28T13:00:25.870341+0000","last_became_active":"2023-11-28T13:00:25.460230+0000","last_became_peered":"2023-11-28T13:00:25.460230+0000","last_unstale":"2023-11-28T13:00:25.870341+0000","last_undegraded":"2023-11-28T13:00:25.870341+0000","last_fullsized":"2023-11-28T13:00:25.870341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:25.897486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442429+0000","last_change":"2023-11-28T13:00:21.001229+0000","last_active":"2023-11-28T13:00:25.442429+0000","last_peered":"2023-11-28T13:00:25.442429+0000","last_clean":"2023-11-28T13:00:25.442429+0000","last_became_active":"2023-11-28T13:00:21.000871+0000","last_became_peered":"2023-11-28T13:00:21.000871+0000","last_unstale":"2023-11-28T13:00:25.442429+0000","last_undegraded":"2023-11-28T13:00:25.442429+0000","last_fullsized":"2023-11-28T13:00:25.442429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:49.179565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460903+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:00:25.460903+0000","last_peered":"2023-11-28T13:00:25.460903+0000","last_clean":"2023-11-28T13:00:25.460903+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:00:25.460903+0000","last_undegraded":"2023-11-28T13:00:25.460903+0000","last_fullsized":"2023-11-28T13:00:25.460903+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441199+0000","last_change":"2023-11-28T13:00:21.429135+0000","last_active":"2023-11-28T13:00:25.441199+0000","last_peered":"2023-11-28T13:00:25.441199+0000","last_clean":"2023-11-28T13:00:25.441199+0000","last_became_active":"2023-11-28T13:00:21.428808+0000","last_became_peered":"2023-11-28T13:00:21.428808+0000","last_unstale":"2023-11-28T13:00:25.441199+0000","last_undegraded":"2023-11-28T13:00:25.441199+0000","last_fullsized":"2023-11-28T13:00:25.441199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:28:36.151382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.868731+0000","last_change":"2023-11-28T13:00:25.472188+0000","last_active":"2023-11-28T13:00:25.868731+0000","last_peered":"2023-11-28T13:00:25.868731+0000","last_clean":"2023-11-28T13:00:25.868731+0000","last_became_active":"2023-11-28T13:00:25.471768+0000","last_became_peered":"2023-11-28T13:00:25.471768+0000","last_unstale":"2023-11-28T13:00:25.868731+0000","last_undegraded":"2023-11-28T13:00:25.868731+0000","last_fullsized":"2023-11-28T13:00:25.868731+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:17:47.454963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441547+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:00:25.441547+0000","last_peered":"2023-11-28T13:00:25.441547+0000","last_clean":"2023-11-28T13:00:25.441547+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:00:25.441547+0000","last_undegraded":"2023-11-28T13:00:25.441547+0000","last_fullsized":"2023-11-28T13:00:25.441547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073655+0000","last_change":"2023-11-28T13:00:20.998799+0000","last_active":"2023-11-28T13:00:26.073655+0000","last_peered":"2023-11-28T13:00:26.073655+0000","last_clean":"2023-11-28T13:00:26.073655+0000","last_became_active":"2023-11-28T13:00:20.998482+0000","last_became_peered":"2023-11-28T13:00:20.998482+0000","last_unstale":"2023-11-28T13:00:26.073655+0000","last_undegraded":"2023-11-28T13:00:26.073655+0000","last_fullsized":"2023-11-28T13:00:26.073655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:28.673649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459183+0000","last_change":"2023-11-28T13:00:21.440454+0000","last_active":"2023-11-28T13:00:25.459183+0000","last_peered":"2023-11-28T13:00:25.459183+0000","last_clean":"2023-11-28T13:00:25.459183+0000","last_became_active":"2023-11-28T13:00:21.439616+0000","last_became_peered":"2023-11-28T13:00:21.439616+0000","last_unstale":"2023-11-28T13:00:25.459183+0000","last_undegraded":"2023-11-28T13:00:25.459183+0000","last_fullsized":"2023-11-28T13:00:25.459183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:36.472288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347208+0000","last_change":"2023-11-28T13:00:25.468244+0000","last_active":"2023-11-28T13:00:26.347208+0000","last_peered":"2023-11-28T13:00:26.347208+0000","last_clean":"2023-11-28T13:00:26.347208+0000","last_became_active":"2023-11-28T13:00:25.467850+0000","last_became_peered":"2023-11-28T13:00:25.467850+0000","last_unstale":"2023-11-28T13:00:26.347208+0000","last_undegraded":"2023-11-28T13:00:26.347208+0000","last_fullsized":"2023-11-28T13:00:26.347208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:47.712713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.456847+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:00:25.456847+0000","last_peered":"2023-11-28T13:00:25.456847+0000","last_clean":"2023-11-28T13:00:25.456847+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:00:25.456847+0000","last_undegraded":"2023-11-28T13:00:25.456847+0000","last_fullsized":"2023-11-28T13:00:25.456847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446771+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:00:25.446771+0000","last_peered":"2023-11-28T13:00:25.446771+0000","last_clean":"2023-11-28T13:00:25.446771+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:00:25.446771+0000","last_undegraded":"2023-11-28T13:00:25.446771+0000","last_fullsized":"2023-11-28T13:00:25.446771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.447961+0000","last_change":"2023-11-28T13:00:21.005940+0000","last_active":"2023-11-28T13:00:25.447961+0000","last_peered":"2023-11-28T13:00:25.447961+0000","last_clean":"2023-11-28T13:00:25.447961+0000","last_became_active":"2023-11-28T13:00:21.005231+0000","last_became_peered":"2023-11-28T13:00:21.005231+0000","last_unstale":"2023-11-28T13:00:25.447961+0000","last_undegraded":"2023-11-28T13:00:25.447961+0000","last_fullsized":"2023-11-28T13:00:25.447961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:49.179565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448989+0000","last_change":"2023-11-28T13:00:21.426195+0000","last_active":"2023-11-28T13:00:25.448989+0000","last_peered":"2023-11-28T13:00:25.448989+0000","last_clean":"2023-11-28T13:00:25.448989+0000","last_became_active":"2023-11-28T13:00:21.423397+0000","last_became_peered":"2023-11-28T13:00:21.423397+0000","last_unstale":"2023-11-28T13:00:25.448989+0000","last_undegraded":"2023-11-28T13:00:25.448989+0000","last_fullsized":"2023-11-28T13:00:25.448989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231539+0000","last_change":"2023-11-28T13:00:25.468335+0000","last_active":"2023-11-28T13:00:26.231539+0000","last_peered":"2023-11-28T13:00:26.231539+0000","last_clean":"2023-11-28T13:00:26.231539+0000","last_became_active":"2023-11-28T13:00:25.467898+0000","last_became_peered":"2023-11-28T13:00:25.467898+0000","last_unstale":"2023-11-28T13:00:26.231539+0000","last_undegraded":"2023-11-28T13:00:26.231539+0000","last_fullsized":"2023-11-28T13:00:26.231539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:20.872923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443294+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:00:25.443294+0000","last_peered":"2023-11-28T13:00:25.443294+0000","last_clean":"2023-11-28T13:00:25.443294+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:00:25.443294+0000","last_undegraded":"2023-11-28T13:00:25.443294+0000","last_fullsized":"2023-11-28T13:00:25.443294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460654+0000","last_change":"2023-11-28T13:00:21.007444+0000","last_active":"2023-11-28T13:00:25.460654+0000","last_peered":"2023-11-28T13:00:25.460654+0000","last_clean":"2023-11-28T13:00:25.460654+0000","last_became_active":"2023-11-28T13:00:21.007159+0000","last_became_peered":"2023-11-28T13:00:21.007159+0000","last_unstale":"2023-11-28T13:00:25.460654+0000","last_undegraded":"2023-11-28T13:00:25.460654+0000","last_fullsized":"2023-11-28T13:00:25.460654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:55.415887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458654+0000","last_change":"2023-11-28T13:00:21.437327+0000","last_active":"2023-11-28T13:00:25.458654+0000","last_peered":"2023-11-28T13:00:25.458654+0000","last_clean":"2023-11-28T13:00:25.458654+0000","last_became_active":"2023-11-28T13:00:21.437058+0000","last_became_peered":"2023-11-28T13:00:21.437058+0000","last_unstale":"2023-11-28T13:00:25.458654+0000","last_undegraded":"2023-11-28T13:00:25.458654+0000","last_fullsized":"2023-11-28T13:00:25.458654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871438+0000","last_change":"2023-11-28T13:00:25.462821+0000","last_active":"2023-11-28T13:00:25.871438+0000","last_peered":"2023-11-28T13:00:25.871438+0000","last_clean":"2023-11-28T13:00:25.871438+0000","last_became_active":"2023-11-28T13:00:25.462378+0000","last_became_peered":"2023-11-28T13:00:25.462378+0000","last_unstale":"2023-11-28T13:00:25.871438+0000","last_undegraded":"2023-11-28T13:00:25.871438+0000","last_fullsized":"2023-11-28T13:00:25.871438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:06:52.582450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446128+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:00:25.446128+0000","last_peered":"2023-11-28T13:00:25.446128+0000","last_clean":"2023-11-28T13:00:25.446128+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:00:25.446128+0000","last_undegraded":"2023-11-28T13:00:25.446128+0000","last_fullsized":"2023-11-28T13:00:25.446128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441243+0000","last_change":"2023-11-28T13:00:21.005259+0000","last_active":"2023-11-28T13:00:25.441243+0000","last_peered":"2023-11-28T13:00:25.441243+0000","last_clean":"2023-11-28T13:00:25.441243+0000","last_became_active":"2023-11-28T13:00:21.004973+0000","last_became_peered":"2023-11-28T13:00:21.004973+0000","last_unstale":"2023-11-28T13:00:25.441243+0000","last_undegraded":"2023-11-28T13:00:25.441243+0000","last_fullsized":"2023-11-28T13:00:25.441243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:18.754380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442055+0000","last_change":"2023-11-28T13:00:21.428421+0000","last_active":"2023-11-28T13:00:25.442055+0000","last_peered":"2023-11-28T13:00:25.442055+0000","last_clean":"2023-11-28T13:00:25.442055+0000","last_became_active":"2023-11-28T13:00:21.427914+0000","last_became_peered":"2023-11-28T13:00:21.427914+0000","last_unstale":"2023-11-28T13:00:25.442055+0000","last_undegraded":"2023-11-28T13:00:25.442055+0000","last_fullsized":"2023-11-28T13:00:25.442055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:13.464393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073795+0000","last_change":"2023-11-28T13:00:25.468907+0000","last_active":"2023-11-28T13:00:26.073795+0000","last_peered":"2023-11-28T13:00:26.073795+0000","last_clean":"2023-11-28T13:00:26.073795+0000","last_became_active":"2023-11-28T13:00:25.468477+0000","last_became_peered":"2023-11-28T13:00:25.468477+0000","last_unstale":"2023-11-28T13:00:26.073795+0000","last_undegraded":"2023-11-28T13:00:26.073795+0000","last_fullsized":"2023-11-28T13:00:26.073795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:36:04.014440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448786+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:00:25.448786+0000","last_peered":"2023-11-28T13:00:25.448786+0000","last_clean":"2023-11-28T13:00:25.448786+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:00:25.448786+0000","last_undegraded":"2023-11-28T13:00:25.448786+0000","last_fullsized":"2023-11-28T13:00:25.448786+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460768+0000","last_change":"2023-11-28T13:00:21.008135+0000","last_active":"2023-11-28T13:00:25.460768+0000","last_peered":"2023-11-28T13:00:25.460768+0000","last_clean":"2023-11-28T13:00:25.460768+0000","last_became_active":"2023-11-28T13:00:21.007883+0000","last_became_peered":"2023-11-28T13:00:21.007883+0000","last_unstale":"2023-11-28T13:00:25.460768+0000","last_undegraded":"2023-11-28T13:00:25.460768+0000","last_fullsized":"2023-11-28T13:00:25.460768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455821+0000","last_change":"2023-11-28T13:00:21.425530+0000","last_active":"2023-11-28T13:00:25.455821+0000","last_peered":"2023-11-28T13:00:25.455821+0000","last_clean":"2023-11-28T13:00:25.455821+0000","last_became_active":"2023-11-28T13:00:21.425325+0000","last_became_peered":"2023-11-28T13:00:21.425325+0000","last_unstale":"2023-11-28T13:00:25.455821+0000","last_undegraded":"2023-11-28T13:00:25.455821+0000","last_fullsized":"2023-11-28T13:00:25.455821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:43.401826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346874+0000","last_change":"2023-11-28T13:00:25.457191+0000","last_active":"2023-11-28T13:00:26.346874+0000","last_peered":"2023-11-28T13:00:26.346874+0000","last_clean":"2023-11-28T13:00:26.346874+0000","last_became_active":"2023-11-28T13:00:25.456577+0000","last_became_peered":"2023-11-28T13:00:25.456577+0000","last_unstale":"2023-11-28T13:00:26.346874+0000","last_undegraded":"2023-11-28T13:00:26.346874+0000","last_fullsized":"2023-11-28T13:00:26.346874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:11.602479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442371+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:00:25.442371+0000","last_peered":"2023-11-28T13:00:25.442371+0000","last_clean":"2023-11-28T13:00:25.442371+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:00:25.442371+0000","last_undegraded":"2023-11-28T13:00:25.442371+0000","last_fullsized":"2023-11-28T13:00:25.442371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455999+0000","last_change":"2023-11-28T13:00:21.006815+0000","last_active":"2023-11-28T13:00:25.455999+0000","last_peered":"2023-11-28T13:00:25.455999+0000","last_clean":"2023-11-28T13:00:25.455999+0000","last_became_active":"2023-11-28T13:00:21.006608+0000","last_became_peered":"2023-11-28T13:00:21.006608+0000","last_unstale":"2023-11-28T13:00:25.455999+0000","last_undegraded":"2023-11-28T13:00:25.455999+0000","last_fullsized":"2023-11-28T13:00:25.455999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:28.673649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448670+0000","last_change":"2023-11-28T13:00:21.421641+0000","last_active":"2023-11-28T13:00:25.448670+0000","last_peered":"2023-11-28T13:00:25.448670+0000","last_clean":"2023-11-28T13:00:25.448670+0000","last_became_active":"2023-11-28T13:00:21.421196+0000","last_became_peered":"2023-11-28T13:00:21.421196+0000","last_unstale":"2023-11-28T13:00:25.448670+0000","last_undegraded":"2023-11-28T13:00:25.448670+0000","last_fullsized":"2023-11-28T13:00:25.448670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:49.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073549+0000","last_change":"2023-11-28T13:00:25.468640+0000","last_active":"2023-11-28T13:00:26.073549+0000","last_peered":"2023-11-28T13:00:26.073549+0000","last_clean":"2023-11-28T13:00:26.073549+0000","last_became_active":"2023-11-28T13:00:25.467992+0000","last_became_peered":"2023-11-28T13:00:25.467992+0000","last_unstale":"2023-11-28T13:00:26.073549+0000","last_undegraded":"2023-11-28T13:00:26.073549+0000","last_fullsized":"2023-11-28T13:00:26.073549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:45:50.526405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442668+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:00:25.442668+0000","last_peered":"2023-11-28T13:00:25.442668+0000","last_clean":"2023-11-28T13:00:25.442668+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:00:25.442668+0000","last_undegraded":"2023-11-28T13:00:25.442668+0000","last_fullsized":"2023-11-28T13:00:25.442668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450339+0000","last_change":"2023-11-28T13:00:21.010100+0000","last_active":"2023-11-28T13:00:25.450339+0000","last_peered":"2023-11-28T13:00:25.450339+0000","last_clean":"2023-11-28T13:00:25.450339+0000","last_became_active":"2023-11-28T13:00:21.009797+0000","last_became_peered":"2023-11-28T13:00:21.009797+0000","last_unstale":"2023-11-28T13:00:25.450339+0000","last_undegraded":"2023-11-28T13:00:25.450339+0000","last_fullsized":"2023-11-28T13:00:25.450339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:18.850685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457393+0000","last_change":"2023-11-28T13:00:21.433346+0000","last_active":"2023-11-28T13:00:25.457393+0000","last_peered":"2023-11-28T13:00:25.457393+0000","last_clean":"2023-11-28T13:00:25.457393+0000","last_became_active":"2023-11-28T13:00:21.433022+0000","last_became_peered":"2023-11-28T13:00:21.433022+0000","last_unstale":"2023-11-28T13:00:25.457393+0000","last_undegraded":"2023-11-28T13:00:25.457393+0000","last_fullsized":"2023-11-28T13:00:25.457393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:34:14.133792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076291+0000","last_change":"2023-11-28T13:00:25.474463+0000","last_active":"2023-11-28T13:00:26.076291+0000","last_peered":"2023-11-28T13:00:26.076291+0000","last_clean":"2023-11-28T13:00:26.076291+0000","last_became_active":"2023-11-28T13:00:25.473727+0000","last_became_peered":"2023-11-28T13:00:25.473727+0000","last_unstale":"2023-11-28T13:00:26.076291+0000","last_undegraded":"2023-11-28T13:00:26.076291+0000","last_fullsized":"2023-11-28T13:00:26.076291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:35:49.356939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448491+0000","last_change":"2023-11-28T13:00:21.005791+0000","last_active":"2023-11-28T13:00:25.448491+0000","last_peered":"2023-11-28T13:00:25.448491+0000","last_clean":"2023-11-28T13:00:25.448491+0000","last_became_active":"2023-11-28T13:00:21.005474+0000","last_became_peered":"2023-11-28T13:00:21.005474+0000","last_unstale":"2023-11-28T13:00:25.448491+0000","last_undegraded":"2023-11-28T13:00:25.448491+0000","last_fullsized":"2023-11-28T13:00:25.448491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:14.890428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455579+0000","last_change":"2023-11-28T13:00:21.424133+0000","last_active":"2023-11-28T13:00:25.455579+0000","last_peered":"2023-11-28T13:00:25.455579+0000","last_clean":"2023-11-28T13:00:25.455579+0000","last_became_active":"2023-11-28T13:00:21.423849+0000","last_became_peered":"2023-11-28T13:00:21.423849+0000","last_unstale":"2023-11-28T13:00:25.455579+0000","last_undegraded":"2023-11-28T13:00:25.455579+0000","last_fullsized":"2023-11-28T13:00:25.455579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:05.801837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871653+0000","last_change":"2023-11-28T13:00:25.468445+0000","last_active":"2023-11-28T13:00:25.871653+0000","last_peered":"2023-11-28T13:00:25.871653+0000","last_clean":"2023-11-28T13:00:25.871653+0000","last_became_active":"2023-11-28T13:00:25.468042+0000","last_became_peered":"2023-11-28T13:00:25.468042+0000","last_unstale":"2023-11-28T13:00:25.871653+0000","last_undegraded":"2023-11-28T13:00:25.871653+0000","last_fullsized":"2023-11-28T13:00:25.871653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.716007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449852+0000","last_change":"2023-11-28T13:00:21.006819+0000","last_active":"2023-11-28T13:00:25.449852+0000","last_peered":"2023-11-28T13:00:25.449852+0000","last_clean":"2023-11-28T13:00:25.449852+0000","last_became_active":"2023-11-28T13:00:21.006504+0000","last_became_peered":"2023-11-28T13:00:21.006504+0000","last_unstale":"2023-11-28T13:00:25.449852+0000","last_undegraded":"2023-11-28T13:00:25.449852+0000","last_fullsized":"2023-11-28T13:00:25.449852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:37.060637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459182+0000","last_change":"2023-11-28T13:00:21.436419+0000","last_active":"2023-11-28T13:00:25.459182+0000","last_peered":"2023-11-28T13:00:25.459182+0000","last_clean":"2023-11-28T13:00:25.459182+0000","last_became_active":"2023-11-28T13:00:21.436087+0000","last_became_peered":"2023-11-28T13:00:21.436087+0000","last_unstale":"2023-11-28T13:00:25.459182+0000","last_undegraded":"2023-11-28T13:00:25.459182+0000","last_fullsized":"2023-11-28T13:00:25.459182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:18.316015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346972+0000","last_change":"2023-11-28T13:00:25.464176+0000","last_active":"2023-11-28T13:00:26.346972+0000","last_peered":"2023-11-28T13:00:26.346972+0000","last_clean":"2023-11-28T13:00:26.346972+0000","last_became_active":"2023-11-28T13:00:25.463779+0000","last_became_peered":"2023-11-28T13:00:25.463779+0000","last_unstale":"2023-11-28T13:00:26.346972+0000","last_undegraded":"2023-11-28T13:00:26.346972+0000","last_fullsized":"2023-11-28T13:00:26.346972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.716007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450222+0000","last_change":"2023-11-28T13:00:21.006862+0000","last_active":"2023-11-28T13:00:25.450222+0000","last_peered":"2023-11-28T13:00:25.450222+0000","last_clean":"2023-11-28T13:00:25.450222+0000","last_became_active":"2023-11-28T13:00:21.006580+0000","last_became_peered":"2023-11-28T13:00:21.006580+0000","last_unstale":"2023-11-28T13:00:25.450222+0000","last_undegraded":"2023-11-28T13:00:25.450222+0000","last_fullsized":"2023-11-28T13:00:25.450222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:01.729666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460103+0000","last_change":"2023-11-28T13:00:21.427093+0000","last_active":"2023-11-28T13:00:25.460103+0000","last_peered":"2023-11-28T13:00:25.460103+0000","last_clean":"2023-11-28T13:00:25.460103+0000","last_became_active":"2023-11-28T13:00:21.426329+0000","last_became_peered":"2023-11-28T13:00:21.426329+0000","last_unstale":"2023-11-28T13:00:25.460103+0000","last_undegraded":"2023-11-28T13:00:25.460103+0000","last_fullsized":"2023-11-28T13:00:25.460103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:26:04.532312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869619+0000","last_change":"2023-11-28T13:00:25.475157+0000","last_active":"2023-11-28T13:00:25.869619+0000","last_peered":"2023-11-28T13:00:25.869619+0000","last_clean":"2023-11-28T13:00:25.869619+0000","last_became_active":"2023-11-28T13:00:25.474714+0000","last_became_peered":"2023-11-28T13:00:25.474714+0000","last_unstale":"2023-11-28T13:00:25.869619+0000","last_undegraded":"2023-11-28T13:00:25.869619+0000","last_fullsized":"2023-11-28T13:00:25.869619+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:43.383854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441539+0000","last_change":"2023-11-28T13:00:21.004157+0000","last_active":"2023-11-28T13:00:25.441539+0000","last_peered":"2023-11-28T13:00:25.441539+0000","last_clean":"2023-11-28T13:00:25.441539+0000","last_became_active":"2023-11-28T13:00:21.003717+0000","last_became_peered":"2023-11-28T13:00:21.003717+0000","last_unstale":"2023-11-28T13:00:25.441539+0000","last_undegraded":"2023-11-28T13:00:25.441539+0000","last_fullsized":"2023-11-28T13:00:25.441539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:14.599072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441293+0000","last_change":"2023-11-28T13:00:21.427401+0000","last_active":"2023-11-28T13:00:25.441293+0000","last_peered":"2023-11-28T13:00:25.441293+0000","last_clean":"2023-11-28T13:00:25.441293+0000","last_became_active":"2023-11-28T13:00:21.425193+0000","last_became_peered":"2023-11-28T13:00:21.425193+0000","last_unstale":"2023-11-28T13:00:25.441293+0000","last_undegraded":"2023-11-28T13:00:25.441293+0000","last_fullsized":"2023-11-28T13:00:25.441293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:16.691765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076011+0000","last_change":"2023-11-28T13:00:25.466935+0000","last_active":"2023-11-28T13:00:26.076011+0000","last_peered":"2023-11-28T13:00:26.076011+0000","last_clean":"2023-11-28T13:00:26.076011+0000","last_became_active":"2023-11-28T13:00:25.466438+0000","last_became_peered":"2023-11-28T13:00:25.466438+0000","last_unstale":"2023-11-28T13:00:26.076011+0000","last_undegraded":"2023-11-28T13:00:26.076011+0000","last_fullsized":"2023-11-28T13:00:26.076011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:44.975678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448944+0000","last_change":"2023-11-28T13:00:21.003458+0000","last_active":"2023-11-28T13:00:25.448944+0000","last_peered":"2023-11-28T13:00:25.448944+0000","last_clean":"2023-11-28T13:00:25.448944+0000","last_became_active":"2023-11-28T13:00:21.002912+0000","last_became_peered":"2023-11-28T13:00:21.002912+0000","last_unstale":"2023-11-28T13:00:25.448944+0000","last_undegraded":"2023-11-28T13:00:25.448944+0000","last_fullsized":"2023-11-28T13:00:25.448944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:18.754380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458864+0000","last_change":"2023-11-28T13:00:21.422316+0000","last_active":"2023-11-28T13:00:25.458864+0000","last_peered":"2023-11-28T13:00:25.458864+0000","last_clean":"2023-11-28T13:00:25.458864+0000","last_became_active":"2023-11-28T13:00:21.422012+0000","last_became_peered":"2023-11-28T13:00:21.422012+0000","last_unstale":"2023-11-28T13:00:25.458864+0000","last_undegraded":"2023-11-28T13:00:25.458864+0000","last_fullsized":"2023-11-28T13:00:25.458864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442053+0000","last_change":"2023-11-28T13:00:21.423769+0000","last_active":"2023-11-28T13:00:25.442053+0000","last_peered":"2023-11-28T13:00:25.442053+0000","last_clean":"2023-11-28T13:00:25.442053+0000","last_became_active":"2023-11-28T13:00:21.423458+0000","last_became_peered":"2023-11-28T13:00:21.423458+0000","last_unstale":"2023-11-28T13:00:25.442053+0000","last_undegraded":"2023-11-28T13:00:25.442053+0000","last_fullsized":"2023-11-28T13:00:25.442053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:51.438322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441431+0000","last_change":"2023-11-28T13:00:21.432489+0000","last_active":"2023-11-28T13:00:25.441431+0000","last_peered":"2023-11-28T13:00:25.441431+0000","last_clean":"2023-11-28T13:00:25.441431+0000","last_became_active":"2023-11-28T13:00:21.432224+0000","last_became_peered":"2023-11-28T13:00:21.432224+0000","last_unstale":"2023-11-28T13:00:25.441431+0000","last_undegraded":"2023-11-28T13:00:25.441431+0000","last_fullsized":"2023-11-28T13:00:25.441431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:27.442626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460221+0000","last_change":"2023-11-28T13:00:21.427335+0000","last_active":"2023-11-28T13:00:25.460221+0000","last_peered":"2023-11-28T13:00:25.460221+0000","last_clean":"2023-11-28T13:00:25.460221+0000","last_became_active":"2023-11-28T13:00:21.426777+0000","last_became_peered":"2023-11-28T13:00:21.426777+0000","last_unstale":"2023-11-28T13:00:25.460221+0000","last_undegraded":"2023-11-28T13:00:25.460221+0000","last_fullsized":"2023-11-28T13:00:25.460221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:24.706933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441759+0000","last_change":"2023-11-28T13:00:21.436760+0000","last_active":"2023-11-28T13:00:25.441759+0000","last_peered":"2023-11-28T13:00:25.441759+0000","last_clean":"2023-11-28T13:00:25.441759+0000","last_became_active":"2023-11-28T13:00:21.436476+0000","last_became_peered":"2023-11-28T13:00:21.436476+0000","last_unstale":"2023-11-28T13:00:25.441759+0000","last_undegraded":"2023-11-28T13:00:25.441759+0000","last_fullsized":"2023-11-28T13:00:25.441759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:49.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455911+0000","last_change":"2023-11-28T13:00:21.415227+0000","last_active":"2023-11-28T13:00:25.455911+0000","last_peered":"2023-11-28T13:00:25.455911+0000","last_clean":"2023-11-28T13:00:25.455911+0000","last_became_active":"2023-11-28T13:00:21.414918+0000","last_became_peered":"2023-11-28T13:00:21.414918+0000","last_unstale":"2023-11-28T13:00:25.455911+0000","last_undegraded":"2023-11-28T13:00:25.455911+0000","last_fullsized":"2023-11-28T13:00:25.455911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:35:41.989664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4387,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":703951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9228,"kb_used_data":836,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362612,"statfs":{"total":96636764160,"available":96627314688,"internally_reserved":0,"allocated":856064,"data_stored":699125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8140,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363700,"statfs":{"total":96636764160,"available":96628428800,"internally_reserved":0,"allocated":266240,"data_stored":109342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8084,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363756,"statfs":{"total":96636764160,"available":96628486144,"internally_reserved":0,"allocated":274432,"data_stored":110398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:39.081 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:39.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.239+0000 7fe0fe59a700 1 Processor -- start 2023-11-28T13:00:39.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.239+0000 7fe0fe59a700 1 -- start start 2023-11-28T13:00:39.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0fe59a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0f815c990 0x7fe0f80cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0fe59a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe0f8161090 0x7fe0f80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0fe59a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f80ce340 con 0x7fe0f80caeb0 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0fe59a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f80cebe0 con 0x7fe0f8161090 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0fe59a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f80cb3f0 con 0x7fe0f815c990 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0f77fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe0f8161090 0x7fe0f80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0fcb37700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe0f80caeb0 0x7fe0f80cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45382/0 (socket says 172.21.15.6:45382) 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0fcb37700 1 -- 172.21.15.6:0/1584596469 learned_addr learned my addr 172.21.15.6:0/1584596469 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0f7fff700 1 --2- 172.21.15.6:0/1584596469 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0f815c990 0x7fe0f80cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0f77fe700 1 -- 172.21.15.6:0/1584596469 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0f815c990 msgr2=0x7fe0f80cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0f77fe700 1 --2- 172.21.15.6:0/1584596469 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0f815c990 0x7fe0f80cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0f77fe700 1 -- 172.21.15.6:0/1584596469 >> v1:172.21.15.6:6789/0 conn(0x7fe0f80caeb0 legacy=0x7fe0f80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0f6ffd700 1 -- 172.21.15.6:0/1584596469 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2440999659 0 0) 0x7fe0f80ce340 con 0x7fe0f80caeb0 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0f77fe700 1 -- 172.21.15.6:0/1584596469 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80ce590 con 0x7fe0f8161090 2023-11-28T13:00:39.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.240+0000 7fe0f77fe700 1 --2- 172.21.15.6:0/1584596469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f8161090 0x7fe0f80ca970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009840 tx=0x7fe0e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd64999139df2cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0f6ffd700 1 -- 172.21.15.6:0/1584596469 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0011830 con 0x7fe0f8161090 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0f6ffd700 1 -- 172.21.15.6:0/1584596469 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e0011990 con 0x7fe0f8161090 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0f6ffd700 1 -- 172.21.15.6:0/1584596469 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0011d20 con 0x7fe0f8161090 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0fe59a700 1 -- 172.21.15.6:0/1584596469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f8161090 msgr2=0x7fe0f80ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0fe59a700 1 --2- 172.21.15.6:0/1584596469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f8161090 0x7fe0f80ca970 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009840 tx=0x7fe0e0009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0fe59a700 1 -- 172.21.15.6:0/1584596469 shutdown_connections 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0fe59a700 1 --2- 172.21.15.6:0/1584596469 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0f815c990 0x7fe0f80cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0fe59a700 1 --2- 172.21.15.6:0/1584596469 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f8161090 0x7fe0f80ca970 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0fe59a700 1 -- 172.21.15.6:0/1584596469 >> 172.21.15.6:0/1584596469 conn(0x7fe0f80b1650 msgr2=0x7fe0f80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0fe59a700 1 -- 172.21.15.6:0/1584596469 shutdown_connections 2023-11-28T13:00:39.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.241+0000 7fe0fe59a700 1 -- 172.21.15.6:0/1584596469 wait complete. 2023-11-28T13:00:39.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.242+0000 7fe0fe59a700 1 Processor -- start 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.243+0000 7fe0fe59a700 1 -- start start 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.243+0000 7fe0fe59a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0f80caeb0 0x7fe0f815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.243+0000 7fe0fe59a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f815c990 0x7fe0f815e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.243+0000 7fe0fe59a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0f8161090 0x7fe0f80c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.243+0000 7fe0fe59a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f815ee30 con 0x7fe0f815c990 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.243+0000 7fe0fe59a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f815f110 con 0x7fe0f8161090 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.243+0000 7fe0fe59a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f815f3f0 con 0x7fe0f80caeb0 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f815c990 0x7fe0f815e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f815c990 0x7fe0f815e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58066/0 (socket says 172.21.15.6:58066) 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f77fe700 1 -- 172.21.15.6:0/469482296 learned_addr learned my addr 172.21.15.6:0/469482296 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f7fff700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0f80caeb0 0x7fe0f815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0fcb37700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0f8161090 0x7fe0f80c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f77fe700 1 -- 172.21.15.6:0/469482296 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0f80caeb0 msgr2=0x7fe0f815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f77fe700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0f80caeb0 0x7fe0f815ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f77fe700 1 -- 172.21.15.6:0/469482296 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0f8161090 msgr2=0x7fe0f80c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f77fe700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0f8161090 0x7fe0f80c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f77fe700 1 -- 172.21.15.6:0/469482296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e00094a0 con 0x7fe0f815c990 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f77fe700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f815c990 0x7fe0f815e3e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0000c00 tx=0x7fe0e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f4ff9700 1 -- 172.21.15.6:0/469482296 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00045d0 con 0x7fe0f815c990 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f4ff9700 1 -- 172.21.15.6:0/469482296 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e001f070 con 0x7fe0f815c990 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0fe59a700 1 -- 172.21.15.6:0/469482296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f80cf5b0 con 0x7fe0f815c990 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0f4ff9700 1 -- 172.21.15.6:0/469482296 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00038c0 con 0x7fe0f815c990 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.244+0000 7fe0fe59a700 1 -- 172.21.15.6:0/469482296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f80ce870 con 0x7fe0f815c990 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.245+0000 7fe0fe59a700 1 -- 172.21.15.6:0/469482296 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f80cf740 con 0x7fe0f815c990 2023-11-28T13:00:39.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.246+0000 7fe0f4ff9700 1 -- 172.21.15.6:0/469482296 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe0e000f050 con 0x7fe0f815c990 2023-11-28T13:00:39.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.246+0000 7fe0f4ff9700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8068320 0x7fe0d806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.246+0000 7fe0f4ff9700 1 -- 172.21.15.6:0/469482296 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe0e0021070 con 0x7fe0f815c990 2023-11-28T13:00:39.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.246+0000 7fe0f7fff700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8068320 0x7fe0d806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.246+0000 7fe0f7fff700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8068320 0x7fe0d806a7d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8006b20 tx=0x7fe0e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:39.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.250+0000 7fe0f4ff9700 1 -- 172.21.15.6:0/469482296 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe0e0053d80 con 0x7fe0f815c990 2023-11-28T13:00:39.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.441+0000 7fe0fe59a700 1 -- 172.21.15.6:0/469482296 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0f80ced50 con 0x7fe0d8068320 2023-11-28T13:00:39.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.451+0000 7fe0f4ff9700 1 -- 172.21.15.6:0/469482296 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367539 (secure 0 0 0) 0x7fe0f80ced50 con 0x7fe0d8068320 2023-11-28T13:00:39.457 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:39.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.454+0000 7fe0fe59a700 1 -- 172.21.15.6:0/469482296 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8068320 msgr2=0x7fe0d806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.454+0000 7fe0fe59a700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8068320 0x7fe0d806a7d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8006b20 tx=0x7fe0e8008040 comp rx=0 tx=0).stop 2023-11-28T13:00:39.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.454+0000 7fe0fe59a700 1 -- 172.21.15.6:0/469482296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f815c990 msgr2=0x7fe0f815e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.454+0000 7fe0fe59a700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f815c990 0x7fe0f815e3e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0000c00 tx=0x7fe0e0004750 comp rx=0 tx=0).stop 2023-11-28T13:00:39.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.454+0000 7fe0fe59a700 1 -- 172.21.15.6:0/469482296 shutdown_connections 2023-11-28T13:00:39.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.454+0000 7fe0fe59a700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0f8161090 0x7fe0f80c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.454+0000 7fe0fe59a700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8068320 0x7fe0d806a7d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.454+0000 7fe0fe59a700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0f80caeb0 0x7fe0f815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.454+0000 7fe0fe59a700 1 --2- 172.21.15.6:0/469482296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0f815c990 0x7fe0f815e3e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.454+0000 7fe0fe59a700 1 -- 172.21.15.6:0/469482296 >> 172.21.15.6:0/469482296 conn(0x7fe0f80b1650 msgr2=0x7fe0f806bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:39.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.455+0000 7fe0fe59a700 1 -- 172.21.15.6:0/469482296 shutdown_connections 2023-11-28T13:00:39.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.455+0000 7fe0fe59a700 1 -- 172.21.15.6:0/469482296 wait complete. 2023-11-28T13:00:39.461 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:00:39.468 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-11-28T13:00:38.946157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594755,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67232,"kb_used_data":3296,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907488,"statfs":{"total":773094113280,"available":773025267712,"internally_reserved":0,"allocated":3375104,"data_stored":2065615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2057,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002619"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.456044+0000","last_change":"2023-11-28T13:00:21.430642+0000","last_active":"2023-11-28T13:00:25.456044+0000","last_peered":"2023-11-28T13:00:25.456044+0000","last_clean":"2023-11-28T13:00:25.456044+0000","last_became_active":"2023-11-28T13:00:21.430357+0000","last_became_peered":"2023-11-28T13:00:21.430357+0000","last_unstale":"2023-11-28T13:00:25.456044+0000","last_undegraded":"2023-11-28T13:00:25.456044+0000","last_fullsized":"2023-11-28T13:00:25.456044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:43.433683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452930+0000","last_change":"2023-11-28T13:00:21.432299+0000","last_active":"2023-11-28T13:00:25.452930+0000","last_peered":"2023-11-28T13:00:25.452930+0000","last_clean":"2023-11-28T13:00:25.452930+0000","last_became_active":"2023-11-28T13:00:21.431566+0000","last_became_peered":"2023-11-28T13:00:21.431566+0000","last_unstale":"2023-11-28T13:00:25.452930+0000","last_undegraded":"2023-11-28T13:00:25.452930+0000","last_fullsized":"2023-11-28T13:00:25.452930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:24.765270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.461121+0000","last_change":"2023-11-28T13:00:21.430850+0000","last_active":"2023-11-28T13:00:25.461121+0000","last_peered":"2023-11-28T13:00:25.461121+0000","last_clean":"2023-11-28T13:00:25.461121+0000","last_became_active":"2023-11-28T13:00:21.430557+0000","last_became_peered":"2023-11-28T13:00:21.430557+0000","last_unstale":"2023-11-28T13:00:25.461121+0000","last_undegraded":"2023-11-28T13:00:25.461121+0000","last_fullsized":"2023-11-28T13:00:25.461121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:00.080768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442074+0000","last_change":"2023-11-28T13:00:21.437362+0000","last_active":"2023-11-28T13:00:25.442074+0000","last_peered":"2023-11-28T13:00:25.442074+0000","last_clean":"2023-11-28T13:00:25.442074+0000","last_became_active":"2023-11-28T13:00:21.435046+0000","last_became_peered":"2023-11-28T13:00:21.435046+0000","last_unstale":"2023-11-28T13:00:25.442074+0000","last_undegraded":"2023-11-28T13:00:25.442074+0000","last_fullsized":"2023-11-28T13:00:25.442074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:39.731416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448772+0000","last_change":"2023-11-28T13:00:21.436404+0000","last_active":"2023-11-28T13:00:25.448772+0000","last_peered":"2023-11-28T13:00:25.448772+0000","last_clean":"2023-11-28T13:00:25.448772+0000","last_became_active":"2023-11-28T13:00:21.436158+0000","last_became_peered":"2023-11-28T13:00:21.436158+0000","last_unstale":"2023-11-28T13:00:25.448772+0000","last_undegraded":"2023-11-28T13:00:25.448772+0000","last_fullsized":"2023-11-28T13:00:25.448772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:42.706331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460337+0000","last_change":"2023-11-28T13:00:21.431231+0000","last_active":"2023-11-28T13:00:25.460337+0000","last_peered":"2023-11-28T13:00:25.460337+0000","last_clean":"2023-11-28T13:00:25.460337+0000","last_became_active":"2023-11-28T13:00:21.430602+0000","last_became_peered":"2023-11-28T13:00:21.430602+0000","last_unstale":"2023-11-28T13:00:25.460337+0000","last_undegraded":"2023-11-28T13:00:25.460337+0000","last_fullsized":"2023-11-28T13:00:25.460337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:13.072237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443249+0000","last_change":"2023-11-28T13:00:21.419513+0000","last_active":"2023-11-28T13:00:25.443249+0000","last_peered":"2023-11-28T13:00:25.443249+0000","last_clean":"2023-11-28T13:00:25.443249+0000","last_became_active":"2023-11-28T13:00:21.418668+0000","last_became_peered":"2023-11-28T13:00:21.418668+0000","last_unstale":"2023-11-28T13:00:25.443249+0000","last_undegraded":"2023-11-28T13:00:25.443249+0000","last_fullsized":"2023-11-28T13:00:25.443249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448590+0000","last_change":"2023-11-28T13:00:21.432362+0000","last_active":"2023-11-28T13:00:25.448590+0000","last_peered":"2023-11-28T13:00:25.448590+0000","last_clean":"2023-11-28T13:00:25.448590+0000","last_became_active":"2023-11-28T13:00:21.432056+0000","last_became_peered":"2023-11-28T13:00:21.432056+0000","last_unstale":"2023-11-28T13:00:25.448590+0000","last_undegraded":"2023-11-28T13:00:25.448590+0000","last_fullsized":"2023-11-28T13:00:25.448590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:13.039552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448588+0000","last_change":"2023-11-28T13:00:21.434750+0000","last_active":"2023-11-28T13:00:25.448588+0000","last_peered":"2023-11-28T13:00:25.448588+0000","last_clean":"2023-11-28T13:00:25.448588+0000","last_became_active":"2023-11-28T13:00:21.434060+0000","last_became_peered":"2023-11-28T13:00:21.434060+0000","last_unstale":"2023-11-28T13:00:25.448588+0000","last_undegraded":"2023-11-28T13:00:25.448588+0000","last_fullsized":"2023-11-28T13:00:25.448588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:34.468496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.453105+0000","last_change":"2023-11-28T13:00:21.433386+0000","last_active":"2023-11-28T13:00:25.453105+0000","last_peered":"2023-11-28T13:00:25.453105+0000","last_clean":"2023-11-28T13:00:25.453105+0000","last_became_active":"2023-11-28T13:00:21.432005+0000","last_became_peered":"2023-11-28T13:00:21.432005+0000","last_unstale":"2023-11-28T13:00:25.453105+0000","last_undegraded":"2023-11-28T13:00:25.453105+0000","last_fullsized":"2023-11-28T13:00:25.453105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:11.589800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.461025+0000","last_change":"2023-11-28T13:00:21.427275+0000","last_active":"2023-11-28T13:00:25.461025+0000","last_peered":"2023-11-28T13:00:25.461025+0000","last_clean":"2023-11-28T13:00:25.461025+0000","last_became_active":"2023-11-28T13:00:21.426893+0000","last_became_peered":"2023-11-28T13:00:21.426893+0000","last_unstale":"2023-11-28T13:00:25.461025+0000","last_undegraded":"2023-11-28T13:00:25.461025+0000","last_fullsized":"2023-11-28T13:00:25.461025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:24.765270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455341+0000","last_change":"2023-11-28T13:00:21.436650+0000","last_active":"2023-11-28T13:00:25.455341+0000","last_peered":"2023-11-28T13:00:25.455341+0000","last_clean":"2023-11-28T13:00:25.455341+0000","last_became_active":"2023-11-28T13:00:21.435700+0000","last_became_peered":"2023-11-28T13:00:21.435700+0000","last_unstale":"2023-11-28T13:00:25.455341+0000","last_undegraded":"2023-11-28T13:00:25.455341+0000","last_fullsized":"2023-11-28T13:00:25.455341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:02.740267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.454713+0000","last_change":"2023-11-28T13:00:21.434352+0000","last_active":"2023-11-28T13:00:25.454713+0000","last_peered":"2023-11-28T13:00:25.454713+0000","last_clean":"2023-11-28T13:00:25.454713+0000","last_became_active":"2023-11-28T13:00:21.434067+0000","last_became_peered":"2023-11-28T13:00:21.434067+0000","last_unstale":"2023-11-28T13:00:25.454713+0000","last_undegraded":"2023-11-28T13:00:25.454713+0000","last_fullsized":"2023-11-28T13:00:25.454713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:30.629627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449953+0000","last_change":"2023-11-28T13:00:21.431774+0000","last_active":"2023-11-28T13:00:25.449953+0000","last_peered":"2023-11-28T13:00:25.449953+0000","last_clean":"2023-11-28T13:00:25.449953+0000","last_became_active":"2023-11-28T13:00:21.431445+0000","last_became_peered":"2023-11-28T13:00:21.431445+0000","last_unstale":"2023-11-28T13:00:25.449953+0000","last_undegraded":"2023-11-28T13:00:25.449953+0000","last_fullsized":"2023-11-28T13:00:25.449953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:24.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443018+0000","last_change":"2023-11-28T13:00:21.420404+0000","last_active":"2023-11-28T13:00:25.443018+0000","last_peered":"2023-11-28T13:00:25.443018+0000","last_clean":"2023-11-28T13:00:25.443018+0000","last_became_active":"2023-11-28T13:00:21.419228+0000","last_became_peered":"2023-11-28T13:00:21.419228+0000","last_unstale":"2023-11-28T13:00:25.443018+0000","last_undegraded":"2023-11-28T13:00:25.443018+0000","last_fullsized":"2023-11-28T13:00:25.443018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:40:24.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458890+0000","last_change":"2023-11-28T13:00:21.430943+0000","last_active":"2023-11-28T13:00:25.458890+0000","last_peered":"2023-11-28T13:00:25.458890+0000","last_clean":"2023-11-28T13:00:25.458890+0000","last_became_active":"2023-11-28T13:00:21.430664+0000","last_became_peered":"2023-11-28T13:00:21.430664+0000","last_unstale":"2023-11-28T13:00:25.458890+0000","last_undegraded":"2023-11-28T13:00:25.458890+0000","last_fullsized":"2023-11-28T13:00:25.458890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:12:14.186445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448670+0000","last_change":"2023-11-28T13:00:21.425335+0000","last_active":"2023-11-28T13:00:25.448670+0000","last_peered":"2023-11-28T13:00:25.448670+0000","last_clean":"2023-11-28T13:00:25.448670+0000","last_became_active":"2023-11-28T13:00:21.425073+0000","last_became_peered":"2023-11-28T13:00:21.425073+0000","last_unstale":"2023-11-28T13:00:25.448670+0000","last_undegraded":"2023-11-28T13:00:25.448670+0000","last_fullsized":"2023-11-28T13:00:25.448670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:45.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460831+0000","last_change":"2023-11-28T13:00:21.426161+0000","last_active":"2023-11-28T13:00:25.460831+0000","last_peered":"2023-11-28T13:00:25.460831+0000","last_clean":"2023-11-28T13:00:25.460831+0000","last_became_active":"2023-11-28T13:00:21.425796+0000","last_became_peered":"2023-11-28T13:00:21.425796+0000","last_unstale":"2023-11-28T13:00:25.460831+0000","last_undegraded":"2023-11-28T13:00:25.460831+0000","last_fullsized":"2023-11-28T13:00:25.460831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:17.422401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455990+0000","last_change":"2023-11-28T13:00:21.423565+0000","last_active":"2023-11-28T13:00:25.455990+0000","last_peered":"2023-11-28T13:00:25.455990+0000","last_clean":"2023-11-28T13:00:25.455990+0000","last_became_active":"2023-11-28T13:00:21.423270+0000","last_became_peered":"2023-11-28T13:00:21.423270+0000","last_unstale":"2023-11-28T13:00:25.455990+0000","last_undegraded":"2023-11-28T13:00:25.455990+0000","last_fullsized":"2023-11-28T13:00:25.455990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:31.139532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460193+0000","last_change":"2023-11-28T13:00:21.426452+0000","last_active":"2023-11-28T13:00:25.460193+0000","last_peered":"2023-11-28T13:00:25.460193+0000","last_clean":"2023-11-28T13:00:25.460193+0000","last_became_active":"2023-11-28T13:00:21.426096+0000","last_became_peered":"2023-11-28T13:00:21.426096+0000","last_unstale":"2023-11-28T13:00:25.460193+0000","last_undegraded":"2023-11-28T13:00:25.460193+0000","last_fullsized":"2023-11-28T13:00:25.460193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:33.439707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443668+0000","last_change":"2023-11-28T13:00:21.435483+0000","last_active":"2023-11-28T13:00:25.443668+0000","last_peered":"2023-11-28T13:00:25.443668+0000","last_clean":"2023-11-28T13:00:25.443668+0000","last_became_active":"2023-11-28T13:00:21.435162+0000","last_became_peered":"2023-11-28T13:00:21.435162+0000","last_unstale":"2023-11-28T13:00:25.443668+0000","last_undegraded":"2023-11-28T13:00:25.443668+0000","last_fullsized":"2023-11-28T13:00:25.443668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:13.039552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448596+0000","last_change":"2023-11-28T13:00:21.417958+0000","last_active":"2023-11-28T13:00:25.448596+0000","last_peered":"2023-11-28T13:00:25.448596+0000","last_clean":"2023-11-28T13:00:25.448596+0000","last_became_active":"2023-11-28T13:00:21.417664+0000","last_became_peered":"2023-11-28T13:00:21.417664+0000","last_unstale":"2023-11-28T13:00:25.448596+0000","last_undegraded":"2023-11-28T13:00:25.448596+0000","last_fullsized":"2023-11-28T13:00:25.448596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:35.062024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441395+0000","last_change":"2023-11-28T13:00:21.432315+0000","last_active":"2023-11-28T13:00:25.441395+0000","last_peered":"2023-11-28T13:00:25.441395+0000","last_clean":"2023-11-28T13:00:25.441395+0000","last_became_active":"2023-11-28T13:00:21.431878+0000","last_became_peered":"2023-11-28T13:00:21.431878+0000","last_unstale":"2023-11-28T13:00:25.441395+0000","last_undegraded":"2023-11-28T13:00:25.441395+0000","last_fullsized":"2023-11-28T13:00:25.441395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:26:43.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446220+0000","last_change":"2023-11-28T13:00:21.429436+0000","last_active":"2023-11-28T13:00:25.446220+0000","last_peered":"2023-11-28T13:00:25.446220+0000","last_clean":"2023-11-28T13:00:25.446220+0000","last_became_active":"2023-11-28T13:00:21.429051+0000","last_became_peered":"2023-11-28T13:00:21.429051+0000","last_unstale":"2023-11-28T13:00:25.446220+0000","last_undegraded":"2023-11-28T13:00:25.446220+0000","last_fullsized":"2023-11-28T13:00:25.446220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:11.589800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458565+0000","last_change":"2023-11-28T13:00:21.423512+0000","last_active":"2023-11-28T13:00:25.458565+0000","last_peered":"2023-11-28T13:00:25.458565+0000","last_clean":"2023-11-28T13:00:25.458565+0000","last_became_active":"2023-11-28T13:00:21.423223+0000","last_became_peered":"2023-11-28T13:00:21.423223+0000","last_unstale":"2023-11-28T13:00:25.458565+0000","last_undegraded":"2023-11-28T13:00:25.458565+0000","last_fullsized":"2023-11-28T13:00:25.458565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:35.062024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457987+0000","last_change":"2023-11-28T13:00:21.426702+0000","last_active":"2023-11-28T13:00:25.457987+0000","last_peered":"2023-11-28T13:00:25.457987+0000","last_clean":"2023-11-28T13:00:25.457987+0000","last_became_active":"2023-11-28T13:00:21.425807+0000","last_became_peered":"2023-11-28T13:00:21.425807+0000","last_unstale":"2023-11-28T13:00:25.457987+0000","last_undegraded":"2023-11-28T13:00:25.457987+0000","last_fullsized":"2023-11-28T13:00:25.457987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:04.077060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452451+0000","last_change":"2023-11-28T13:00:21.434848+0000","last_active":"2023-11-28T13:00:25.452451+0000","last_peered":"2023-11-28T13:00:25.452451+0000","last_clean":"2023-11-28T13:00:25.452451+0000","last_became_active":"2023-11-28T13:00:21.432193+0000","last_became_peered":"2023-11-28T13:00:21.432193+0000","last_unstale":"2023-11-28T13:00:25.452451+0000","last_undegraded":"2023-11-28T13:00:25.452451+0000","last_fullsized":"2023-11-28T13:00:25.452451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:43.321870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871441+0000","last_change":"2023-11-28T13:00:25.470736+0000","last_active":"2023-11-28T13:00:25.871441+0000","last_peered":"2023-11-28T13:00:25.871441+0000","last_clean":"2023-11-28T13:00:25.871441+0000","last_became_active":"2023-11-28T13:00:25.470009+0000","last_became_peered":"2023-11-28T13:00:25.470009+0000","last_unstale":"2023-11-28T13:00:25.871441+0000","last_undegraded":"2023-11-28T13:00:25.871441+0000","last_fullsized":"2023-11-28T13:00:25.871441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:05.706754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459220+0000","last_change":"2023-11-28T13:00:21.006957+0000","last_active":"2023-11-28T13:00:25.459220+0000","last_peered":"2023-11-28T13:00:25.459220+0000","last_clean":"2023-11-28T13:00:25.459220+0000","last_became_active":"2023-11-28T13:00:21.006690+0000","last_became_peered":"2023-11-28T13:00:21.006690+0000","last_unstale":"2023-11-28T13:00:25.459220+0000","last_undegraded":"2023-11-28T13:00:25.459220+0000","last_fullsized":"2023-11-28T13:00:25.459220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:30.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441650+0000","last_change":"2023-11-28T13:00:21.430560+0000","last_active":"2023-11-28T13:00:25.441650+0000","last_peered":"2023-11-28T13:00:25.441650+0000","last_clean":"2023-11-28T13:00:25.441650+0000","last_became_active":"2023-11-28T13:00:21.430312+0000","last_became_peered":"2023-11-28T13:00:21.430312+0000","last_unstale":"2023-11-28T13:00:25.441650+0000","last_undegraded":"2023-11-28T13:00:25.441650+0000","last_fullsized":"2023-11-28T13:00:25.441650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347196+0000","last_change":"2023-11-28T13:00:25.460012+0000","last_active":"2023-11-28T13:00:26.347196+0000","last_peered":"2023-11-28T13:00:26.347196+0000","last_clean":"2023-11-28T13:00:26.347196+0000","last_became_active":"2023-11-28T13:00:25.459532+0000","last_became_peered":"2023-11-28T13:00:25.459532+0000","last_unstale":"2023-11-28T13:00:26.347196+0000","last_undegraded":"2023-11-28T13:00:26.347196+0000","last_fullsized":"2023-11-28T13:00:26.347196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:36.890229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448628+0000","last_change":"2023-11-28T13:00:21.005058+0000","last_active":"2023-11-28T13:00:25.448628+0000","last_peered":"2023-11-28T13:00:25.448628+0000","last_clean":"2023-11-28T13:00:25.448628+0000","last_became_active":"2023-11-28T13:00:21.004749+0000","last_became_peered":"2023-11-28T13:00:21.004749+0000","last_unstale":"2023-11-28T13:00:25.448628+0000","last_undegraded":"2023-11-28T13:00:25.448628+0000","last_fullsized":"2023-11-28T13:00:25.448628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:22.962275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448569+0000","last_change":"2023-11-28T13:00:21.416645+0000","last_active":"2023-11-28T13:00:25.448569+0000","last_peered":"2023-11-28T13:00:25.448569+0000","last_clean":"2023-11-28T13:00:25.448569+0000","last_became_active":"2023-11-28T13:00:21.416365+0000","last_became_peered":"2023-11-28T13:00:21.416365+0000","last_unstale":"2023-11-28T13:00:25.448569+0000","last_undegraded":"2023-11-28T13:00:25.448569+0000","last_fullsized":"2023-11-28T13:00:25.448569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:14.429821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073812+0000","last_change":"2023-11-28T13:00:25.468574+0000","last_active":"2023-11-28T13:00:26.073812+0000","last_peered":"2023-11-28T13:00:26.073812+0000","last_clean":"2023-11-28T13:00:26.073812+0000","last_became_active":"2023-11-28T13:00:25.468177+0000","last_became_peered":"2023-11-28T13:00:25.468177+0000","last_unstale":"2023-11-28T13:00:26.073812+0000","last_undegraded":"2023-11-28T13:00:26.073812+0000","last_fullsized":"2023-11-28T13:00:26.073812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:13:55.892226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441558+0000","last_change":"2023-11-28T13:00:21.005438+0000","last_active":"2023-11-28T13:00:25.441558+0000","last_peered":"2023-11-28T13:00:25.441558+0000","last_clean":"2023-11-28T13:00:25.441558+0000","last_became_active":"2023-11-28T13:00:21.004885+0000","last_became_peered":"2023-11-28T13:00:21.004885+0000","last_unstale":"2023-11-28T13:00:25.441558+0000","last_undegraded":"2023-11-28T13:00:25.441558+0000","last_fullsized":"2023-11-28T13:00:25.441558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:47.845351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442760+0000","last_change":"2023-11-28T13:00:21.437422+0000","last_active":"2023-11-28T13:00:25.442760+0000","last_peered":"2023-11-28T13:00:25.442760+0000","last_clean":"2023-11-28T13:00:25.442760+0000","last_became_active":"2023-11-28T13:00:21.437095+0000","last_became_peered":"2023-11-28T13:00:21.437095+0000","last_unstale":"2023-11-28T13:00:25.442760+0000","last_undegraded":"2023-11-28T13:00:25.442760+0000","last_fullsized":"2023-11-28T13:00:25.442760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:52.972220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346252+0000","last_change":"2023-11-28T13:00:25.468548+0000","last_active":"2023-11-28T13:00:26.346252+0000","last_peered":"2023-11-28T13:00:26.346252+0000","last_clean":"2023-11-28T13:00:26.346252+0000","last_became_active":"2023-11-28T13:00:25.468071+0000","last_became_peered":"2023-11-28T13:00:25.468071+0000","last_unstale":"2023-11-28T13:00:26.346252+0000","last_undegraded":"2023-11-28T13:00:26.346252+0000","last_fullsized":"2023-11-28T13:00:26.346252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:20.872923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448774+0000","last_change":"2023-11-28T13:00:21.003619+0000","last_active":"2023-11-28T13:00:25.448774+0000","last_peered":"2023-11-28T13:00:25.448774+0000","last_clean":"2023-11-28T13:00:25.448774+0000","last_became_active":"2023-11-28T13:00:21.003189+0000","last_became_peered":"2023-11-28T13:00:21.003189+0000","last_unstale":"2023-11-28T13:00:25.448774+0000","last_undegraded":"2023-11-28T13:00:25.448774+0000","last_fullsized":"2023-11-28T13:00:25.448774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:53.009716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441443+0000","last_change":"2023-11-28T13:00:21.428000+0000","last_active":"2023-11-28T13:00:25.441443+0000","last_peered":"2023-11-28T13:00:25.441443+0000","last_clean":"2023-11-28T13:00:25.441443+0000","last_became_active":"2023-11-28T13:00:21.427738+0000","last_became_peered":"2023-11-28T13:00:21.427738+0000","last_unstale":"2023-11-28T13:00:25.441443+0000","last_undegraded":"2023-11-28T13:00:25.441443+0000","last_fullsized":"2023-11-28T13:00:25.441443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:24.529486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871318+0000","last_change":"2023-11-28T13:00:25.463375+0000","last_active":"2023-11-28T13:00:25.871318+0000","last_peered":"2023-11-28T13:00:25.871318+0000","last_clean":"2023-11-28T13:00:25.871318+0000","last_became_active":"2023-11-28T13:00:25.462962+0000","last_became_peered":"2023-11-28T13:00:25.462962+0000","last_unstale":"2023-11-28T13:00:25.871318+0000","last_undegraded":"2023-11-28T13:00:25.871318+0000","last_fullsized":"2023-11-28T13:00:25.871318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:16.522760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452987+0000","last_change":"2023-11-28T13:00:21.011149+0000","last_active":"2023-11-28T13:00:25.452987+0000","last_peered":"2023-11-28T13:00:25.452987+0000","last_clean":"2023-11-28T13:00:25.452987+0000","last_became_active":"2023-11-28T13:00:21.010813+0000","last_became_peered":"2023-11-28T13:00:21.010813+0000","last_unstale":"2023-11-28T13:00:25.452987+0000","last_undegraded":"2023-11-28T13:00:25.452987+0000","last_fullsized":"2023-11-28T13:00:25.452987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441165+0000","last_change":"2023-11-28T13:00:21.428947+0000","last_active":"2023-11-28T13:00:25.441165+0000","last_peered":"2023-11-28T13:00:25.441165+0000","last_clean":"2023-11-28T13:00:25.441165+0000","last_became_active":"2023-11-28T13:00:21.428690+0000","last_became_peered":"2023-11-28T13:00:21.428690+0000","last_unstale":"2023-11-28T13:00:25.441165+0000","last_undegraded":"2023-11-28T13:00:25.441165+0000","last_fullsized":"2023-11-28T13:00:25.441165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:55:41.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869911+0000","last_change":"2023-11-28T13:00:25.468634+0000","last_active":"2023-11-28T13:00:25.869911+0000","last_peered":"2023-11-28T13:00:25.869911+0000","last_clean":"2023-11-28T13:00:25.869911+0000","last_became_active":"2023-11-28T13:00:25.468224+0000","last_became_peered":"2023-11-28T13:00:25.468224+0000","last_unstale":"2023-11-28T13:00:25.869911+0000","last_undegraded":"2023-11-28T13:00:25.869911+0000","last_fullsized":"2023-11-28T13:00:25.869911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:21.591804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446244+0000","last_change":"2023-11-28T13:00:21.006053+0000","last_active":"2023-11-28T13:00:25.446244+0000","last_peered":"2023-11-28T13:00:25.446244+0000","last_clean":"2023-11-28T13:00:25.446244+0000","last_became_active":"2023-11-28T13:00:21.005693+0000","last_became_peered":"2023-11-28T13:00:21.005693+0000","last_unstale":"2023-11-28T13:00:25.446244+0000","last_undegraded":"2023-11-28T13:00:25.446244+0000","last_fullsized":"2023-11-28T13:00:25.446244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:38.720815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457494+0000","last_change":"2023-11-28T13:00:21.426158+0000","last_active":"2023-11-28T13:00:25.457494+0000","last_peered":"2023-11-28T13:00:25.457494+0000","last_clean":"2023-11-28T13:00:25.457494+0000","last_became_active":"2023-11-28T13:00:21.425771+0000","last_became_peered":"2023-11-28T13:00:21.425771+0000","last_unstale":"2023-11-28T13:00:25.457494+0000","last_undegraded":"2023-11-28T13:00:25.457494+0000","last_fullsized":"2023-11-28T13:00:25.457494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:48.996841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076004+0000","last_change":"2023-11-28T13:00:25.467575+0000","last_active":"2023-11-28T13:00:26.076004+0000","last_peered":"2023-11-28T13:00:26.076004+0000","last_clean":"2023-11-28T13:00:26.076004+0000","last_became_active":"2023-11-28T13:00:25.466169+0000","last_became_peered":"2023-11-28T13:00:25.466169+0000","last_unstale":"2023-11-28T13:00:26.076004+0000","last_undegraded":"2023-11-28T13:00:26.076004+0000","last_fullsized":"2023-11-28T13:00:26.076004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:57.102759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:29.849985+0000","last_change":"2023-11-28T13:00:21.000813+0000","last_active":"2023-11-28T13:00:29.849985+0000","last_peered":"2023-11-28T13:00:29.849985+0000","last_clean":"2023-11-28T13:00:29.849985+0000","last_became_active":"2023-11-28T13:00:20.999079+0000","last_became_peered":"2023-11-28T13:00:20.999079+0000","last_unstale":"2023-11-28T13:00:29.849985+0000","last_undegraded":"2023-11-28T13:00:29.849985+0000","last_fullsized":"2023-11-28T13:00:29.849985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:04.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3605,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460800+0000","last_change":"2023-11-28T13:00:21.422098+0000","last_active":"2023-11-28T13:00:25.460800+0000","last_peered":"2023-11-28T13:00:25.460800+0000","last_clean":"2023-11-28T13:00:25.460800+0000","last_became_active":"2023-11-28T13:00:21.421774+0000","last_became_peered":"2023-11-28T13:00:21.421774+0000","last_unstale":"2023-11-28T13:00:25.460800+0000","last_undegraded":"2023-11-28T13:00:25.460800+0000","last_fullsized":"2023-11-28T13:00:25.460800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:08.151957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347474+0000","last_change":"2023-11-28T13:00:25.468792+0000","last_active":"2023-11-28T13:00:26.347474+0000","last_peered":"2023-11-28T13:00:26.347474+0000","last_clean":"2023-11-28T13:00:26.347474+0000","last_became_active":"2023-11-28T13:00:25.468376+0000","last_became_peered":"2023-11-28T13:00:25.468376+0000","last_unstale":"2023-11-28T13:00:26.347474+0000","last_undegraded":"2023-11-28T13:00:26.347474+0000","last_fullsized":"2023-11-28T13:00:26.347474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:51.024234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443541+0000","last_change":"2023-11-28T13:00:21.003466+0000","last_active":"2023-11-28T13:00:25.443541+0000","last_peered":"2023-11-28T13:00:25.443541+0000","last_clean":"2023-11-28T13:00:25.443541+0000","last_became_active":"2023-11-28T13:00:21.003127+0000","last_became_peered":"2023-11-28T13:00:21.003127+0000","last_unstale":"2023-11-28T13:00:25.443541+0000","last_undegraded":"2023-11-28T13:00:25.443541+0000","last_fullsized":"2023-11-28T13:00:25.443541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441650+0000","last_change":"2023-11-28T13:00:21.429933+0000","last_active":"2023-11-28T13:00:25.441650+0000","last_peered":"2023-11-28T13:00:25.441650+0000","last_clean":"2023-11-28T13:00:25.441650+0000","last_became_active":"2023-11-28T13:00:21.429681+0000","last_became_peered":"2023-11-28T13:00:21.429681+0000","last_unstale":"2023-11-28T13:00:25.441650+0000","last_undegraded":"2023-11-28T13:00:25.441650+0000","last_fullsized":"2023-11-28T13:00:25.441650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:45.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869412+0000","last_change":"2023-11-28T13:00:25.468518+0000","last_active":"2023-11-28T13:00:25.869412+0000","last_peered":"2023-11-28T13:00:25.869412+0000","last_clean":"2023-11-28T13:00:25.869412+0000","last_became_active":"2023-11-28T13:00:25.468083+0000","last_became_peered":"2023-11-28T13:00:25.468083+0000","last_unstale":"2023-11-28T13:00:25.869412+0000","last_undegraded":"2023-11-28T13:00:25.869412+0000","last_fullsized":"2023-11-28T13:00:25.869412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:53:56.156971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448763+0000","last_change":"2023-11-28T13:00:20.997868+0000","last_active":"2023-11-28T13:00:25.448763+0000","last_peered":"2023-11-28T13:00:25.448763+0000","last_clean":"2023-11-28T13:00:25.448763+0000","last_became_active":"2023-11-28T13:00:20.997565+0000","last_became_peered":"2023-11-28T13:00:20.997565+0000","last_unstale":"2023-11-28T13:00:25.448763+0000","last_undegraded":"2023-11-28T13:00:25.448763+0000","last_fullsized":"2023-11-28T13:00:25.448763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:37.465748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.451059+0000","last_change":"2023-11-28T13:00:21.431844+0000","last_active":"2023-11-28T13:00:25.451059+0000","last_peered":"2023-11-28T13:00:25.451059+0000","last_clean":"2023-11-28T13:00:25.451059+0000","last_became_active":"2023-11-28T13:00:21.431496+0000","last_became_peered":"2023-11-28T13:00:21.431496+0000","last_unstale":"2023-11-28T13:00:25.451059+0000","last_undegraded":"2023-11-28T13:00:25.451059+0000","last_fullsized":"2023-11-28T13:00:25.451059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.154908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869150+0000","last_change":"2023-11-28T13:00:25.469324+0000","last_active":"2023-11-28T13:00:25.869150+0000","last_peered":"2023-11-28T13:00:25.869150+0000","last_clean":"2023-11-28T13:00:25.869150+0000","last_became_active":"2023-11-28T13:00:25.468890+0000","last_became_peered":"2023-11-28T13:00:25.468890+0000","last_unstale":"2023-11-28T13:00:25.869150+0000","last_undegraded":"2023-11-28T13:00:25.869150+0000","last_fullsized":"2023-11-28T13:00:25.869150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:41.487196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443130+0000","last_change":"2023-11-28T13:00:21.002295+0000","last_active":"2023-11-28T13:00:25.443130+0000","last_peered":"2023-11-28T13:00:25.443130+0000","last_clean":"2023-11-28T13:00:25.443130+0000","last_became_active":"2023-11-28T13:00:21.001997+0000","last_became_peered":"2023-11-28T13:00:21.001997+0000","last_unstale":"2023-11-28T13:00:25.443130+0000","last_undegraded":"2023-11-28T13:00:25.443130+0000","last_fullsized":"2023-11-28T13:00:25.443130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449818+0000","last_change":"2023-11-28T13:00:21.431066+0000","last_active":"2023-11-28T13:00:25.449818+0000","last_peered":"2023-11-28T13:00:25.449818+0000","last_clean":"2023-11-28T13:00:25.449818+0000","last_became_active":"2023-11-28T13:00:21.430741+0000","last_became_peered":"2023-11-28T13:00:21.430741+0000","last_unstale":"2023-11-28T13:00:25.449818+0000","last_undegraded":"2023-11-28T13:00:25.449818+0000","last_fullsized":"2023-11-28T13:00:25.449818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:55.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347017+0000","last_change":"2023-11-28T13:00:25.468987+0000","last_active":"2023-11-28T13:00:26.347017+0000","last_peered":"2023-11-28T13:00:26.347017+0000","last_clean":"2023-11-28T13:00:26.347017+0000","last_became_active":"2023-11-28T13:00:25.468345+0000","last_became_peered":"2023-11-28T13:00:25.468345+0000","last_unstale":"2023-11-28T13:00:26.347017+0000","last_undegraded":"2023-11-28T13:00:26.347017+0000","last_fullsized":"2023-11-28T13:00:26.347017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:16.201562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441951+0000","last_change":"2023-11-28T13:00:20.993647+0000","last_active":"2023-11-28T13:00:25.441951+0000","last_peered":"2023-11-28T13:00:25.441951+0000","last_clean":"2023-11-28T13:00:25.441951+0000","last_became_active":"2023-11-28T13:00:20.993298+0000","last_became_peered":"2023-11-28T13:00:20.993298+0000","last_unstale":"2023-11-28T13:00:25.441951+0000","last_undegraded":"2023-11-28T13:00:25.441951+0000","last_fullsized":"2023-11-28T13:00:25.441951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:14:48.024917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442632+0000","last_change":"2023-11-28T13:00:21.429127+0000","last_active":"2023-11-28T13:00:25.442632+0000","last_peered":"2023-11-28T13:00:25.442632+0000","last_clean":"2023-11-28T13:00:25.442632+0000","last_became_active":"2023-11-28T13:00:21.428830+0000","last_became_peered":"2023-11-28T13:00:21.428830+0000","last_unstale":"2023-11-28T13:00:25.442632+0000","last_undegraded":"2023-11-28T13:00:25.442632+0000","last_fullsized":"2023-11-28T13:00:25.442632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:55.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871898+0000","last_change":"2023-11-28T13:00:25.464255+0000","last_active":"2023-11-28T13:00:25.871898+0000","last_peered":"2023-11-28T13:00:25.871898+0000","last_clean":"2023-11-28T13:00:25.871898+0000","last_became_active":"2023-11-28T13:00:25.463731+0000","last_became_peered":"2023-11-28T13:00:25.463731+0000","last_unstale":"2023-11-28T13:00:25.871898+0000","last_undegraded":"2023-11-28T13:00:25.871898+0000","last_fullsized":"2023-11-28T13:00:25.871898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:23:35.278662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.447931+0000","last_change":"2023-11-28T13:00:21.002097+0000","last_active":"2023-11-28T13:00:25.447931+0000","last_peered":"2023-11-28T13:00:25.447931+0000","last_clean":"2023-11-28T13:00:25.447931+0000","last_became_active":"2023-11-28T13:00:21.001830+0000","last_became_peered":"2023-11-28T13:00:21.001830+0000","last_unstale":"2023-11-28T13:00:25.447931+0000","last_undegraded":"2023-11-28T13:00:25.447931+0000","last_fullsized":"2023-11-28T13:00:25.447931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:18.850685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448366+0000","last_change":"2023-11-28T13:00:21.426617+0000","last_active":"2023-11-28T13:00:25.448366+0000","last_peered":"2023-11-28T13:00:25.448366+0000","last_clean":"2023-11-28T13:00:25.448366+0000","last_became_active":"2023-11-28T13:00:21.426326+0000","last_became_peered":"2023-11-28T13:00:21.426326+0000","last_unstale":"2023-11-28T13:00:25.448366+0000","last_undegraded":"2023-11-28T13:00:25.448366+0000","last_fullsized":"2023-11-28T13:00:25.448366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231013+0000","last_change":"2023-11-28T13:00:25.476625+0000","last_active":"2023-11-28T13:00:26.231013+0000","last_peered":"2023-11-28T13:00:26.231013+0000","last_clean":"2023-11-28T13:00:26.231013+0000","last_became_active":"2023-11-28T13:00:25.476137+0000","last_became_peered":"2023-11-28T13:00:25.476137+0000","last_unstale":"2023-11-28T13:00:26.231013+0000","last_undegraded":"2023-11-28T13:00:26.231013+0000","last_fullsized":"2023-11-28T13:00:26.231013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:28.631772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446518+0000","last_change":"2023-11-28T13:00:21.005624+0000","last_active":"2023-11-28T13:00:25.446518+0000","last_peered":"2023-11-28T13:00:25.446518+0000","last_clean":"2023-11-28T13:00:25.446518+0000","last_became_active":"2023-11-28T13:00:21.005092+0000","last_became_peered":"2023-11-28T13:00:21.005092+0000","last_unstale":"2023-11-28T13:00:25.446518+0000","last_undegraded":"2023-11-28T13:00:25.446518+0000","last_fullsized":"2023-11-28T13:00:25.446518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:37.060637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459224+0000","last_change":"2023-11-28T13:00:21.439380+0000","last_active":"2023-11-28T13:00:25.459224+0000","last_peered":"2023-11-28T13:00:25.459224+0000","last_clean":"2023-11-28T13:00:25.459224+0000","last_became_active":"2023-11-28T13:00:21.439094+0000","last_became_peered":"2023-11-28T13:00:21.439094+0000","last_unstale":"2023-11-28T13:00:25.459224+0000","last_undegraded":"2023-11-28T13:00:25.459224+0000","last_fullsized":"2023-11-28T13:00:25.459224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:02.248744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231207+0000","last_change":"2023-11-28T13:00:25.464879+0000","last_active":"2023-11-28T13:00:26.231207+0000","last_peered":"2023-11-28T13:00:26.231207+0000","last_clean":"2023-11-28T13:00:26.231207+0000","last_became_active":"2023-11-28T13:00:25.464486+0000","last_became_peered":"2023-11-28T13:00:25.464486+0000","last_unstale":"2023-11-28T13:00:26.231207+0000","last_undegraded":"2023-11-28T13:00:26.231207+0000","last_fullsized":"2023-11-28T13:00:26.231207+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:39.922810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446121+0000","last_change":"2023-11-28T13:00:21.003453+0000","last_active":"2023-11-28T13:00:25.446121+0000","last_peered":"2023-11-28T13:00:25.446121+0000","last_clean":"2023-11-28T13:00:25.446121+0000","last_became_active":"2023-11-28T13:00:21.003010+0000","last_became_peered":"2023-11-28T13:00:21.003010+0000","last_unstale":"2023-11-28T13:00:25.446121+0000","last_undegraded":"2023-11-28T13:00:25.446121+0000","last_fullsized":"2023-11-28T13:00:25.446121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:50.427721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442176+0000","last_change":"2023-11-28T13:00:21.428448+0000","last_active":"2023-11-28T13:00:25.442176+0000","last_peered":"2023-11-28T13:00:25.442176+0000","last_clean":"2023-11-28T13:00:25.442176+0000","last_became_active":"2023-11-28T13:00:21.427997+0000","last_became_peered":"2023-11-28T13:00:21.427997+0000","last_unstale":"2023-11-28T13:00:25.442176+0000","last_undegraded":"2023-11-28T13:00:25.442176+0000","last_fullsized":"2023-11-28T13:00:25.442176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:41.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231241+0000","last_change":"2023-11-28T13:00:25.465904+0000","last_active":"2023-11-28T13:00:26.231241+0000","last_peered":"2023-11-28T13:00:26.231241+0000","last_clean":"2023-11-28T13:00:26.231241+0000","last_became_active":"2023-11-28T13:00:25.465285+0000","last_became_peered":"2023-11-28T13:00:25.465285+0000","last_unstale":"2023-11-28T13:00:26.231241+0000","last_undegraded":"2023-11-28T13:00:26.231241+0000","last_fullsized":"2023-11-28T13:00:26.231241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:36.890229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450628+0000","last_change":"2023-11-28T13:00:21.010220+0000","last_active":"2023-11-28T13:00:25.450628+0000","last_peered":"2023-11-28T13:00:25.450628+0000","last_clean":"2023-11-28T13:00:25.450628+0000","last_became_active":"2023-11-28T13:00:21.009892+0000","last_became_peered":"2023-11-28T13:00:21.009892+0000","last_unstale":"2023-11-28T13:00:25.450628+0000","last_undegraded":"2023-11-28T13:00:25.450628+0000","last_fullsized":"2023-11-28T13:00:25.450628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448937+0000","last_change":"2023-11-28T13:00:21.421459+0000","last_active":"2023-11-28T13:00:25.448937+0000","last_peered":"2023-11-28T13:00:25.448937+0000","last_clean":"2023-11-28T13:00:25.448937+0000","last_became_active":"2023-11-28T13:00:21.420822+0000","last_became_peered":"2023-11-28T13:00:21.420822+0000","last_unstale":"2023-11-28T13:00:25.448937+0000","last_undegraded":"2023-11-28T13:00:25.448937+0000","last_fullsized":"2023-11-28T13:00:25.448937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:32:01.515518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231366+0000","last_change":"2023-11-28T13:00:25.464941+0000","last_active":"2023-11-28T13:00:26.231366+0000","last_peered":"2023-11-28T13:00:26.231366+0000","last_clean":"2023-11-28T13:00:26.231366+0000","last_became_active":"2023-11-28T13:00:25.464504+0000","last_became_peered":"2023-11-28T13:00:25.464504+0000","last_unstale":"2023-11-28T13:00:26.231366+0000","last_undegraded":"2023-11-28T13:00:26.231366+0000","last_fullsized":"2023-11-28T13:00:26.231366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:11.602479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441429+0000","last_change":"2023-11-28T13:00:21.008391+0000","last_active":"2023-11-28T13:00:25.441429+0000","last_peered":"2023-11-28T13:00:25.441429+0000","last_clean":"2023-11-28T13:00:25.441429+0000","last_became_active":"2023-11-28T13:00:21.006513+0000","last_became_peered":"2023-11-28T13:00:21.006513+0000","last_unstale":"2023-11-28T13:00:25.441429+0000","last_undegraded":"2023-11-28T13:00:25.441429+0000","last_fullsized":"2023-11-28T13:00:25.441429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:58.676237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460081+0000","last_change":"2023-11-28T13:00:21.431780+0000","last_active":"2023-11-28T13:00:25.460081+0000","last_peered":"2023-11-28T13:00:25.460081+0000","last_clean":"2023-11-28T13:00:25.460081+0000","last_became_active":"2023-11-28T13:00:21.430937+0000","last_became_peered":"2023-11-28T13:00:21.430937+0000","last_unstale":"2023-11-28T13:00:25.460081+0000","last_undegraded":"2023-11-28T13:00:25.460081+0000","last_fullsized":"2023-11-28T13:00:25.460081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:12:13.983682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076305+0000","last_change":"2023-11-28T13:00:25.466505+0000","last_active":"2023-11-28T13:00:26.076305+0000","last_peered":"2023-11-28T13:00:26.076305+0000","last_clean":"2023-11-28T13:00:26.076305+0000","last_became_active":"2023-11-28T13:00:25.466059+0000","last_became_peered":"2023-11-28T13:00:25.466059+0000","last_unstale":"2023-11-28T13:00:26.076305+0000","last_undegraded":"2023-11-28T13:00:26.076305+0000","last_fullsized":"2023-11-28T13:00:26.076305+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:12.856136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442306+0000","last_change":"2023-11-28T13:00:21.003064+0000","last_active":"2023-11-28T13:00:25.442306+0000","last_peered":"2023-11-28T13:00:25.442306+0000","last_clean":"2023-11-28T13:00:25.442306+0000","last_became_active":"2023-11-28T13:00:21.002742+0000","last_became_peered":"2023-11-28T13:00:21.002742+0000","last_unstale":"2023-11-28T13:00:25.442306+0000","last_undegraded":"2023-11-28T13:00:25.442306+0000","last_fullsized":"2023-11-28T13:00:25.442306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:04.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460629+0000","last_change":"2023-11-28T13:00:21.427213+0000","last_active":"2023-11-28T13:00:25.460629+0000","last_peered":"2023-11-28T13:00:25.460629+0000","last_clean":"2023-11-28T13:00:25.460629+0000","last_became_active":"2023-11-28T13:00:21.426737+0000","last_became_peered":"2023-11-28T13:00:21.426737+0000","last_unstale":"2023-11-28T13:00:25.460629+0000","last_undegraded":"2023-11-28T13:00:25.460629+0000","last_fullsized":"2023-11-28T13:00:25.460629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:43.321870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347427+0000","last_change":"2023-11-28T13:00:25.466146+0000","last_active":"2023-11-28T13:00:26.347427+0000","last_peered":"2023-11-28T13:00:26.347427+0000","last_clean":"2023-11-28T13:00:26.347427+0000","last_became_active":"2023-11-28T13:00:25.465652+0000","last_became_peered":"2023-11-28T13:00:25.465652+0000","last_unstale":"2023-11-28T13:00:26.347427+0000","last_undegraded":"2023-11-28T13:00:26.347427+0000","last_fullsized":"2023-11-28T13:00:26.347427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:14.273148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459071+0000","last_change":"2023-11-28T13:00:21.001745+0000","last_active":"2023-11-28T13:00:25.459071+0000","last_peered":"2023-11-28T13:00:25.459071+0000","last_clean":"2023-11-28T13:00:25.459071+0000","last_became_active":"2023-11-28T13:00:21.001473+0000","last_became_peered":"2023-11-28T13:00:21.001473+0000","last_unstale":"2023-11-28T13:00:25.459071+0000","last_undegraded":"2023-11-28T13:00:25.459071+0000","last_fullsized":"2023-11-28T13:00:25.459071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.462841+0000","last_change":"2023-11-28T13:00:21.422938+0000","last_active":"2023-11-28T13:00:25.462841+0000","last_peered":"2023-11-28T13:00:25.462841+0000","last_clean":"2023-11-28T13:00:25.462841+0000","last_became_active":"2023-11-28T13:00:21.422654+0000","last_became_peered":"2023-11-28T13:00:21.422654+0000","last_unstale":"2023-11-28T13:00:25.462841+0000","last_undegraded":"2023-11-28T13:00:25.462841+0000","last_fullsized":"2023-11-28T13:00:25.462841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:28:59.158475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.870965+0000","last_change":"2023-11-28T13:00:25.463080+0000","last_active":"2023-11-28T13:00:25.870965+0000","last_peered":"2023-11-28T13:00:25.870965+0000","last_clean":"2023-11-28T13:00:25.870965+0000","last_became_active":"2023-11-28T13:00:25.462460+0000","last_became_peered":"2023-11-28T13:00:25.462460+0000","last_unstale":"2023-11-28T13:00:25.870965+0000","last_undegraded":"2023-11-28T13:00:25.870965+0000","last_fullsized":"2023-11-28T13:00:25.870965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:45.428316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441326+0000","last_change":"2023-11-28T13:00:21.009988+0000","last_active":"2023-11-28T13:00:25.441326+0000","last_peered":"2023-11-28T13:00:25.441326+0000","last_clean":"2023-11-28T13:00:25.441326+0000","last_became_active":"2023-11-28T13:00:21.009637+0000","last_became_peered":"2023-11-28T13:00:21.009637+0000","last_unstale":"2023-11-28T13:00:25.441326+0000","last_undegraded":"2023-11-28T13:00:25.441326+0000","last_fullsized":"2023-11-28T13:00:25.441326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:53.009716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458749+0000","last_change":"2023-11-28T13:00:21.433621+0000","last_active":"2023-11-28T13:00:25.458749+0000","last_peered":"2023-11-28T13:00:25.458749+0000","last_clean":"2023-11-28T13:00:25.458749+0000","last_became_active":"2023-11-28T13:00:21.433293+0000","last_became_peered":"2023-11-28T13:00:21.433293+0000","last_unstale":"2023-11-28T13:00:25.458749+0000","last_undegraded":"2023-11-28T13:00:25.458749+0000","last_fullsized":"2023-11-28T13:00:25.458749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:18.248057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.870341+0000","last_change":"2023-11-28T13:00:25.460697+0000","last_active":"2023-11-28T13:00:25.870341+0000","last_peered":"2023-11-28T13:00:25.870341+0000","last_clean":"2023-11-28T13:00:25.870341+0000","last_became_active":"2023-11-28T13:00:25.460230+0000","last_became_peered":"2023-11-28T13:00:25.460230+0000","last_unstale":"2023-11-28T13:00:25.870341+0000","last_undegraded":"2023-11-28T13:00:25.870341+0000","last_fullsized":"2023-11-28T13:00:25.870341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:25.897486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442429+0000","last_change":"2023-11-28T13:00:21.001229+0000","last_active":"2023-11-28T13:00:25.442429+0000","last_peered":"2023-11-28T13:00:25.442429+0000","last_clean":"2023-11-28T13:00:25.442429+0000","last_became_active":"2023-11-28T13:00:21.000871+0000","last_became_peered":"2023-11-28T13:00:21.000871+0000","last_unstale":"2023-11-28T13:00:25.442429+0000","last_undegraded":"2023-11-28T13:00:25.442429+0000","last_fullsized":"2023-11-28T13:00:25.442429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:49.179565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460903+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:00:25.460903+0000","last_peered":"2023-11-28T13:00:25.460903+0000","last_clean":"2023-11-28T13:00:25.460903+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:00:25.460903+0000","last_undegraded":"2023-11-28T13:00:25.460903+0000","last_fullsized":"2023-11-28T13:00:25.460903+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441199+0000","last_change":"2023-11-28T13:00:21.429135+0000","last_active":"2023-11-28T13:00:25.441199+0000","last_peered":"2023-11-28T13:00:25.441199+0000","last_clean":"2023-11-28T13:00:25.441199+0000","last_became_active":"2023-11-28T13:00:21.428808+0000","last_became_peered":"2023-11-28T13:00:21.428808+0000","last_unstale":"2023-11-28T13:00:25.441199+0000","last_undegraded":"2023-11-28T13:00:25.441199+0000","last_fullsized":"2023-11-28T13:00:25.441199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:28:36.151382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.868731+0000","last_change":"2023-11-28T13:00:25.472188+0000","last_active":"2023-11-28T13:00:25.868731+0000","last_peered":"2023-11-28T13:00:25.868731+0000","last_clean":"2023-11-28T13:00:25.868731+0000","last_became_active":"2023-11-28T13:00:25.471768+0000","last_became_peered":"2023-11-28T13:00:25.471768+0000","last_unstale":"2023-11-28T13:00:25.868731+0000","last_undegraded":"2023-11-28T13:00:25.868731+0000","last_fullsized":"2023-11-28T13:00:25.868731+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:17:47.454963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441547+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:00:25.441547+0000","last_peered":"2023-11-28T13:00:25.441547+0000","last_clean":"2023-11-28T13:00:25.441547+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:00:25.441547+0000","last_undegraded":"2023-11-28T13:00:25.441547+0000","last_fullsized":"2023-11-28T13:00:25.441547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073655+0000","last_change":"2023-11-28T13:00:20.998799+0000","last_active":"2023-11-28T13:00:26.073655+0000","last_peered":"2023-11-28T13:00:26.073655+0000","last_clean":"2023-11-28T13:00:26.073655+0000","last_became_active":"2023-11-28T13:00:20.998482+0000","last_became_peered":"2023-11-28T13:00:20.998482+0000","last_unstale":"2023-11-28T13:00:26.073655+0000","last_undegraded":"2023-11-28T13:00:26.073655+0000","last_fullsized":"2023-11-28T13:00:26.073655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:28.673649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459183+0000","last_change":"2023-11-28T13:00:21.440454+0000","last_active":"2023-11-28T13:00:25.459183+0000","last_peered":"2023-11-28T13:00:25.459183+0000","last_clean":"2023-11-28T13:00:25.459183+0000","last_became_active":"2023-11-28T13:00:21.439616+0000","last_became_peered":"2023-11-28T13:00:21.439616+0000","last_unstale":"2023-11-28T13:00:25.459183+0000","last_undegraded":"2023-11-28T13:00:25.459183+0000","last_fullsized":"2023-11-28T13:00:25.459183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:36.472288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347208+0000","last_change":"2023-11-28T13:00:25.468244+0000","last_active":"2023-11-28T13:00:26.347208+0000","last_peered":"2023-11-28T13:00:26.347208+0000","last_clean":"2023-11-28T13:00:26.347208+0000","last_became_active":"2023-11-28T13:00:25.467850+0000","last_became_peered":"2023-11-28T13:00:25.467850+0000","last_unstale":"2023-11-28T13:00:26.347208+0000","last_undegraded":"2023-11-28T13:00:26.347208+0000","last_fullsized":"2023-11-28T13:00:26.347208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:47.712713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.456847+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:00:25.456847+0000","last_peered":"2023-11-28T13:00:25.456847+0000","last_clean":"2023-11-28T13:00:25.456847+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:00:25.456847+0000","last_undegraded":"2023-11-28T13:00:25.456847+0000","last_fullsized":"2023-11-28T13:00:25.456847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446771+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:00:25.446771+0000","last_peered":"2023-11-28T13:00:25.446771+0000","last_clean":"2023-11-28T13:00:25.446771+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:00:25.446771+0000","last_undegraded":"2023-11-28T13:00:25.446771+0000","last_fullsized":"2023-11-28T13:00:25.446771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.447961+0000","last_change":"2023-11-28T13:00:21.005940+0000","last_active":"2023-11-28T13:00:25.447961+0000","last_peered":"2023-11-28T13:00:25.447961+0000","last_clean":"2023-11-28T13:00:25.447961+0000","last_became_active":"2023-11-28T13:00:21.005231+0000","last_became_peered":"2023-11-28T13:00:21.005231+0000","last_unstale":"2023-11-28T13:00:25.447961+0000","last_undegraded":"2023-11-28T13:00:25.447961+0000","last_fullsized":"2023-11-28T13:00:25.447961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:49.179565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448989+0000","last_change":"2023-11-28T13:00:21.426195+0000","last_active":"2023-11-28T13:00:25.448989+0000","last_peered":"2023-11-28T13:00:25.448989+0000","last_clean":"2023-11-28T13:00:25.448989+0000","last_became_active":"2023-11-28T13:00:21.423397+0000","last_became_peered":"2023-11-28T13:00:21.423397+0000","last_unstale":"2023-11-28T13:00:25.448989+0000","last_undegraded":"2023-11-28T13:00:25.448989+0000","last_fullsized":"2023-11-28T13:00:25.448989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231539+0000","last_change":"2023-11-28T13:00:25.468335+0000","last_active":"2023-11-28T13:00:26.231539+0000","last_peered":"2023-11-28T13:00:26.231539+0000","last_clean":"2023-11-28T13:00:26.231539+0000","last_became_active":"2023-11-28T13:00:25.467898+0000","last_became_peered":"2023-11-28T13:00:25.467898+0000","last_unstale":"2023-11-28T13:00:26.231539+0000","last_undegraded":"2023-11-28T13:00:26.231539+0000","last_fullsized":"2023-11-28T13:00:26.231539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:20.872923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443294+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:00:25.443294+0000","last_peered":"2023-11-28T13:00:25.443294+0000","last_clean":"2023-11-28T13:00:25.443294+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:00:25.443294+0000","last_undegraded":"2023-11-28T13:00:25.443294+0000","last_fullsized":"2023-11-28T13:00:25.443294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460654+0000","last_change":"2023-11-28T13:00:21.007444+0000","last_active":"2023-11-28T13:00:25.460654+0000","last_peered":"2023-11-28T13:00:25.460654+0000","last_clean":"2023-11-28T13:00:25.460654+0000","last_became_active":"2023-11-28T13:00:21.007159+0000","last_became_peered":"2023-11-28T13:00:21.007159+0000","last_unstale":"2023-11-28T13:00:25.460654+0000","last_undegraded":"2023-11-28T13:00:25.460654+0000","last_fullsized":"2023-11-28T13:00:25.460654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:55.415887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458654+0000","last_change":"2023-11-28T13:00:21.437327+0000","last_active":"2023-11-28T13:00:25.458654+0000","last_peered":"2023-11-28T13:00:25.458654+0000","last_clean":"2023-11-28T13:00:25.458654+0000","last_became_active":"2023-11-28T13:00:21.437058+0000","last_became_peered":"2023-11-28T13:00:21.437058+0000","last_unstale":"2023-11-28T13:00:25.458654+0000","last_undegraded":"2023-11-28T13:00:25.458654+0000","last_fullsized":"2023-11-28T13:00:25.458654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871438+0000","last_change":"2023-11-28T13:00:25.462821+0000","last_active":"2023-11-28T13:00:25.871438+0000","last_peered":"2023-11-28T13:00:25.871438+0000","last_clean":"2023-11-28T13:00:25.871438+0000","last_became_active":"2023-11-28T13:00:25.462378+0000","last_became_peered":"2023-11-28T13:00:25.462378+0000","last_unstale":"2023-11-28T13:00:25.871438+0000","last_undegraded":"2023-11-28T13:00:25.871438+0000","last_fullsized":"2023-11-28T13:00:25.871438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:06:52.582450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446128+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:00:25.446128+0000","last_peered":"2023-11-28T13:00:25.446128+0000","last_clean":"2023-11-28T13:00:25.446128+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:00:25.446128+0000","last_undegraded":"2023-11-28T13:00:25.446128+0000","last_fullsized":"2023-11-28T13:00:25.446128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441243+0000","last_change":"2023-11-28T13:00:21.005259+0000","last_active":"2023-11-28T13:00:25.441243+0000","last_peered":"2023-11-28T13:00:25.441243+0000","last_clean":"2023-11-28T13:00:25.441243+0000","last_became_active":"2023-11-28T13:00:21.004973+0000","last_became_peered":"2023-11-28T13:00:21.004973+0000","last_unstale":"2023-11-28T13:00:25.441243+0000","last_undegraded":"2023-11-28T13:00:25.441243+0000","last_fullsized":"2023-11-28T13:00:25.441243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:18.754380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442055+0000","last_change":"2023-11-28T13:00:21.428421+0000","last_active":"2023-11-28T13:00:25.442055+0000","last_peered":"2023-11-28T13:00:25.442055+0000","last_clean":"2023-11-28T13:00:25.442055+0000","last_became_active":"2023-11-28T13:00:21.427914+0000","last_became_peered":"2023-11-28T13:00:21.427914+0000","last_unstale":"2023-11-28T13:00:25.442055+0000","last_undegraded":"2023-11-28T13:00:25.442055+0000","last_fullsized":"2023-11-28T13:00:25.442055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:13.464393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073795+0000","last_change":"2023-11-28T13:00:25.468907+0000","last_active":"2023-11-28T13:00:26.073795+0000","last_peered":"2023-11-28T13:00:26.073795+0000","last_clean":"2023-11-28T13:00:26.073795+0000","last_became_active":"2023-11-28T13:00:25.468477+0000","last_became_peered":"2023-11-28T13:00:25.468477+0000","last_unstale":"2023-11-28T13:00:26.073795+0000","last_undegraded":"2023-11-28T13:00:26.073795+0000","last_fullsized":"2023-11-28T13:00:26.073795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:36:04.014440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448786+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:00:25.448786+0000","last_peered":"2023-11-28T13:00:25.448786+0000","last_clean":"2023-11-28T13:00:25.448786+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:00:25.448786+0000","last_undegraded":"2023-11-28T13:00:25.448786+0000","last_fullsized":"2023-11-28T13:00:25.448786+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460768+0000","last_change":"2023-11-28T13:00:21.008135+0000","last_active":"2023-11-28T13:00:25.460768+0000","last_peered":"2023-11-28T13:00:25.460768+0000","last_clean":"2023-11-28T13:00:25.460768+0000","last_became_active":"2023-11-28T13:00:21.007883+0000","last_became_peered":"2023-11-28T13:00:21.007883+0000","last_unstale":"2023-11-28T13:00:25.460768+0000","last_undegraded":"2023-11-28T13:00:25.460768+0000","last_fullsized":"2023-11-28T13:00:25.460768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455821+0000","last_change":"2023-11-28T13:00:21.425530+0000","last_active":"2023-11-28T13:00:25.455821+0000","last_peered":"2023-11-28T13:00:25.455821+0000","last_clean":"2023-11-28T13:00:25.455821+0000","last_became_active":"2023-11-28T13:00:21.425325+0000","last_became_peered":"2023-11-28T13:00:21.425325+0000","last_unstale":"2023-11-28T13:00:25.455821+0000","last_undegraded":"2023-11-28T13:00:25.455821+0000","last_fullsized":"2023-11-28T13:00:25.455821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:43.401826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346874+0000","last_change":"2023-11-28T13:00:25.457191+0000","last_active":"2023-11-28T13:00:26.346874+0000","last_peered":"2023-11-28T13:00:26.346874+0000","last_clean":"2023-11-28T13:00:26.346874+0000","last_became_active":"2023-11-28T13:00:25.456577+0000","last_became_peered":"2023-11-28T13:00:25.456577+0000","last_unstale":"2023-11-28T13:00:26.346874+0000","last_undegraded":"2023-11-28T13:00:26.346874+0000","last_fullsized":"2023-11-28T13:00:26.346874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:11.602479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442371+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:00:25.442371+0000","last_peered":"2023-11-28T13:00:25.442371+0000","last_clean":"2023-11-28T13:00:25.442371+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:00:25.442371+0000","last_undegraded":"2023-11-28T13:00:25.442371+0000","last_fullsized":"2023-11-28T13:00:25.442371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455999+0000","last_change":"2023-11-28T13:00:21.006815+0000","last_active":"2023-11-28T13:00:25.455999+0000","last_peered":"2023-11-28T13:00:25.455999+0000","last_clean":"2023-11-28T13:00:25.455999+0000","last_became_active":"2023-11-28T13:00:21.006608+0000","last_became_peered":"2023-11-28T13:00:21.006608+0000","last_unstale":"2023-11-28T13:00:25.455999+0000","last_undegraded":"2023-11-28T13:00:25.455999+0000","last_fullsized":"2023-11-28T13:00:25.455999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:28.673649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448670+0000","last_change":"2023-11-28T13:00:21.421641+0000","last_active":"2023-11-28T13:00:25.448670+0000","last_peered":"2023-11-28T13:00:25.448670+0000","last_clean":"2023-11-28T13:00:25.448670+0000","last_became_active":"2023-11-28T13:00:21.421196+0000","last_became_peered":"2023-11-28T13:00:21.421196+0000","last_unstale":"2023-11-28T13:00:25.448670+0000","last_undegraded":"2023-11-28T13:00:25.448670+0000","last_fullsized":"2023-11-28T13:00:25.448670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:49.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073549+0000","last_change":"2023-11-28T13:00:25.468640+0000","last_active":"2023-11-28T13:00:26.073549+0000","last_peered":"2023-11-28T13:00:26.073549+0000","last_clean":"2023-11-28T13:00:26.073549+0000","last_became_active":"2023-11-28T13:00:25.467992+0000","last_became_peered":"2023-11-28T13:00:25.467992+0000","last_unstale":"2023-11-28T13:00:26.073549+0000","last_undegraded":"2023-11-28T13:00:26.073549+0000","last_fullsized":"2023-11-28T13:00:26.073549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:45:50.526405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442668+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:00:25.442668+0000","last_peered":"2023-11-28T13:00:25.442668+0000","last_clean":"2023-11-28T13:00:25.442668+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:00:25.442668+0000","last_undegraded":"2023-11-28T13:00:25.442668+0000","last_fullsized":"2023-11-28T13:00:25.442668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450339+0000","last_change":"2023-11-28T13:00:21.010100+0000","last_active":"2023-11-28T13:00:25.450339+0000","last_peered":"2023-11-28T13:00:25.450339+0000","last_clean":"2023-11-28T13:00:25.450339+0000","last_became_active":"2023-11-28T13:00:21.009797+0000","last_became_peered":"2023-11-28T13:00:21.009797+0000","last_unstale":"2023-11-28T13:00:25.450339+0000","last_undegraded":"2023-11-28T13:00:25.450339+0000","last_fullsized":"2023-11-28T13:00:25.450339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:18.850685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457393+0000","last_change":"2023-11-28T13:00:21.433346+0000","last_active":"2023-11-28T13:00:25.457393+0000","last_peered":"2023-11-28T13:00:25.457393+0000","last_clean":"2023-11-28T13:00:25.457393+0000","last_became_active":"2023-11-28T13:00:21.433022+0000","last_became_peered":"2023-11-28T13:00:21.433022+0000","last_unstale":"2023-11-28T13:00:25.457393+0000","last_undegraded":"2023-11-28T13:00:25.457393+0000","last_fullsized":"2023-11-28T13:00:25.457393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:34:14.133792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076291+0000","last_change":"2023-11-28T13:00:25.474463+0000","last_active":"2023-11-28T13:00:26.076291+0000","last_peered":"2023-11-28T13:00:26.076291+0000","last_clean":"2023-11-28T13:00:26.076291+0000","last_became_active":"2023-11-28T13:00:25.473727+0000","last_became_peered":"2023-11-28T13:00:25.473727+0000","last_unstale":"2023-11-28T13:00:26.076291+0000","last_undegraded":"2023-11-28T13:00:26.076291+0000","last_fullsized":"2023-11-28T13:00:26.076291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:35:49.356939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448491+0000","last_change":"2023-11-28T13:00:21.005791+0000","last_active":"2023-11-28T13:00:25.448491+0000","last_peered":"2023-11-28T13:00:25.448491+0000","last_clean":"2023-11-28T13:00:25.448491+0000","last_became_active":"2023-11-28T13:00:21.005474+0000","last_became_peered":"2023-11-28T13:00:21.005474+0000","last_unstale":"2023-11-28T13:00:25.448491+0000","last_undegraded":"2023-11-28T13:00:25.448491+0000","last_fullsized":"2023-11-28T13:00:25.448491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:14.890428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455579+0000","last_change":"2023-11-28T13:00:21.424133+0000","last_active":"2023-11-28T13:00:25.455579+0000","last_peered":"2023-11-28T13:00:25.455579+0000","last_clean":"2023-11-28T13:00:25.455579+0000","last_became_active":"2023-11-28T13:00:21.423849+0000","last_became_peered":"2023-11-28T13:00:21.423849+0000","last_unstale":"2023-11-28T13:00:25.455579+0000","last_undegraded":"2023-11-28T13:00:25.455579+0000","last_fullsized":"2023-11-28T13:00:25.455579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:05.801837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871653+0000","last_change":"2023-11-28T13:00:25.468445+0000","last_active":"2023-11-28T13:00:25.871653+0000","last_peered":"2023-11-28T13:00:25.871653+0000","last_clean":"2023-11-28T13:00:25.871653+0000","last_became_active":"2023-11-28T13:00:25.468042+0000","last_became_peered":"2023-11-28T13:00:25.468042+0000","last_unstale":"2023-11-28T13:00:25.871653+0000","last_undegraded":"2023-11-28T13:00:25.871653+0000","last_fullsized":"2023-11-28T13:00:25.871653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.716007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449852+0000","last_change":"2023-11-28T13:00:21.006819+0000","last_active":"2023-11-28T13:00:25.449852+0000","last_peered":"2023-11-28T13:00:25.449852+0000","last_clean":"2023-11-28T13:00:25.449852+0000","last_became_active":"2023-11-28T13:00:21.006504+0000","last_became_peered":"2023-11-28T13:00:21.006504+0000","last_unstale":"2023-11-28T13:00:25.449852+0000","last_undegraded":"2023-11-28T13:00:25.449852+0000","last_fullsized":"2023-11-28T13:00:25.449852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:37.060637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459182+0000","last_change":"2023-11-28T13:00:21.436419+0000","last_active":"2023-11-28T13:00:25.459182+0000","last_peered":"2023-11-28T13:00:25.459182+0000","last_clean":"2023-11-28T13:00:25.459182+0000","last_became_active":"2023-11-28T13:00:21.436087+0000","last_became_peered":"2023-11-28T13:00:21.436087+0000","last_unstale":"2023-11-28T13:00:25.459182+0000","last_undegraded":"2023-11-28T13:00:25.459182+0000","last_fullsized":"2023-11-28T13:00:25.459182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:18.316015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346972+0000","last_change":"2023-11-28T13:00:25.464176+0000","last_active":"2023-11-28T13:00:26.346972+0000","last_peered":"2023-11-28T13:00:26.346972+0000","last_clean":"2023-11-28T13:00:26.346972+0000","last_became_active":"2023-11-28T13:00:25.463779+0000","last_became_peered":"2023-11-28T13:00:25.463779+0000","last_unstale":"2023-11-28T13:00:26.346972+0000","last_undegraded":"2023-11-28T13:00:26.346972+0000","last_fullsized":"2023-11-28T13:00:26.346972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.716007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450222+0000","last_change":"2023-11-28T13:00:21.006862+0000","last_active":"2023-11-28T13:00:25.450222+0000","last_peered":"2023-11-28T13:00:25.450222+0000","last_clean":"2023-11-28T13:00:25.450222+0000","last_became_active":"2023-11-28T13:00:21.006580+0000","last_became_peered":"2023-11-28T13:00:21.006580+0000","last_unstale":"2023-11-28T13:00:25.450222+0000","last_undegraded":"2023-11-28T13:00:25.450222+0000","last_fullsized":"2023-11-28T13:00:25.450222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:01.729666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460103+0000","last_change":"2023-11-28T13:00:21.427093+0000","last_active":"2023-11-28T13:00:25.460103+0000","last_peered":"2023-11-28T13:00:25.460103+0000","last_clean":"2023-11-28T13:00:25.460103+0000","last_became_active":"2023-11-28T13:00:21.426329+0000","last_became_peered":"2023-11-28T13:00:21.426329+0000","last_unstale":"2023-11-28T13:00:25.460103+0000","last_undegraded":"2023-11-28T13:00:25.460103+0000","last_fullsized":"2023-11-28T13:00:25.460103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:26:04.532312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869619+0000","last_change":"2023-11-28T13:00:25.475157+0000","last_active":"2023-11-28T13:00:25.869619+0000","last_peered":"2023-11-28T13:00:25.869619+0000","last_clean":"2023-11-28T13:00:25.869619+0000","last_became_active":"2023-11-28T13:00:25.474714+0000","last_became_peered":"2023-11-28T13:00:25.474714+0000","last_unstale":"2023-11-28T13:00:25.869619+0000","last_undegraded":"2023-11-28T13:00:25.869619+0000","last_fullsized":"2023-11-28T13:00:25.869619+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:43.383854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441539+0000","last_change":"2023-11-28T13:00:21.004157+0000","last_active":"2023-11-28T13:00:25.441539+0000","last_peered":"2023-11-28T13:00:25.441539+0000","last_clean":"2023-11-28T13:00:25.441539+0000","last_became_active":"2023-11-28T13:00:21.003717+0000","last_became_peered":"2023-11-28T13:00:21.003717+0000","last_unstale":"2023-11-28T13:00:25.441539+0000","last_undegraded":"2023-11-28T13:00:25.441539+0000","last_fullsized":"2023-11-28T13:00:25.441539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:14.599072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441293+0000","last_change":"2023-11-28T13:00:21.427401+0000","last_active":"2023-11-28T13:00:25.441293+0000","last_peered":"2023-11-28T13:00:25.441293+0000","last_clean":"2023-11-28T13:00:25.441293+0000","last_became_active":"2023-11-28T13:00:21.425193+0000","last_became_peered":"2023-11-28T13:00:21.425193+0000","last_unstale":"2023-11-28T13:00:25.441293+0000","last_undegraded":"2023-11-28T13:00:25.441293+0000","last_fullsized":"2023-11-28T13:00:25.441293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:16.691765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076011+0000","last_change":"2023-11-28T13:00:25.466935+0000","last_active":"2023-11-28T13:00:26.076011+0000","last_peered":"2023-11-28T13:00:26.076011+0000","last_clean":"2023-11-28T13:00:26.076011+0000","last_became_active":"2023-11-28T13:00:25.466438+0000","last_became_peered":"2023-11-28T13:00:25.466438+0000","last_unstale":"2023-11-28T13:00:26.076011+0000","last_undegraded":"2023-11-28T13:00:26.076011+0000","last_fullsized":"2023-11-28T13:00:26.076011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:44.975678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448944+0000","last_change":"2023-11-28T13:00:21.003458+0000","last_active":"2023-11-28T13:00:25.448944+0000","last_peered":"2023-11-28T13:00:25.448944+0000","last_clean":"2023-11-28T13:00:25.448944+0000","last_became_active":"2023-11-28T13:00:21.002912+0000","last_became_peered":"2023-11-28T13:00:21.002912+0000","last_unstale":"2023-11-28T13:00:25.448944+0000","last_undegraded":"2023-11-28T13:00:25.448944+0000","last_fullsized":"2023-11-28T13:00:25.448944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:18.754380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458864+0000","last_change":"2023-11-28T13:00:21.422316+0000","last_active":"2023-11-28T13:00:25.458864+0000","last_peered":"2023-11-28T13:00:25.458864+0000","last_clean":"2023-11-28T13:00:25.458864+0000","last_became_active":"2023-11-28T13:00:21.422012+0000","last_became_peered":"2023-11-28T13:00:21.422012+0000","last_unstale":"2023-11-28T13:00:25.458864+0000","last_undegraded":"2023-11-28T13:00:25.458864+0000","last_fullsized":"2023-11-28T13:00:25.458864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442053+0000","last_change":"2023-11-28T13:00:21.423769+0000","last_active":"2023-11-28T13:00:25.442053+0000","last_peered":"2023-11-28T13:00:25.442053+0000","last_clean":"2023-11-28T13:00:25.442053+0000","last_became_active":"2023-11-28T13:00:21.423458+0000","last_became_peered":"2023-11-28T13:00:21.423458+0000","last_unstale":"2023-11-28T13:00:25.442053+0000","last_undegraded":"2023-11-28T13:00:25.442053+0000","last_fullsized":"2023-11-28T13:00:25.442053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:51.438322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441431+0000","last_change":"2023-11-28T13:00:21.432489+0000","last_active":"2023-11-28T13:00:25.441431+0000","last_peered":"2023-11-28T13:00:25.441431+0000","last_clean":"2023-11-28T13:00:25.441431+0000","last_became_active":"2023-11-28T13:00:21.432224+0000","last_became_peered":"2023-11-28T13:00:21.432224+0000","last_unstale":"2023-11-28T13:00:25.441431+0000","last_undegraded":"2023-11-28T13:00:25.441431+0000","last_fullsized":"2023-11-28T13:00:25.441431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:27.442626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460221+0000","last_change":"2023-11-28T13:00:21.427335+0000","last_active":"2023-11-28T13:00:25.460221+0000","last_peered":"2023-11-28T13:00:25.460221+0000","last_clean":"2023-11-28T13:00:25.460221+0000","last_became_active":"2023-11-28T13:00:21.426777+0000","last_became_peered":"2023-11-28T13:00:21.426777+0000","last_unstale":"2023-11-28T13:00:25.460221+0000","last_undegraded":"2023-11-28T13:00:25.460221+0000","last_fullsized":"2023-11-28T13:00:25.460221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:24.706933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441759+0000","last_change":"2023-11-28T13:00:21.436760+0000","last_active":"2023-11-28T13:00:25.441759+0000","last_peered":"2023-11-28T13:00:25.441759+0000","last_clean":"2023-11-28T13:00:25.441759+0000","last_became_active":"2023-11-28T13:00:21.436476+0000","last_became_peered":"2023-11-28T13:00:21.436476+0000","last_unstale":"2023-11-28T13:00:25.441759+0000","last_undegraded":"2023-11-28T13:00:25.441759+0000","last_fullsized":"2023-11-28T13:00:25.441759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:49.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455911+0000","last_change":"2023-11-28T13:00:21.415227+0000","last_active":"2023-11-28T13:00:25.455911+0000","last_peered":"2023-11-28T13:00:25.455911+0000","last_clean":"2023-11-28T13:00:25.455911+0000","last_became_active":"2023-11-28T13:00:21.414918+0000","last_became_peered":"2023-11-28T13:00:21.414918+0000","last_unstale":"2023-11-28T13:00:25.455911+0000","last_undegraded":"2023-11-28T13:00:25.455911+0000","last_fullsized":"2023-11-28T13:00:25.455911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:35:41.989664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4387,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":703951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9228,"kb_used_data":836,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362612,"statfs":{"total":96636764160,"available":96627314688,"internally_reserved":0,"allocated":856064,"data_stored":699125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8140,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363700,"statfs":{"total":96636764160,"available":96628428800,"internally_reserved":0,"allocated":266240,"data_stored":109342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8084,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363756,"statfs":{"total":96636764160,"available":96628486144,"internally_reserved":0,"allocated":274432,"data_stored":110398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:39.477 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:00:39.477 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T13:00:39.478 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T13:00:39.478 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T13:00:39.479 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T13:00:39.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.632+0000 7f8227f6e700 1 Processor -- start 2023-11-28T13:00:39.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.632+0000 7f8227f6e700 1 -- start start 2023-11-28T13:00:39.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f8227f6e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006bc30 0x7f822015b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f8227f6e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f822006dda0 0x7f822015e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f8227f6e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822006c3a0 con 0x7f822006d9d0 2023-11-28T13:00:39.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f8227f6e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822015ee90 con 0x7f822006dda0 2023-11-28T13:00:39.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f8227f6e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822006c0c0 con 0x7f822006bc30 2023-11-28T13:00:39.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f822650b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f822006dda0 0x7f822015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f8225509700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006bc30 0x7f822015b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f8225d0a700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f822006d9d0 0x7f8220150eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:35422/0 (socket says 172.21.15.6:35422) 2023-11-28T13:00:39.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f822650b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f822006dda0 0x7f822015e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:33562/0 (socket says 172.21.15.6:33562) 2023-11-28T13:00:39.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f8225509700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006bc30 0x7f822015b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47778/0 (socket says 172.21.15.6:47778) 2023-11-28T13:00:39.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.633+0000 7f8225d0a700 1 -- 172.21.15.6:0/2610114038 learned_addr learned my addr 172.21.15.6:0/2610114038 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:39.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.634+0000 7f822650b700 1 -- 172.21.15.6:0/2610114038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006bc30 msgr2=0x7f822015b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:39.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.634+0000 7f822650b700 1 --2- 172.21.15.6:0/2610114038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006bc30 0x7f822015b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.634+0000 7f822650b700 1 -- 172.21.15.6:0/2610114038 >> v1:172.21.15.61:6789/0 conn(0x7f822006d9d0 legacy=0x7f8220150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.634+0000 7f822650b700 1 -- 172.21.15.6:0/2610114038 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822015ab60 con 0x7f822006dda0 2023-11-28T13:00:39.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.634+0000 7f822650b700 1 --2- 172.21.15.6:0/2610114038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f822006dda0 0x7f822015e950 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8218009d90 tx=0x7f821800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b68c5b569776182 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:39.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.634+0000 7f8224d08700 1 -- 172.21.15.6:0/2610114038 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821800e070 con 0x7f822006dda0 2023-11-28T13:00:39.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.634+0000 7f8224d08700 1 -- 172.21.15.6:0/2610114038 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821800bb20 con 0x7f822006dda0 2023-11-28T13:00:39.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.634+0000 7f8224d08700 1 -- 172.21.15.6:0/2610114038 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218012430 con 0x7f822006dda0 2023-11-28T13:00:39.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.635+0000 7f8227f6e700 1 -- 172.21.15.6:0/2610114038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f822006dda0 msgr2=0x7f822015e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.635+0000 7f8227f6e700 1 --2- 172.21.15.6:0/2610114038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f822006dda0 0x7f822015e950 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8218009d90 tx=0x7f821800b330 comp rx=0 tx=0).stop 2023-11-28T13:00:39.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.635+0000 7f8227f6e700 1 -- 172.21.15.6:0/2610114038 shutdown_connections 2023-11-28T13:00:39.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.635+0000 7f8227f6e700 1 --2- 172.21.15.6:0/2610114038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f822006dda0 0x7f822015e950 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.635+0000 7f8227f6e700 1 --2- 172.21.15.6:0/2610114038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006bc30 0x7f822015b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.635+0000 7f8227f6e700 1 -- 172.21.15.6:0/2610114038 >> 172.21.15.6:0/2610114038 conn(0x7f82200b0dd0 msgr2=0x7f82200b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:39.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.635+0000 7f8227f6e700 1 -- 172.21.15.6:0/2610114038 shutdown_connections 2023-11-28T13:00:39.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.635+0000 7f8227f6e700 1 -- 172.21.15.6:0/2610114038 wait complete. 2023-11-28T13:00:39.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.636+0000 7f8227f6e700 1 Processor -- start 2023-11-28T13:00:39.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.636+0000 7f8227f6e700 1 -- start start 2023-11-28T13:00:39.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f8227f6e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f822006bc30 0x7f82201f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f8227f6e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006d9d0 0x7f822015c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f8227f6e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f822006dda0 0x7f82200b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f8227f6e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201f3710 con 0x7f822006dda0 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f8227f6e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201f39b0 con 0x7f822006bc30 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f8227f6e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82201f3c90 con 0x7f822006d9d0 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f822650b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f822006dda0 0x7f82200b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f822650b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f822006dda0 0x7f82200b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58082/0 (socket says 172.21.15.6:58082) 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f822650b700 1 -- 172.21.15.6:0/2298440346 learned_addr learned my addr 172.21.15.6:0/2298440346 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f8225d0a700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f822006bc30 0x7f82201f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f8225509700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006d9d0 0x7f822015c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f822650b700 1 -- 172.21.15.6:0/2298440346 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006d9d0 msgr2=0x7f822015c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f822650b700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006d9d0 0x7f822015c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f822650b700 1 -- 172.21.15.6:0/2298440346 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f822006bc30 msgr2=0x7f82201f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f822650b700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f822006bc30 0x7f82201f31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f822650b700 1 -- 172.21.15.6:0/2298440346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8218009a70 con 0x7f822006dda0 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f822650b700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f822006dda0 0x7f82200b2110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8218014040 tx=0x7f8218004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.637+0000 7f82167fc700 1 -- 172.21.15.6:0/2298440346 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218012430 con 0x7f822006dda0 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.638+0000 7f82167fc700 1 -- 172.21.15.6:0/2298440346 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8218020070 con 0x7f822006dda0 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.638+0000 7f82167fc700 1 -- 172.21.15.6:0/2298440346 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218003ea0 con 0x7f822006dda0 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.638+0000 7f8227f6e700 1 -- 172.21.15.6:0/2298440346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822015b120 con 0x7f822006dda0 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.638+0000 7f8227f6e700 1 -- 172.21.15.6:0/2298440346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822015ae40 con 0x7f822006dda0 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.638+0000 7f8227f6e700 1 -- 172.21.15.6:0/2298440346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822006fac0 con 0x7f822006dda0 2023-11-28T13:00:39.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.639+0000 7f82167fc700 1 -- 172.21.15.6:0/2298440346 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f821802e070 con 0x7f822006dda0 2023-11-28T13:00:39.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.639+0000 7f82167fc700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82040682f0 0x7f820406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:39.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.639+0000 7f82167fc700 1 -- 172.21.15.6:0/2298440346 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8218022070 con 0x7f822006dda0 2023-11-28T13:00:39.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.639+0000 7f8225d0a700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82040682f0 0x7f820406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:39.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.640+0000 7f8225d0a700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82040682f0 0x7f820406a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8210005910 tx=0x7f8210009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:39.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.644+0000 7f82167fc700 1 -- 172.21.15.6:0/2298440346 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8218054280 con 0x7f822006dda0 2023-11-28T13:00:39.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.894+0000 7f8227f6e700 1 -- 172.21.15.6:0/2298440346 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f82200675a0 con 0x7f822006dda0 2023-11-28T13:00:39.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.897+0000 7f82167fc700 1 -- 172.21.15.6:0/2298440346 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166369 (secure 0 0 0) 0x7f821802c090 con 0x7f822006dda0 2023-11-28T13:00:39.900 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:39.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 -- 172.21.15.6:0/2298440346 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82040682f0 msgr2=0x7f820406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82040682f0 0x7f820406a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8210005910 tx=0x7f8210009040 comp rx=0 tx=0).stop 2023-11-28T13:00:39.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 -- 172.21.15.6:0/2298440346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f822006dda0 msgr2=0x7f82200b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:39.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f822006dda0 0x7f82200b2110 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8218014040 tx=0x7f8218004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 -- 172.21.15.6:0/2298440346 shutdown_connections 2023-11-28T13:00:39.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f822006dda0 0x7f82200b2110 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f822006bc30 0x7f82201f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82040682f0 0x7f820406a7a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 --2- 172.21.15.6:0/2298440346 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f822006d9d0 0x7f822015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:39.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 -- 172.21.15.6:0/2298440346 >> 172.21.15.6:0/2298440346 conn(0x7f82200b0dd0 msgr2=0x7f8220153250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:39.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 -- 172.21.15.6:0/2298440346 shutdown_connections 2023-11-28T13:00:39.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:39.900+0000 7f8227f6e700 1 -- 172.21.15.6:0/2298440346 wait complete. 2023-11-28T13:00:39.914 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":103231},{"type":"v1","addr":"172.21.15.6:6833","nonce":103231}]},"active_addr":"172.21.15.6:6833/103231","active_change":"2023-11-28T13:00:08.936005+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":3275070463}]},{"addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":682687250}]},{"addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":2841403999}]},{"addrvec":[{"type":"v2","addr":"172.21.15.6:0","nonce":52620840}]}]} 2023-11-28T13:00:39.917 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T13:00:39.917 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T13:00:39.917 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:40.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.041+0000 7ff45b67e700 1 Processor -- start 2023-11-28T13:00:40.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.041+0000 7ff45b67e700 1 -- start start 2023-11-28T13:00:40.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.042+0000 7ff45b67e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff45415c990 0x7ff4540cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.042+0000 7ff45b67e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff454161090 0x7ff4540ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.042+0000 7ff45b67e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4540ce340 con 0x7ff4540caeb0 2023-11-28T13:00:40.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.042+0000 7ff45b67e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4540cebe0 con 0x7ff454161090 2023-11-28T13:00:40.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.042+0000 7ff45b67e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540cb3f0 con 0x7ff45415c990 2023-11-28T13:00:40.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff458c19700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff454161090 0x7ff4540ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff45941a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff45415c990 0x7ff4540cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff459c1b700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7ff4540caeb0 0x7ff4540cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:35440/0 (socket says 172.21.15.6:35440) 2023-11-28T13:00:40.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff45941a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff45415c990 0x7ff4540cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47790/0 (socket says 172.21.15.6:47790) 2023-11-28T13:00:40.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff458c19700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff454161090 0x7ff4540ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:33574/0 (socket says 172.21.15.6:33574) 2023-11-28T13:00:40.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff459c1b700 1 -- 172.21.15.6:0/863406385 learned_addr learned my addr 172.21.15.6:0/863406385 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:40.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff45941a700 1 -- 172.21.15.6:0/863406385 >> v2:172.21.15.61:3300/0 conn(0x7ff454161090 msgr2=0x7ff4540ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:40.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff45941a700 1 --2- 172.21.15.6:0/863406385 >> v2:172.21.15.61:3300/0 conn(0x7ff454161090 0x7ff4540ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff45941a700 1 -- 172.21.15.6:0/863406385 >> v1:172.21.15.61:6789/0 conn(0x7ff4540caeb0 legacy=0x7ff4540cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff45941a700 1 -- 172.21.15.6:0/863406385 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4540ce590 con 0x7ff45415c990 2023-11-28T13:00:40.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff458c19700 1 --2- 172.21.15.6:0/863406385 >> v2:172.21.15.61:3300/0 conn(0x7ff454161090 0x7ff4540ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:40.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff45941a700 1 --2- 172.21.15.6:0/863406385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff45415c990 0x7ff4540cf070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff444009840 tx=0x7ff444009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f46047f3e8ba19f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff44bfff700 1 -- 172.21.15.6:0/863406385 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444011830 con 0x7ff45415c990 2023-11-28T13:00:40.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.043+0000 7ff44bfff700 1 -- 172.21.15.6:0/863406385 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff444011990 con 0x7ff45415c990 2023-11-28T13:00:40.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.044+0000 7ff44bfff700 1 -- 172.21.15.6:0/863406385 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444011d20 con 0x7ff45415c990 2023-11-28T13:00:40.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.044+0000 7ff45b67e700 1 -- 172.21.15.6:0/863406385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff45415c990 msgr2=0x7ff4540cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.044+0000 7ff45b67e700 1 --2- 172.21.15.6:0/863406385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff45415c990 0x7ff4540cf070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff444009840 tx=0x7ff444009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:40.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.044+0000 7ff45b67e700 1 -- 172.21.15.6:0/863406385 shutdown_connections 2023-11-28T13:00:40.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.044+0000 7ff45b67e700 1 --2- 172.21.15.6:0/863406385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff45415c990 0x7ff4540cf070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.044+0000 7ff45b67e700 1 --2- 172.21.15.6:0/863406385 >> v2:172.21.15.61:3300/0 conn(0x7ff454161090 0x7ff4540ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.044+0000 7ff45b67e700 1 -- 172.21.15.6:0/863406385 >> 172.21.15.6:0/863406385 conn(0x7ff4540b1650 msgr2=0x7ff4540b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.044+0000 7ff45b67e700 1 -- 172.21.15.6:0/863406385 shutdown_connections 2023-11-28T13:00:40.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.044+0000 7ff45b67e700 1 -- 172.21.15.6:0/863406385 wait complete. 2023-11-28T13:00:40.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.045+0000 7ff45b67e700 1 Processor -- start 2023-11-28T13:00:40.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.045+0000 7ff45b67e700 1 -- start start 2023-11-28T13:00:40.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff45b67e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4540caeb0 0x7ff4541f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff45b67e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff45415c990 0x7ff45415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff45b67e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff454161090 0x7ff4540ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff45b67e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541f3380 con 0x7ff454161090 2023-11-28T13:00:40.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff45b67e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540af090 con 0x7ff45415c990 2023-11-28T13:00:40.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff45b67e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540adff0 con 0x7ff4540caeb0 2023-11-28T13:00:40.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff459c1b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff454161090 0x7ff4540ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff459c1b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff454161090 0x7ff4540ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58096/0 (socket says 172.21.15.6:58096) 2023-11-28T13:00:40.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff459c1b700 1 -- 172.21.15.6:0/2506372581 learned_addr learned my addr 172.21.15.6:0/2506372581 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff458c19700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff45415c990 0x7ff45415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.046+0000 7ff45941a700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4540caeb0 0x7ff4541f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff459c1b700 1 -- 172.21.15.6:0/2506372581 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4540caeb0 msgr2=0x7ff4541f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff459c1b700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4540caeb0 0x7ff4541f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff459c1b700 1 -- 172.21.15.6:0/2506372581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff45415c990 msgr2=0x7ff45415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff459c1b700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff45415c990 0x7ff45415afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff459c1b700 1 -- 172.21.15.6:0/2506372581 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4500094a0 con 0x7ff454161090 2023-11-28T13:00:40.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff459c1b700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff454161090 0x7ff4540ab8c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00cbc0 tx=0x7ff44c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff449ffb700 1 -- 172.21.15.6:0/2506372581 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c00a9e0 con 0x7ff454161090 2023-11-28T13:00:40.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff449ffb700 1 -- 172.21.15.6:0/2506372581 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c00ab40 con 0x7ff454161090 2023-11-28T13:00:40.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff45b67e700 1 -- 172.21.15.6:0/2506372581 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4440094a0 con 0x7ff454161090 2023-11-28T13:00:40.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff449ffb700 1 -- 172.21.15.6:0/2506372581 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c012400 con 0x7ff454161090 2023-11-28T13:00:40.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.047+0000 7ff45b67e700 1 -- 172.21.15.6:0/2506372581 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4540cf5b0 con 0x7ff454161090 2023-11-28T13:00:40.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.048+0000 7ff45b67e700 1 -- 172.21.15.6:0/2506372581 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4540ce870 con 0x7ff454161090 2023-11-28T13:00:40.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.049+0000 7ff449ffb700 1 -- 172.21.15.6:0/2506372581 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff44c019030 con 0x7ff454161090 2023-11-28T13:00:40.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.049+0000 7ff449ffb700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4380682f0 0x7ff43806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.049+0000 7ff449ffb700 1 -- 172.21.15.6:0/2506372581 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff44c085980 con 0x7ff454161090 2023-11-28T13:00:40.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.049+0000 7ff45941a700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4380682f0 0x7ff43806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.050+0000 7ff45941a700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4380682f0 0x7ff43806a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff444009fd0 tx=0x7ff444008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.053+0000 7ff449ffb700 1 -- 172.21.15.6:0/2506372581 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff44c04b5e0 con 0x7ff454161090 2023-11-28T13:00:40.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.250+0000 7ff45b67e700 1 -- 172.21.15.6:0/2506372581 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4540cf740 con 0x7ff454161090 2023-11-28T13:00:40.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.252+0000 7ff449ffb700 1 -- 172.21.15.6:0/2506372581 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7ff44c051fa0 con 0x7ff454161090 2023-11-28T13:00:40.253 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:40.253 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.253+0000 7ff45b67e700 1 -- 172.21.15.6:0/2506372581 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4380682f0 msgr2=0x7ff43806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.253+0000 7ff45b67e700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4380682f0 0x7ff43806a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff444009fd0 tx=0x7ff444008030 comp rx=0 tx=0).stop 2023-11-28T13:00:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.253+0000 7ff45b67e700 1 -- 172.21.15.6:0/2506372581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff454161090 msgr2=0x7ff4540ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.253+0000 7ff45b67e700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff454161090 0x7ff4540ab8c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00cbc0 tx=0x7ff44c00ced0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.254+0000 7ff45b67e700 1 -- 172.21.15.6:0/2506372581 shutdown_connections 2023-11-28T13:00:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.254+0000 7ff45b67e700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff454161090 0x7ff4540ab8c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.254+0000 7ff45b67e700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4380682f0 0x7ff43806a7a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.254+0000 7ff45b67e700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4540caeb0 0x7ff4541f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.254+0000 7ff45b67e700 1 --2- 172.21.15.6:0/2506372581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff45415c990 0x7ff45415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.254+0000 7ff45b67e700 1 -- 172.21.15.6:0/2506372581 >> 172.21.15.6:0/2506372581 conn(0x7ff4540b1650 msgr2=0x7ff4540c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.254+0000 7ff45b67e700 1 -- 172.21.15.6:0/2506372581 shutdown_connections 2023-11-28T13:00:40.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.254+0000 7ff45b67e700 1 -- 172.21.15.6:0/2506372581 wait complete. 2023-11-28T13:00:40.267 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T13:00:40.267 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.390+0000 7f46e6b44700 1 Processor -- start 2023-11-28T13:00:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.391+0000 7f46e6b44700 1 -- start start 2023-11-28T13:00:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.391+0000 7f46e6b44700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f46e006e400 0x7f46e015db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.391+0000 7f46e6b44700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46e00ce890 0x7f46e00ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.391+0000 7f46e6b44700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e015e0b0 con 0x7f46e0159590 2023-11-28T13:00:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.391+0000 7f46e6b44700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e00cf250 con 0x7f46e006e400 2023-11-28T13:00:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.391+0000 7f46e6b44700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e015d120 con 0x7f46e00ce890 2023-11-28T13:00:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.391+0000 7f46dffff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f46e006e400 0x7f46e015db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.391+0000 7f46e48e0700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f46e0159590 0x7f46e00ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45420/0 (socket says 172.21.15.6:45420) 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46dffff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f46e006e400 0x7f46e015db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58100/0 (socket says 172.21.15.6:58100) 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46e48e0700 1 -- 172.21.15.6:0/1840776535 learned_addr learned my addr 172.21.15.6:0/1840776535 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46e50e1700 1 --2- 172.21.15.6:0/1840776535 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46e00ce890 0x7f46e00ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46dffff700 1 -- 172.21.15.6:0/1840776535 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46e00ce890 msgr2=0x7f46e00ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46dffff700 1 --2- 172.21.15.6:0/1840776535 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46e00ce890 0x7f46e00ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46dffff700 1 -- 172.21.15.6:0/1840776535 >> v1:172.21.15.6:6789/0 conn(0x7f46e0159590 legacy=0x7f46e00ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46dffff700 1 -- 172.21.15.6:0/1840776535 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e015d6e0 con 0x7f46e006e400 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46df7fe700 1 -- 172.21.15.6:0/1840776535 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003510090 0 0) 0x7f46d4001110 con 0x7f46e0159590 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46dffff700 1 --2- 172.21.15.6:0/1840776535 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e006e400 0x7f46e015db70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00bae0 tx=0x7f46cc00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabb4cb132668113 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46df7fe700 1 -- 172.21.15.6:0/1840776535 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00dd30 con 0x7f46e006e400 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46df7fe700 1 -- 172.21.15.6:0/1840776535 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc00de90 con 0x7f46e006e400 2023-11-28T13:00:40.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.392+0000 7f46df7fe700 1 -- 172.21.15.6:0/1840776535 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc010690 con 0x7f46e006e400 2023-11-28T13:00:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.393+0000 7f46e6b44700 1 -- 172.21.15.6:0/1840776535 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e006e400 msgr2=0x7f46e015db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.393+0000 7f46e6b44700 1 --2- 172.21.15.6:0/1840776535 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e006e400 0x7f46e015db70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00bae0 tx=0x7f46cc00bdf0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.393+0000 7f46e6b44700 1 -- 172.21.15.6:0/1840776535 shutdown_connections 2023-11-28T13:00:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.393+0000 7f46e6b44700 1 --2- 172.21.15.6:0/1840776535 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46e00ce890 0x7f46e00ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.393+0000 7f46e6b44700 1 --2- 172.21.15.6:0/1840776535 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e006e400 0x7f46e015db70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.393+0000 7f46e6b44700 1 -- 172.21.15.6:0/1840776535 >> 172.21.15.6:0/1840776535 conn(0x7f46e00b1650 msgr2=0x7f46e00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.393+0000 7f46e6b44700 1 -- 172.21.15.6:0/1840776535 shutdown_connections 2023-11-28T13:00:40.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.393+0000 7f46e6b44700 1 -- 172.21.15.6:0/1840776535 wait complete. 2023-11-28T13:00:40.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.394+0000 7f46e6b44700 1 Processor -- start 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.394+0000 7f46e6b44700 1 -- start start 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e6b44700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f46e006e400 0x7f46e00c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e6b44700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46e00ce890 0x7f46e00b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e6b44700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e0159590 0x7f46e00b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e6b44700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00c5050 con 0x7f46e0159590 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e6b44700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00c3fb0 con 0x7f46e006e400 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e6b44700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00c4290 con 0x7f46e00ce890 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e50e1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e0159590 0x7f46e00b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e50e1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e0159590 0x7f46e00b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58110/0 (socket says 172.21.15.6:58110) 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e50e1700 1 -- 172.21.15.6:0/3234267824 learned_addr learned my addr 172.21.15.6:0/3234267824 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46dffff700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46e00ce890 0x7f46e00b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e48e0700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f46e006e400 0x7f46e00c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e50e1700 1 -- 172.21.15.6:0/3234267824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46e00ce890 msgr2=0x7f46e00b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e50e1700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46e00ce890 0x7f46e00b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e50e1700 1 -- 172.21.15.6:0/3234267824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f46e006e400 msgr2=0x7f46e00c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e50e1700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f46e006e400 0x7f46e00c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.395+0000 7f46e50e1700 1 -- 172.21.15.6:0/3234267824 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc00b740 con 0x7f46e0159590 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.396+0000 7f46e50e1700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e0159590 0x7f46e00b3ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46d8007e40 tx=0x7f46d800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.396+0000 7f46dd7fa700 1 -- 172.21.15.6:0/3234267824 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d800ed30 con 0x7f46e0159590 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.396+0000 7f46dd7fa700 1 -- 172.21.15.6:0/3234267824 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d8014420 con 0x7f46e0159590 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.396+0000 7f46e6b44700 1 -- 172.21.15.6:0/3234267824 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e00af240 con 0x7f46e0159590 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.396+0000 7f46dd7fa700 1 -- 172.21.15.6:0/3234267824 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d80147d0 con 0x7f46e0159590 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.396+0000 7f46e6b44700 1 -- 172.21.15.6:0/3234267824 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e00aef60 con 0x7f46e0159590 2023-11-28T13:00:40.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.397+0000 7f46e6b44700 1 -- 172.21.15.6:0/3234267824 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e015d370 con 0x7f46e0159590 2023-11-28T13:00:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.397+0000 7f46dd7fa700 1 -- 172.21.15.6:0/3234267824 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f46d8012050 con 0x7f46e0159590 2023-11-28T13:00:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.398+0000 7f46dd7fa700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46c40682f0 0x7f46c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.398+0000 7f46dd7fa700 1 -- 172.21.15.6:0/3234267824 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f46d8086500 con 0x7f46e0159590 2023-11-28T13:00:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.398+0000 7f46e48e0700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46c40682f0 0x7f46c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.398+0000 7f46e48e0700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46c40682f0 0x7f46c406a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f46d4000f60 tx=0x7f46d4007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.402+0000 7f46dd7fa700 1 -- 172.21.15.6:0/3234267824 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f46d8052a70 con 0x7f46e0159590 2023-11-28T13:00:40.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.595+0000 7f46e6b44700 1 -- 172.21.15.6:0/3234267824 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46e00675a0 con 0x7f46e0159590 2023-11-28T13:00:40.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.596+0000 7f46dd7fa700 1 -- 172.21.15.6:0/3234267824 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f46d8056090 con 0x7f46e0159590 2023-11-28T13:00:40.597 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:40.598 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 -- 172.21.15.6:0/3234267824 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46c40682f0 msgr2=0x7f46c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46c40682f0 0x7f46c406a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f46d4000f60 tx=0x7f46d4007e70 comp rx=0 tx=0).stop 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 -- 172.21.15.6:0/3234267824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e0159590 msgr2=0x7f46e00b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e0159590 0x7f46e00b3ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46d8007e40 tx=0x7f46d800e3c0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 -- 172.21.15.6:0/3234267824 shutdown_connections 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f46e0159590 0x7f46e00b3ee0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f46e006e400 0x7f46e00c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46c40682f0 0x7f46c406a7a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 --2- 172.21.15.6:0/3234267824 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46e00ce890 0x7f46e00b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 -- 172.21.15.6:0/3234267824 >> 172.21.15.6:0/3234267824 conn(0x7f46e00b1650 msgr2=0x7f46e00cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 -- 172.21.15.6:0/3234267824 shutdown_connections 2023-11-28T13:00:40.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.598+0000 7f46e6b44700 1 -- 172.21.15.6:0/3234267824 wait complete. 2023-11-28T13:00:40.612 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T13:00:40.613 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T13:00:40.613 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T13:00:40.613 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T13:00:40.614 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T13:00:40.614 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T13:00:40.614 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T13:00:40.615 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T13:00:40.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.831+0000 7f1497c59700 1 Processor -- start 2023-11-28T13:00:40.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.831+0000 7f1497c59700 1 -- start start 2023-11-28T13:00:40.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.832+0000 7fea129d6700 1 Processor -- start 2023-11-28T13:00:40.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.832+0000 7f1497c59700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f149015a230 0x7f149015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.832+0000 7f1497c59700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f14900b4fe0 0x7f14900d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.832+0000 7f1497c59700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149006abb0 con 0x7f14900cd4a0 2023-11-28T13:00:40.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.832+0000 7f1497c59700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149015a600 con 0x7f14900b4fe0 2023-11-28T13:00:40.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.832+0000 7f1497c59700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149006a280 con 0x7f149015a230 2023-11-28T13:00:40.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.833+0000 7f14951f4700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f14900cd4a0 0x7f14900b48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45434/0 (socket says 172.21.15.6:45434) 2023-11-28T13:00:40.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.833+0000 7f14951f4700 1 -- 172.21.15.6:0/4216987851 learned_addr learned my addr 172.21.15.6:0/4216987851 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:40.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.833+0000 7f14961f6700 1 --2- 172.21.15.6:0/4216987851 >> v2:172.21.15.6:3300/0 conn(0x7f14900b4fe0 0x7f14900d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.833+0000 7fea129d6700 1 -- start start 2023-11-28T13:00:40.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.833+0000 7f14961f6700 1 -- 172.21.15.6:0/4216987851 >> v2:172.21.15.61:3300/0 conn(0x7f149015a230 msgr2=0x7f149015d6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T13:00:40.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.833+0000 7f14961f6700 1 --2- 172.21.15.6:0/4216987851 >> v2:172.21.15.61:3300/0 conn(0x7f149015a230 0x7f149015d6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.833+0000 7f14961f6700 1 -- 172.21.15.6:0/4216987851 >> v1:172.21.15.6:6789/0 conn(0x7f14900cd4a0 legacy=0x7f14900b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.833+0000 7f14961f6700 1 -- 172.21.15.6:0/4216987851 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149006a560 con 0x7f14900b4fe0 2023-11-28T13:00:40.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.834+0000 7fea129d6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fea0c160ac0 0x7fea0c0d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.834+0000 7fea129d6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fea0c164510 0x7fea0c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.834+0000 7fea129d6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0c06a1e0 con 0x7fea0c0b48d0 2023-11-28T13:00:40.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.834+0000 7fea129d6700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0c160e90 con 0x7fea0c164510 2023-11-28T13:00:40.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.834+0000 7fea129d6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea0c1649f0 con 0x7fea0c160ac0 2023-11-28T13:00:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.834+0000 7f14961f6700 1 --2- 172.21.15.6:0/4216987851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14900b4fe0 0x7f14900d0a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14900cf830 tx=0x7f148c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be058187bf2066 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.834+0000 7fea0b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fea0c164510 0x7fea0c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.834+0000 7fea0b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fea0c164510 0x7fea0c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58120/0 (socket says 172.21.15.6:58120) 2023-11-28T13:00:40.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.834+0000 7fea0b7fe700 1 -- 172.21.15.6:0/2497340066 learned_addr learned my addr 172.21.15.6:0/2497340066 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.835+0000 7fea0bfff700 1 --2- 172.21.15.6:0/2497340066 >> v2:172.21.15.61:3300/0 conn(0x7fea0c160ac0 0x7fea0c0d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.835+0000 7fea0b7fe700 1 -- 172.21.15.6:0/2497340066 >> v2:172.21.15.61:3300/0 conn(0x7fea0c160ac0 msgr2=0x7fea0c0d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:40.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.835+0000 7fea0b7fe700 1 --2- 172.21.15.6:0/2497340066 >> v2:172.21.15.61:3300/0 conn(0x7fea0c160ac0 0x7fea0c0d2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.835+0000 7fea0b7fe700 1 -- 172.21.15.6:0/2497340066 >> v1:172.21.15.61:6789/0 conn(0x7fea0c0b48d0 legacy=0x7fea0c06a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:40.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.835+0000 7fea0b7fe700 1 -- 172.21.15.6:0/2497340066 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c164cd0 con 0x7fea0c164510 2023-11-28T13:00:40.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.835+0000 7fea0b7fe700 1 --2- 172.21.15.6:0/2497340066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea0c164510 0x7fea0c0b5da0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea0c08c040 tx=0x7fea00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a4aaa9d0f16aaa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.835+0000 7f35111da700 1 Processor -- start 2023-11-28T13:00:40.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.835+0000 7f35111da700 1 -- start start 2023-11-28T13:00:40.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.836+0000 7f14949f3700 1 -- 172.21.15.6:0/4216987851 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c00e070 con 0x7f14900b4fe0 2023-11-28T13:00:40.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.836+0000 7f14949f3700 1 -- 172.21.15.6:0/4216987851 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c00bad0 con 0x7f14900b4fe0 2023-11-28T13:00:40.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.836+0000 7f14949f3700 1 -- 172.21.15.6:0/4216987851 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c00be80 con 0x7f14900b4fe0 2023-11-28T13:00:40.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.836+0000 7f35111da700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f350c160c70 0x7f350c0adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.836+0000 7f35111da700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f350c1646c0 0x7f350c070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f35111da700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350c164b30 con 0x7f350c070ea0 2023-11-28T13:00:40.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f35111da700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350c071340 con 0x7f350c1646c0 2023-11-28T13:00:40.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f35111da700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c161040 con 0x7f350c160c70 2023-11-28T13:00:40.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f1497c59700 1 -- 172.21.15.6:0/4216987851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14900b4fe0 msgr2=0x7f14900d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f1497c59700 1 --2- 172.21.15.6:0/4216987851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14900b4fe0 0x7f14900d0a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14900cf830 tx=0x7f148c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:40.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f1497c59700 1 -- 172.21.15.6:0/4216987851 shutdown_connections 2023-11-28T13:00:40.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f1497c59700 1 --2- 172.21.15.6:0/4216987851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14900b4fe0 0x7f14900d0a20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f1497c59700 1 --2- 172.21.15.6:0/4216987851 >> v2:172.21.15.61:3300/0 conn(0x7f149015a230 0x7f149015d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f1497c59700 1 -- 172.21.15.6:0/4216987851 >> 172.21.15.6:0/4216987851 conn(0x7f14900cf8d0 msgr2=0x7f14900cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f1497c59700 1 -- 172.21.15.6:0/4216987851 shutdown_connections 2023-11-28T13:00:40.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f1497c59700 1 -- 172.21.15.6:0/4216987851 wait complete. 2023-11-28T13:00:40.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.836+0000 7fea0affd700 1 -- 172.21.15.6:0/2497340066 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0000b870 con 0x7fea0c164510 2023-11-28T13:00:40.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.836+0000 7fea0affd700 1 -- 172.21.15.6:0/2497340066 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0000b9d0 con 0x7fea0c164510 2023-11-28T13:00:40.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.836+0000 7fea0affd700 1 -- 172.21.15.6:0/2497340066 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0000bd60 con 0x7fea0c164510 2023-11-28T13:00:40.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f350b59e700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f350c070ea0 0x7f350c06a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:35476/0 (socket says 172.21.15.6:35476) 2023-11-28T13:00:40.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f350b59e700 1 -- 172.21.15.6:0/3405784841 learned_addr learned my addr 172.21.15.6:0/3405784841 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:40.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f350259c700 1 --2- 172.21.15.6:0/3405784841 >> v2:172.21.15.6:3300/0 conn(0x7f350c1646c0 0x7f350c070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f350259c700 1 -- 172.21.15.6:0/3405784841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f350c160c70 msgr2=0x7f350c0adb00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T13:00:40.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f350259c700 1 --2- 172.21.15.6:0/3405784841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f350c160c70 0x7f350c0adb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f350259c700 1 -- 172.21.15.6:0/3405784841 >> v1:172.21.15.61:6789/0 conn(0x7f350c070ea0 legacy=0x7f350c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.837+0000 7f350259c700 1 -- 172.21.15.6:0/3405784841 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c164e10 con 0x7f350c1646c0 2023-11-28T13:00:40.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.838+0000 7f350259c700 1 --2- 172.21.15.6:0/3405784841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f350c1646c0 0x7f350c070960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009870 tx=0x7f34f4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed141f1d53c0dc59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.838+0000 7fea129d6700 1 -- 172.21.15.6:0/2497340066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea0c164510 msgr2=0x7fea0c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.838+0000 7fea129d6700 1 --2- 172.21.15.6:0/2497340066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea0c164510 0x7fea0c0b5da0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea0c08c040 tx=0x7fea00009bd0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.838+0000 7f350a59c700 1 -- 172.21.15.6:0/3405784841 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4004230 con 0x7f350c1646c0 2023-11-28T13:00:40.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.838+0000 7f350a59c700 1 -- 172.21.15.6:0/3405784841 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f4004390 con 0x7f350c1646c0 2023-11-28T13:00:40.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.838+0000 7f350a59c700 1 -- 172.21.15.6:0/3405784841 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f40036a0 con 0x7f350c1646c0 2023-11-28T13:00:40.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.838+0000 7fea129d6700 1 -- 172.21.15.6:0/2497340066 shutdown_connections 2023-11-28T13:00:40.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.838+0000 7fea129d6700 1 --2- 172.21.15.6:0/2497340066 >> v2:172.21.15.61:3300/0 conn(0x7fea0c160ac0 0x7fea0c0d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.838+0000 7fea129d6700 1 --2- 172.21.15.6:0/2497340066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea0c164510 0x7fea0c0b5da0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.838+0000 7fea129d6700 1 -- 172.21.15.6:0/2497340066 >> 172.21.15.6:0/2497340066 conn(0x7fea0c0cf8d0 msgr2=0x7fea0c0cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.839+0000 7fea129d6700 1 -- 172.21.15.6:0/2497340066 shutdown_connections 2023-11-28T13:00:40.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.839+0000 7f1497c59700 1 Processor -- start 2023-11-28T13:00:40.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.839+0000 7f1497c59700 1 -- start start 2023-11-28T13:00:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.839+0000 7f35111da700 1 -- 172.21.15.6:0/3405784841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f350c1646c0 msgr2=0x7f350c070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.839+0000 7f35111da700 1 --2- 172.21.15.6:0/3405784841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f350c1646c0 0x7f350c070960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009870 tx=0x7f34f4009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f35111da700 1 -- 172.21.15.6:0/3405784841 shutdown_connections 2023-11-28T13:00:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f35111da700 1 --2- 172.21.15.6:0/3405784841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f350c160c70 0x7f350c0adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f35111da700 1 --2- 172.21.15.6:0/3405784841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f350c1646c0 0x7f350c070960 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f35111da700 1 -- 172.21.15.6:0/3405784841 >> 172.21.15.6:0/3405784841 conn(0x7f350c0ce9e0 msgr2=0x7f350c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.839+0000 7fea129d6700 1 -- 172.21.15.6:0/2497340066 wait complete. 2023-11-28T13:00:40.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f1497c59700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14900b4fe0 0x7f149014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f1497c59700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14900cd4a0 0x7f14901f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f1497c59700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f149015a230 0x7f14900c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f1497c59700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900b7d10 con 0x7f14900b4fe0 2023-11-28T13:00:40.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f1497c59700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14900b7100 con 0x7f149015a230 2023-11-28T13:00:40.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f1497c59700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14900b73e0 con 0x7f14900cd4a0 2023-11-28T13:00:40.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f14959f5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14900b4fe0 0x7f149014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f14959f5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14900b4fe0 0x7f149014eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58132/0 (socket says 172.21.15.6:58132) 2023-11-28T13:00:40.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.840+0000 7f14959f5700 1 -- 172.21.15.6:0/3774654808 learned_addr learned my addr 172.21.15.6:0/3774654808 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7f35111da700 1 -- 172.21.15.6:0/3405784841 shutdown_connections 2023-11-28T13:00:40.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7f35111da700 1 -- 172.21.15.6:0/3405784841 wait complete. 2023-11-28T13:00:40.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7fea129d6700 1 Processor -- start 2023-11-28T13:00:40.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7f14951f4700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14900cd4a0 0x7f14901f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7f14959f5700 1 -- 172.21.15.6:0/3774654808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14900cd4a0 msgr2=0x7f14901f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7f14959f5700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14900cd4a0 0x7f14901f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7f14959f5700 1 -- 172.21.15.6:0/3774654808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f149015a230 msgr2=0x7f14900c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7f14959f5700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f149015a230 0x7f14900c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7f14959f5700 1 -- 172.21.15.6:0/3774654808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c009a70 con 0x7f14900b4fe0 2023-11-28T13:00:40.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7f14959f5700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14900b4fe0 0x7f149014eef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f148000b4a0 tx=0x7f148000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7f38d5fb8700 1 Processor -- start 2023-11-28T13:00:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7f38d5fb8700 1 -- start start 2023-11-28T13:00:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7f38d5fb8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 0x7f38d006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7f38d5fb8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f38d006b860 0x7f38d00c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7f38d5fb8700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d006a280 con 0x7f38d01588a0 2023-11-28T13:00:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7f38d5fb8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d00ce720 con 0x7f38d006b860 2023-11-28T13:00:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7f38d5fb8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d006a560 con 0x7f38d00cd120 2023-11-28T13:00:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.841+0000 7fea129d6700 1 -- start start 2023-11-28T13:00:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7fea129d6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea0c0b48d0 0x7fea0c1febc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7fea129d6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea0c160ac0 0x7fea0c0cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7fea129d6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea0c164510 0x7fea0c0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7fea129d6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c1614a0 con 0x7fea0c160ac0 2023-11-28T13:00:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7fea129d6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c161780 con 0x7fea0c0b48d0 2023-11-28T13:00:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7fea129d6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0c161a60 con 0x7fea0c164510 2023-11-28T13:00:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7fea10f73700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea0c164510 0x7fea0c0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7fea10f73700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea0c164510 0x7fea0c0b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47844/0 (socket says 172.21.15.6:47844) 2023-11-28T13:00:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7fea10f73700 1 -- 172.21.15.6:0/2106906386 learned_addr learned my addr 172.21.15.6:0/2106906386 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7f38cffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f38d006b860 0x7f38d00c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7f38cffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f38d006b860 0x7f38d00c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:33624/0 (socket says 172.21.15.6:33624) 2023-11-28T13:00:40.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7f38cffff700 1 -- 172.21.15.6:0/129503841 learned_addr learned my addr 172.21.15.6:0/129503841 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7f14867fc700 1 -- 172.21.15.6:0/3774654808 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480011830 con 0x7f14900b4fe0 2023-11-28T13:00:40.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7f14867fc700 1 -- 172.21.15.6:0/3774654808 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14800119b0 con 0x7f14900b4fe0 2023-11-28T13:00:40.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7f14867fc700 1 -- 172.21.15.6:0/3774654808 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480011d60 con 0x7f14900b4fe0 2023-11-28T13:00:40.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14900cd920 con 0x7f14900b4fe0 2023-11-28T13:00:40.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149006a840 con 0x7f14900b4fe0 2023-11-28T13:00:40.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.842+0000 7f35111da700 1 Processor -- start 2023-11-28T13:00:40.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7f38ceffd700 1 --2- 172.21.15.6:0/129503841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 0x7f38d006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7fea0bfff700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea0c0b48d0 0x7fea0c1febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7f38cffff700 1 -- 172.21.15.6:0/129503841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 msgr2=0x7f38d006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:40.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7f38cffff700 1 --2- 172.21.15.6:0/129503841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 0x7f38d006b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7f38cffff700 1 -- 172.21.15.6:0/129503841 >> v1:172.21.15.6:6789/0 conn(0x7f38d01588a0 legacy=0x7f38d00ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7f38cffff700 1 -- 172.21.15.6:0/129503841 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d006a840 con 0x7f38d006b860 2023-11-28T13:00:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.843+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1478000fc0 con 0x7f14900b4fe0 2023-11-28T13:00:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7f38cffff700 1 --2- 172.21.15.6:0/129503841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38d006b860 0x7f38d00c9920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009310 tx=0x7f38b8009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87ca3e47be56edff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7f35111da700 1 -- start start 2023-11-28T13:00:40.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7fea0bfff700 1 -- 172.21.15.6:0/2106906386 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea0c164510 msgr2=0x7fea0c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7fea0bfff700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fea0c164510 0x7fea0c0b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7fea0bfff700 1 -- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea0c160ac0 msgr2=0x7fea0c0cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7fea0bfff700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea0c160ac0 0x7fea0c0cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7fea0bfff700 1 -- 172.21.15.6:0/2106906386 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea00009550 con 0x7fea0c0b48d0 2023-11-28T13:00:40.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7fea0bfff700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea0c0b48d0 0x7fea0c1febc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00e7e0 tx=0x7fe9fc00eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.844+0000 7fea08ff9700 1 -- 172.21.15.6:0/2106906386 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00c070 con 0x7fea0c0b48d0 2023-11-28T13:00:40.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7fea08ff9700 1 -- 172.21.15.6:0/2106906386 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc013610 con 0x7fea0c0b48d0 2023-11-28T13:00:40.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7fea08ff9700 1 -- 172.21.15.6:0/2106906386 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc013980 con 0x7fea0c0b48d0 2023-11-28T13:00:40.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c165290 con 0x7fea0c0b48d0 2023-11-28T13:00:40.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0c164fb0 con 0x7fea0c0b48d0 2023-11-28T13:00:40.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f35111da700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f350c070ea0 0x7f350c155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f35111da700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f350c160c70 0x7f350c0b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f35111da700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f350c1646c0 0x7f350c1fa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f35111da700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0b3cd0 con 0x7f350c1646c0 2023-11-28T13:00:40.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f35111da700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0b3470 con 0x7f350c160c70 2023-11-28T13:00:40.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f35111da700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c1fa9c0 con 0x7f350c070ea0 2023-11-28T13:00:40.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f350ad9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f350c070ea0 0x7f350c155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f350ad9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f350c070ea0 0x7f350c155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47854/0 (socket says 172.21.15.6:47854) 2023-11-28T13:00:40.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f350ad9d700 1 -- 172.21.15.6:0/93417998 learned_addr learned my addr 172.21.15.6:0/93417998 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f350259c700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f350c160c70 0x7f350c0b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f350ad9d700 1 -- 172.21.15.6:0/93417998 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f350c160c70 msgr2=0x7f350c0b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f350ad9d700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f350c160c70 0x7f350c0b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f350ad9d700 1 -- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f350c1646c0 msgr2=0x7f350c1fa480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T13:00:40.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f350ad9d700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f350c1646c0 0x7f350c1fa480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.845+0000 7f350ad9d700 1 -- 172.21.15.6:0/93417998 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f40094a0 con 0x7f350c070ea0 2023-11-28T13:00:40.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea0c0d2a50 con 0x7fea0c0b48d0 2023-11-28T13:00:40.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f350ad9d700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f350c070ea0 0x7f350c155290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00b550 tx=0x7f34fc00b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f14867fc700 1 -- 172.21.15.6:0/3774654808 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f148000f050 con 0x7f14900b4fe0 2023-11-28T13:00:40.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f3503fff700 1 -- 172.21.15.6:0/93417998 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc011830 con 0x7f350c070ea0 2023-11-28T13:00:40.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f3503fff700 1 -- 172.21.15.6:0/93417998 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34fc0119b0 con 0x7f350c070ea0 2023-11-28T13:00:40.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f3503fff700 1 -- 172.21.15.6:0/93417998 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc011d60 con 0x7f350c070ea0 2023-11-28T13:00:40.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f9bee28a700 1 Processor -- start 2023-11-28T13:00:40.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f38ce7fc700 1 -- 172.21.15.6:0/129503841 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b800ba20 con 0x7f38d006b860 2023-11-28T13:00:40.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7f14867fc700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1474068340 0x7f147406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7f14867fc700 1 -- 172.21.15.6:0/3774654808 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1480085a20 con 0x7f14900b4fe0 2023-11-28T13:00:40.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7f14867fc700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x7f1474072010 0x7f1474074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7f14867fc700 1 -- 172.21.15.6:0/3774654808 --> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1474074b10 con 0x7f1474072010 2023-11-28T13:00:40.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7f14867fc700 1 -- 172.21.15.6:0/3774654808 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f148008a2a0 con 0x7f14900b4fe0 2023-11-28T13:00:40.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f14961f6700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x7f1474072010 0x7f1474074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f14951f4700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1474068340 0x7f147406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7f14951f4700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1474068340 0x7f147406a7f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1488005f30 tx=0x7f1488007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7fea08ff9700 1 -- 172.21.15.6:0/2106906386 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe9fc023070 con 0x7fea0c0b48d0 2023-11-28T13:00:40.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7fea08ff9700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe9f80682f0 0x7fe9f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7fea08ff9700 1 -- 172.21.15.6:0/2106906386 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe9fc085df0 con 0x7fea0c0b48d0 2023-11-28T13:00:40.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7fea08ff9700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] conn(0x7fe9f8071fc0 0x7fe9f80743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7fea08ff9700 1 -- 172.21.15.6:0/2106906386 --> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9f8074ac0 con 0x7fe9f8071fc0 2023-11-28T13:00:40.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7fea08ff9700 1 -- 172.21.15.6:0/2106906386 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9fc0861a0 con 0x7fea0c0b48d0 2023-11-28T13:00:40.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7fea10f73700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] conn(0x7fe9f8071fc0 0x7fe9f80743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7fea0b7fe700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe9f80682f0 0x7fe9f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7fea10f73700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] conn(0x7fe9f8071fc0 0x7fe9f80743f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c1653d0 con 0x7f350c070ea0 2023-11-28T13:00:40.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c1650f0 con 0x7f350c070ea0 2023-11-28T13:00:40.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f34f0000fc0 con 0x7f350c070ea0 2023-11-28T13:00:40.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f9bee28a700 1 -- start start 2023-11-28T13:00:40.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9bee28a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9be81588a0 0x7f9be80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9bee28a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9be80cd120 0x7f9be806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9bee28a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be806a280 con 0x7f9be806b860 2023-11-28T13:00:40.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9bee28a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be806a560 con 0x7f9be81588a0 2023-11-28T13:00:40.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9bee28a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be806a840 con 0x7f9be80cd120 2023-11-28T13:00:40.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9be77fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9be80cd120 0x7f9be806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9be77fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9be80cd120 0x7f9be806b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:33650/0 (socket says 172.21.15.6:33650) 2023-11-28T13:00:40.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9be77fe700 1 -- 172.21.15.6:0/4237573300 learned_addr learned my addr 172.21.15.6:0/4237573300 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9be77fe700 1 -- 172.21.15.6:0/4237573300 >> v2:172.21.15.6:3300/0 conn(0x7f9be81588a0 msgr2=0x7f9be80ce010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T13:00:40.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9be77fe700 1 --2- 172.21.15.6:0/4237573300 >> v2:172.21.15.6:3300/0 conn(0x7f9be81588a0 0x7f9be80ce010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9be77fe700 1 -- 172.21.15.6:0/4237573300 >> v1:172.21.15.6:6789/0 conn(0x7f9be806b860 legacy=0x7f9be80c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:40.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7f9be77fe700 1 -- 172.21.15.6:0/4237573300 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be806ab20 con 0x7f9be80cd120 2023-11-28T13:00:40.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7f9be77fe700 1 --2- 172.21.15.6:0/4237573300 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9be80cd120 0x7f9be806b320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9be806ff70 tx=0x7f9bdc00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30f8200c20201cff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f38ce7fc700 1 -- 172.21.15.6:0/129503841 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b800bb80 con 0x7f38d006b860 2023-11-28T13:00:40.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7f38ce7fc700 1 -- 172.21.15.6:0/129503841 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8009860 con 0x7f38d006b860 2023-11-28T13:00:40.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7f38d5fb8700 1 -- 172.21.15.6:0/129503841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38d006b860 msgr2=0x7f38d00c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.847+0000 7f38d5fb8700 1 --2- 172.21.15.6:0/129503841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38d006b860 0x7f38d00c9920 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009310 tx=0x7f38b8009620 comp rx=0 tx=0).stop 2023-11-28T13:00:40.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7f14961f6700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x7f1474072010 0x7f1474074440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7fea0b7fe700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe9f80682f0 0x7fe9f806a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea0c164920 tx=0x7fea00010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7f38d5fb8700 1 -- 172.21.15.6:0/129503841 shutdown_connections 2023-11-28T13:00:40.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7f38d5fb8700 1 --2- 172.21.15.6:0/129503841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38d006b860 0x7f38d00c9920 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7f38d5fb8700 1 --2- 172.21.15.6:0/129503841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 0x7f38d006b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7f38d5fb8700 1 -- 172.21.15.6:0/129503841 >> 172.21.15.6:0/129503841 conn(0x7f38d00cf670 msgr2=0x7f38d00cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7fea08ff9700 1 -- 172.21.15.6:0/2106906386 <== osd.1 v2:172.21.15.6:6800/103501 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe9f8074ac0 con 0x7fe9f8071fc0 2023-11-28T13:00:40.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.846+0000 7fab43c11700 1 Processor -- start 2023-11-28T13:00:40.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.848+0000 7fab43c11700 1 -- start start 2023-11-28T13:00:40.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7fab43c11700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c089580 0x7fab3c0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7fab43c11700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fab3c06b5c0 0x7fab3c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7fab43c11700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3c169750 con 0x7fab3c06a1e0 2023-11-28T13:00:40.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7fab43c11700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3c0ca0d0 con 0x7fab3c06b5c0 2023-11-28T13:00:40.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7fab43c11700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0cd9d0 con 0x7fab3c089580 2023-11-28T13:00:40.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7fab421ae700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fab3c06b5c0 0x7fab3c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7fab421ae700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fab3c06b5c0 0x7fab3c06d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:33654/0 (socket says 172.21.15.6:33654) 2023-11-28T13:00:40.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.849+0000 7fab421ae700 1 -- 172.21.15.6:0/1648281376 learned_addr learned my addr 172.21.15.6:0/1648281376 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7fab411ac700 1 --2- 172.21.15.6:0/1648281376 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c089580 0x7fab3c0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7f38d5fb8700 1 -- 172.21.15.6:0/129503841 shutdown_connections 2023-11-28T13:00:40.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7f38d5fb8700 1 -- 172.21.15.6:0/129503841 wait complete. 2023-11-28T13:00:40.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7f9be6ffd700 1 -- 172.21.15.6:0/4237573300 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc010070 con 0x7f9be80cd120 2023-11-28T13:00:40.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7fab411ac700 1 -- 172.21.15.6:0/1648281376 >> v2:172.21.15.61:3300/0 conn(0x7fab3c06b5c0 msgr2=0x7fab3c06d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:40.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7fab411ac700 1 --2- 172.21.15.6:0/1648281376 >> v2:172.21.15.61:3300/0 conn(0x7fab3c06b5c0 0x7fab3c06d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7fab411ac700 1 -- 172.21.15.6:0/1648281376 >> v1:172.21.15.61:6789/0 conn(0x7fab3c06a1e0 legacy=0x7fab3c06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7fab409ab700 1 -- 172.21.15.6:0/1648281376 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337824806 0 0) 0x7fab3c169750 con 0x7fab3c06a1e0 2023-11-28T13:00:40.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7fab411ac700 1 -- 172.21.15.6:0/1648281376 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c0cdcb0 con 0x7fab3c089580 2023-11-28T13:00:40.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7f14867fc700 1 -- 172.21.15.6:0/3774654808 <== osd.0 v2:172.21.15.6:6809/103502 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1474074b10 con 0x7f1474072010 2023-11-28T13:00:40.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.851+0000 7f7b04b6f700 1 Processor -- start 2023-11-28T13:00:40.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7f9be6ffd700 1 -- 172.21.15.6:0/4237573300 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc00da40 con 0x7f9be80cd120 2023-11-28T13:00:40.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.850+0000 7f9be6ffd700 1 -- 172.21.15.6:0/4237573300 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00ddd0 con 0x7f9be80cd120 2023-11-28T13:00:40.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.851+0000 7f9bee28a700 1 -- 172.21.15.6:0/4237573300 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9be80cd120 msgr2=0x7f9be806b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.851+0000 7f9bee28a700 1 --2- 172.21.15.6:0/4237573300 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9be80cd120 0x7f9be806b320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9be806ff70 tx=0x7f9bdc00d330 comp rx=0 tx=0).stop 2023-11-28T13:00:40.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.851+0000 7f9bee28a700 1 -- 172.21.15.6:0/4237573300 shutdown_connections 2023-11-28T13:00:40.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.851+0000 7f9bee28a700 1 --2- 172.21.15.6:0/4237573300 >> v2:172.21.15.6:3300/0 conn(0x7f9be81588a0 0x7f9be80ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.851+0000 7f9bee28a700 1 --2- 172.21.15.6:0/4237573300 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9be80cd120 0x7f9be806b320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.851+0000 7f9bee28a700 1 -- 172.21.15.6:0/4237573300 >> 172.21.15.6:0/4237573300 conn(0x7f9be80cf670 msgr2=0x7f9be80cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7f9bee28a700 1 -- 172.21.15.6:0/4237573300 shutdown_connections 2023-11-28T13:00:40.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7f38d5fb8700 1 Processor -- start 2023-11-28T13:00:40.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7fab411ac700 1 --2- 172.21.15.6:0/1648281376 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c089580 0x7fab3c0c9a40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fab280099c0 tx=0x7fab280099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46ed22e3d990127 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.851+0000 7f7b04b6f700 1 -- start start 2023-11-28T13:00:40.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7f7b04b6f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f7b00160ac0 0x7f7b000d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7f7b04b6f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 0x7f7b000b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7f7b04b6f700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0006a1e0 con 0x7f7b00164510 2023-11-28T13:00:40.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7f7b04b6f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b000b64b0 con 0x7f7b00160ac0 2023-11-28T13:00:40.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7f7b04b6f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000b5290 con 0x7f7b000b48d0 2023-11-28T13:00:40.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7fab409ab700 1 -- 172.21.15.6:0/1648281376 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2800d960 con 0x7fab3c089580 2023-11-28T13:00:40.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7fab409ab700 1 -- 172.21.15.6:0/1648281376 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2800dac0 con 0x7fab3c089580 2023-11-28T13:00:40.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7fab409ab700 1 -- 172.21.15.6:0/1648281376 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2800de70 con 0x7fab3c089580 2023-11-28T13:00:40.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f7afed9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 0x7f7b000b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f7afed9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 0x7f7b000b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47876/0 (socket says 172.21.15.6:47876) 2023-11-28T13:00:40.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f7afed9d700 1 -- 172.21.15.6:0/2304991940 learned_addr learned my addr 172.21.15.6:0/2304991940 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7fab43c11700 1 -- 172.21.15.6:0/1648281376 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c089580 msgr2=0x7fab3c0c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7fab43c11700 1 --2- 172.21.15.6:0/1648281376 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c089580 0x7fab3c0c9a40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fab280099c0 tx=0x7fab280099f0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f7afe59c700 1 --2- 172.21.15.6:0/2304991940 >> v2:172.21.15.61:3300/0 conn(0x7f7b00160ac0 0x7f7b000d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7f38d5fb8700 1 -- start start 2023-11-28T13:00:40.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f38d5fb8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38d006b860 0x7f38d0149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f38d5fb8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 0x7f38d00b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f38d5fb8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38d01588a0 0x7f38d00b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f38d5fb8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d01e4ae0 con 0x7f38d006b860 2023-11-28T13:00:40.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f38d5fb8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d0149650 con 0x7f38d01588a0 2023-11-28T13:00:40.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f38d5fb8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d01e3e60 con 0x7f38d00cd120 2023-11-28T13:00:40.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.852+0000 7f9bee28a700 1 -- 172.21.15.6:0/4237573300 wait complete. 2023-11-28T13:00:40.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f9bee28a700 1 Processor -- start 2023-11-28T13:00:40.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f38ceffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 0x7f38d00b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7fab43c11700 1 -- 172.21.15.6:0/1648281376 shutdown_connections 2023-11-28T13:00:40.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7fab43c11700 1 --2- 172.21.15.6:0/1648281376 >> v2:172.21.15.61:3300/0 conn(0x7fab3c06b5c0 0x7fab3c06d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7fab43c11700 1 --2- 172.21.15.6:0/1648281376 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c089580 0x7fab3c0c9a40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7fab43c11700 1 -- 172.21.15.6:0/1648281376 >> 172.21.15.6:0/1648281376 conn(0x7fab3c0cf670 msgr2=0x7fab3c0cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f38ceffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 0x7f38d00b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47882/0 (socket says 172.21.15.6:47882) 2023-11-28T13:00:40.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f38ceffd700 1 -- 172.21.15.6:0/1988193717 learned_addr learned my addr 172.21.15.6:0/1988193717 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f38cf7fe700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38d006b860 0x7f38d0149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f7afed9d700 1 -- 172.21.15.6:0/2304991940 >> v2:172.21.15.61:3300/0 conn(0x7f7b00160ac0 msgr2=0x7f7b000d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:40.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.853+0000 7f7afed9d700 1 --2- 172.21.15.6:0/2304991940 >> v2:172.21.15.61:3300/0 conn(0x7f7b00160ac0 0x7f7b000d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f7afed9d700 1 -- 172.21.15.6:0/2304991940 >> v1:172.21.15.6:6789/0 conn(0x7f7b00164510 legacy=0x7f7b000b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f7afed9d700 1 -- 172.21.15.6:0/2304991940 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b001649f0 con 0x7f7b000b48d0 2023-11-28T13:00:40.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f9bee28a700 1 -- start start 2023-11-28T13:00:40.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7fab43c11700 1 -- 172.21.15.6:0/1648281376 shutdown_connections 2023-11-28T13:00:40.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f38ceffd700 1 -- 172.21.15.6:0/1988193717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38d01588a0 msgr2=0x7f38d00b1df0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:00:40.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f38ceffd700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38d01588a0 0x7f38d00b1df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f38ceffd700 1 -- 172.21.15.6:0/1988193717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38d006b860 msgr2=0x7f38d0149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f38ceffd700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38d006b860 0x7f38d0149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7f38ceffd700 1 -- 172.21.15.6:0/1988193717 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b8009de0 con 0x7f38d00cd120 2023-11-28T13:00:40.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f7afed9d700 1 --2- 172.21.15.6:0/2304991940 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 0x7f7b000b4d50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7af80098f0 tx=0x7f7af8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f26a69da8ace08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f7afdd9b700 1 -- 172.21.15.6:0/2304991940 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800b8a0 con 0x7f7b000b48d0 2023-11-28T13:00:40.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f7afdd9b700 1 -- 172.21.15.6:0/2304991940 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af800ba00 con 0x7f7b000b48d0 2023-11-28T13:00:40.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f7afdd9b700 1 -- 172.21.15.6:0/2304991940 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800bd90 con 0x7f7b000b48d0 2023-11-28T13:00:40.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f3503fff700 1 -- 172.21.15.6:0/93417998 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f34fc021070 con 0x7f350c070ea0 2023-11-28T13:00:40.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f3503fff700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34f8068520 0x7f34f806a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f3503fff700 1 -- 172.21.15.6:0/93417998 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f34fc0865b0 con 0x7f350c070ea0 2023-11-28T13:00:40.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f3503fff700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] conn(0x7f34f80721f0 0x7f34f8074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f3503fff700 1 -- 172.21.15.6:0/93417998 --> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34f8074cf0 con 0x7f34f80721f0 2023-11-28T13:00:40.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f3503fff700 1 -- 172.21.15.6:0/93417998 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34fc086960 con 0x7f350c070ea0 2023-11-28T13:00:40.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f7b04b6f700 1 -- 172.21.15.6:0/2304991940 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 msgr2=0x7f7b000b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f7b04b6f700 1 --2- 172.21.15.6:0/2304991940 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 0x7f7b000b4d50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7af80098f0 tx=0x7f7af8009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:40.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f38ceffd700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 0x7f38d00b18b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f38c000ce40 tx=0x7f38c000a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f350b59e700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] conn(0x7f34f80721f0 0x7f34f8074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f350259c700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34f8068520 0x7f34f806a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.854+0000 7fab43c11700 1 -- 172.21.15.6:0/1648281376 wait complete. 2023-11-28T13:00:40.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7fab43c11700 1 Processor -- start 2023-11-28T13:00:40.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f7b04b6f700 1 -- 172.21.15.6:0/2304991940 shutdown_connections 2023-11-28T13:00:40.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f7b04b6f700 1 --2- 172.21.15.6:0/2304991940 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 0x7f7b000b4d50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f7b04b6f700 1 --2- 172.21.15.6:0/2304991940 >> v2:172.21.15.61:3300/0 conn(0x7f7b00160ac0 0x7f7b000d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f7b04b6f700 1 -- 172.21.15.6:0/2304991940 >> 172.21.15.6:0/2304991940 conn(0x7f7b000cf8d0 msgr2=0x7f7b000cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f7b04b6f700 1 -- 172.21.15.6:0/2304991940 shutdown_connections 2023-11-28T13:00:40.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f9bee28a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9be806b860 0x7f9be8154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f9bee28a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be80cd120 0x7f9be81fb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f9bee28a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9be81588a0 0x7f9be80b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f9bee28a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8153af0 con 0x7f9be80cd120 2023-11-28T13:00:40.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f9bee28a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be81fb5d0 con 0x7f9be81588a0 2023-11-28T13:00:40.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.855+0000 7f9bee28a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be8153d70 con 0x7f9be806b860 2023-11-28T13:00:40.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f38c7fff700 1 -- 172.21.15.6:0/1988193717 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0003a20 con 0x7f38d00cd120 2023-11-28T13:00:40.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f38c7fff700 1 -- 172.21.15.6:0/1988193717 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c0003b80 con 0x7f38d00cd120 2023-11-28T13:00:40.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f38c7fff700 1 -- 172.21.15.6:0/1988193717 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c0004020 con 0x7f38d00cd120 2023-11-28T13:00:40.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d006ae00 con 0x7f38d00cd120 2023-11-28T13:00:40.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f9be7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9be806b860 0x7f9be8154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f9be7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9be806b860 0x7f9be8154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47892/0 (socket says 172.21.15.6:47892) 2023-11-28T13:00:40.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f9be7fff700 1 -- 172.21.15.6:0/3445221614 learned_addr learned my addr 172.21.15.6:0/3445221614 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f9bec827700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9be81588a0 0x7f9be80b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.857+0000 7f9bec827700 1 -- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9be806b860 msgr2=0x7f9be8154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.857+0000 7f9bec827700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9be806b860 0x7f9be8154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.857+0000 7f9bec827700 1 -- 172.21.15.6:0/3445221614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be80cd120 msgr2=0x7f9be81fb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.857+0000 7f9bec827700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9be80cd120 0x7f9be81fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.857+0000 7f9bec827700 1 -- 172.21.15.6:0/3445221614 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc0085d0 con 0x7f9be81588a0 2023-11-28T13:00:40.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.857+0000 7f9bec827700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9be81588a0 0x7f9be80b2d00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9be806bcc0 tx=0x7f9bd800c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.857+0000 7f350259c700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34f8068520 0x7f34f806a9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34f40097c0 tx=0x7f34f4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.857+0000 7f350b59e700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] conn(0x7f34f80721f0 0x7f34f8074620 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.857+0000 7fab43c11700 1 -- start start 2023-11-28T13:00:40.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.857+0000 7f9be4ff9700 1 -- 172.21.15.6:0/3445221614 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80052d0 con 0x7f9be81588a0 2023-11-28T13:00:40.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.858+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be80ce550 con 0x7f9be81588a0 2023-11-28T13:00:40.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.858+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be806ae00 con 0x7f9be81588a0 2023-11-28T13:00:40.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f7b04b6f700 1 -- 172.21.15.6:0/2304991940 wait complete. 2023-11-28T13:00:40.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.858+0000 7f7b04b6f700 1 Processor -- start 2023-11-28T13:00:40.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.856+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d006ab20 con 0x7f38d00cd120 2023-11-28T13:00:40.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7f3503fff700 1 -- 172.21.15.6:0/93417998 <== osd.2 v2:172.21.15.6:6824/103512 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f34f8074cf0 con 0x7f34f80721f0 2023-11-28T13:00:40.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7f9be4ff9700 1 -- 172.21.15.6:0/3445221614 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd8005430 con 0x7f9be81588a0 2023-11-28T13:00:40.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7f9be4ff9700 1 -- 172.21.15.6:0/3445221614 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80038a0 con 0x7f9be81588a0 2023-11-28T13:00:40.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.858+0000 7fab43c11700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c06a1e0 0x7fab3c212950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.858+0000 7fab43c11700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab3c06b5c0 0x7fab3c1f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.858+0000 7fab43c11700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab3c089580 0x7fab3c1f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.858+0000 7fab43c11700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c1f4850 con 0x7fab3c089580 2023-11-28T13:00:40.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.858+0000 7fab43c11700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0863c0 con 0x7fab3c06b5c0 2023-11-28T13:00:40.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.858+0000 7fab43c11700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c0866a0 con 0x7fab3c06a1e0 2023-11-28T13:00:40.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab419ad700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c06a1e0 0x7fab3c212950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab419ad700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c06a1e0 0x7fab3c212950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47898/0 (socket says 172.21.15.6:47898) 2023-11-28T13:00:40.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab419ad700 1 -- 172.21.15.6:0/1914085364 learned_addr learned my addr 172.21.15.6:0/1914085364 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab421ae700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab3c089580 0x7fab3c1f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab411ac700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab3c06b5c0 0x7fab3c1f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab419ad700 1 -- 172.21.15.6:0/1914085364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab3c06b5c0 msgr2=0x7fab3c1f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab419ad700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab3c06b5c0 0x7fab3c1f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab419ad700 1 -- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab3c089580 msgr2=0x7fab3c1f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab419ad700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab3c089580 0x7fab3c1f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab419ad700 1 -- 172.21.15.6:0/1914085364 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28009620 con 0x7fab3c06a1e0 2023-11-28T13:00:40.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7fab419ad700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c06a1e0 0x7fab3c212950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fab3000c8c0 tx=0x7fab3000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.858+0000 7f7b04b6f700 1 -- start start 2023-11-28T13:00:40.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7f7b04b6f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 0x7f7b0015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.859+0000 7f7b04b6f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b00160ac0 0x7f7b000c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f7b04b6f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b00164510 0x7f7b0006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f7b04b6f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000c94e0 con 0x7f7b00164510 2023-11-28T13:00:40.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f7b04b6f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0006fed0 con 0x7f7b00160ac0 2023-11-28T13:00:40.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f7b04b6f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0006f320 con 0x7f7b000b48d0 2023-11-28T13:00:40.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9bc8000fc0 con 0x7f9be81588a0 2023-11-28T13:00:40.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f38c7fff700 1 -- 172.21.15.6:0/1988193717 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f38c001c020 con 0x7f38d00cd120 2023-11-28T13:00:40.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f38b4000fc0 con 0x7f38d00cd120 2023-11-28T13:00:40.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f7afe59c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 0x7f7b0015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f7af5d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b00160ac0 0x7f7b000c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f38c7fff700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f38b0068340 0x7f38b006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f7af5d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b00160ac0 0x7f7b000c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:33688/0 (socket says 172.21.15.6:33688) 2023-11-28T13:00:40.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f7af5d9b700 1 -- 172.21.15.6:0/3591398370 learned_addr learned my addr 172.21.15.6:0/3591398370 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f38c7fff700 1 -- 172.21.15.6:0/1988193717 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f38c0012070 con 0x7f38d00cd120 2023-11-28T13:00:40.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7f38c7fff700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x7f38b0072010 0x7f38b0074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7f38c7fff700 1 -- 172.21.15.6:0/1988193717 --> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f38b0074b10 con 0x7f38b0072010 2023-11-28T13:00:40.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7f38c7fff700 1 -- 172.21.15.6:0/1988193717 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38c0088a90 con 0x7f38d00cd120 2023-11-28T13:00:40.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7f38cf7fe700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f38b0068340 0x7f38b006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f7441ed7700 1 Processor -- start 2023-11-28T13:00:40.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.860+0000 7f7441ed7700 1 -- start start 2023-11-28T13:00:40.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7f7afe59c700 1 -- 172.21.15.6:0/3591398370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b00160ac0 msgr2=0x7f7b000c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7f7afe59c700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b00160ac0 0x7f7b000c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7f7afe59c700 1 -- 172.21.15.6:0/3591398370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b00164510 msgr2=0x7f7b0006d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7f7afe59c700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b00164510 0x7f7b0006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7f7afe59c700 1 -- 172.21.15.6:0/3591398370 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af8009550 con 0x7f7b000b48d0 2023-11-28T13:00:40.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f38cf7fe700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f38b0068340 0x7f38b006a7f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f38c8007fd0 tx=0x7f38c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f7441ed7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f743c0c2630 0x7f743c0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f7441ed7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce230 0x7f743c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f7441ed7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c0cceb0 con 0x7f743c0ce6a0 2023-11-28T13:00:40.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f7441ed7700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c06b180 con 0x7f743c0c2630 2023-11-28T13:00:40.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f7441ed7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0cd190 con 0x7f743c0ce230 2023-11-28T13:00:40.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f743affd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce230 0x7f743c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f743b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f743c0c2630 0x7f743c0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f743affd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce230 0x7f743c06a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47914/0 (socket says 172.21.15.6:47914) 2023-11-28T13:00:40.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f743bfff700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f743c0ce6a0 0x7f743c06d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:35562/0 (socket says 172.21.15.6:35562) 2023-11-28T13:00:40.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f743affd700 1 -- 172.21.15.6:0/2120726838 learned_addr learned my addr 172.21.15.6:0/2120726838 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f743b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f743c0c2630 0x7f743c0ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58208/0 (socket says 172.21.15.6:58208) 2023-11-28T13:00:40.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7f9be4ff9700 1 -- 172.21.15.6:0/3445221614 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9bd8019030 con 0x7f9be81588a0 2023-11-28T13:00:40.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f38cffff700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x7f38b0072010 0x7f38b0074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f9be4ff9700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9bd40682f0 0x7f9bd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f9be4ff9700 1 -- 172.21.15.6:0/3445221614 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9bd80852b0 con 0x7f9be81588a0 2023-11-28T13:00:40.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f9be4ff9700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x7f9bd4071fc0 0x7f9bd40743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f9be4ff9700 1 -- 172.21.15.6:0/3445221614 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9bd4074ac0 con 0x7f9bd4071fc0 2023-11-28T13:00:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f9be4ff9700 1 -- 172.21.15.6:0/3445221614 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bd80878d0 con 0x7f9be81588a0 2023-11-28T13:00:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7fab3a7fc700 1 -- 172.21.15.6:0/1914085364 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30003e30 con 0x7fab3c06a1e0 2023-11-28T13:00:40.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f38cffff700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x7f38b0072010 0x7f38b0074440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f9be7fff700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9bd40682f0 0x7f9bd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f9be77fe700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x7f9bd4071fc0 0x7f9bd40743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7fab3a7fc700 1 -- 172.21.15.6:0/1914085364 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab30005070 con 0x7fab3c06a1e0 2023-11-28T13:00:40.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c0ce270 con 0x7fab3c06a1e0 2023-11-28T13:00:40.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.861+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c0cdf90 con 0x7fab3c06a1e0 2023-11-28T13:00:40.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab3c0ce790 con 0x7fab3c06a1e0 2023-11-28T13:00:40.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.862+0000 7fab3a7fc700 1 -- 172.21.15.6:0/1914085364 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30005420 con 0x7fab3c06a1e0 2023-11-28T13:00:40.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7fab3a7fc700 1 -- 172.21.15.6:0/1914085364 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fab30010070 con 0x7fab3c06a1e0 2023-11-28T13:00:40.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f743affd700 1 -- 172.21.15.6:0/2120726838 >> v2:172.21.15.6:3300/0 conn(0x7f743c0c2630 msgr2=0x7f743c0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:40.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f743affd700 1 --2- 172.21.15.6:0/2120726838 >> v2:172.21.15.6:3300/0 conn(0x7f743c0c2630 0x7f743c0ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f743affd700 1 -- 172.21.15.6:0/2120726838 >> v1:172.21.15.61:6789/0 conn(0x7f743c0ce6a0 legacy=0x7f743c06d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.863+0000 7f743affd700 1 -- 172.21.15.6:0/2120726838 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c0cd470 con 0x7f743c0ce230 2023-11-28T13:00:40.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.864+0000 7f7afe59c700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 0x7f7b0015dd20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7af000cdc0 tx=0x7f7af000a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.864+0000 7f9be77fe700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x7f9bd4071fc0 0x7f9bd40743f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.864+0000 7f7af77fe700 1 -- 172.21.15.6:0/3591398370 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0003a20 con 0x7f7b000b48d0 2023-11-28T13:00:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.864+0000 7f7af77fe700 1 -- 172.21.15.6:0/3591398370 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af0003b80 con 0x7f7b000b48d0 2023-11-28T13:00:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.864+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b00164fb0 con 0x7f7b000b48d0 2023-11-28T13:00:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.864+0000 7fab3a7fc700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab200682f0 0x7fab2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.864+0000 7fab411ac700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab200682f0 0x7fab2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.864+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b00164cd0 con 0x7f7b000b48d0 2023-11-28T13:00:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.864+0000 7f7af77fe700 1 -- 172.21.15.6:0/3591398370 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0004020 con 0x7f7b000b48d0 2023-11-28T13:00:40.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.864+0000 7f38c7fff700 1 -- 172.21.15.6:0/1988193717 <== osd.6 v2:172.21.15.61:6800/103779 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f38b0074b10 con 0x7f38b0072010 2023-11-28T13:00:40.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.865+0000 7fab411ac700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab200682f0 0x7fab2006a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab28009a30 tx=0x7fab28011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.865+0000 7fab3a7fc700 1 -- 172.21.15.6:0/1914085364 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fab30010300 con 0x7fab3c06a1e0 2023-11-28T13:00:40.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.865+0000 7fab3a7fc700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] conn(0x7fab20071ec0 0x7fab200742f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.865+0000 7fab3a7fc700 1 -- 172.21.15.6:0/1914085364 --> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab200749c0 con 0x7fab20071ec0 2023-11-28T13:00:40.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.865+0000 7fab3a7fc700 1 -- 172.21.15.6:0/1914085364 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab30001910 con 0x7fab3c06a1e0 2023-11-28T13:00:40.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.865+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7ae4000fc0 con 0x7f7b000b48d0 2023-11-28T13:00:40.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.865+0000 7f9be4ff9700 1 -- 172.21.15.6:0/3445221614 <== osd.5 v2:172.21.15.61:6808/103780 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9bd4074ac0 con 0x7f9bd4071fc0 2023-11-28T13:00:40.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.865+0000 7f9be7fff700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9bd40682f0 0x7f9bd406a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9be80c52d0 tx=0x7f9be000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.865+0000 7fab421ae700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] conn(0x7fab20071ec0 0x7fab200742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.866+0000 7fab421ae700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] conn(0x7fab20071ec0 0x7fab200742f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.866+0000 7fab3a7fc700 1 -- 172.21.15.6:0/1914085364 <== osd.3 v2:172.21.15.6:6808/103508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fab200749c0 con 0x7fab20071ec0 2023-11-28T13:00:40.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.868+0000 7f743affd700 1 --2- 172.21.15.6:0/2120726838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce230 0x7f743c06a480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74300098f0 tx=0x7f7430009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245860b1bb077063 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.868+0000 7f743a7fc700 1 -- 172.21.15.6:0/2120726838 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430004230 con 0x7f743c0ce230 2023-11-28T13:00:40.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.868+0000 7f743a7fc700 1 -- 172.21.15.6:0/2120726838 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7430004390 con 0x7f743c0ce230 2023-11-28T13:00:40.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.868+0000 7f743a7fc700 1 -- 172.21.15.6:0/2120726838 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74300036a0 con 0x7f743c0ce230 2023-11-28T13:00:40.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.869+0000 7f7441ed7700 1 -- 172.21.15.6:0/2120726838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce230 msgr2=0x7f743c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.869+0000 7f7441ed7700 1 --2- 172.21.15.6:0/2120726838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce230 0x7f743c06a480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74300098f0 tx=0x7f7430009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:40.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.869+0000 7f7af77fe700 1 -- 172.21.15.6:0/3591398370 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7af001c020 con 0x7f7b000b48d0 2023-11-28T13:00:40.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.869+0000 7f7af77fe700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7ae0068340 0x7f7ae006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.869+0000 7f7af77fe700 1 -- 172.21.15.6:0/3591398370 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7af0012070 con 0x7f7b000b48d0 2023-11-28T13:00:40.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.870+0000 7f7af77fe700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] conn(0x7f7ae0072010 0x7f7ae0074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.870+0000 7f7af77fe700 1 -- 172.21.15.6:0/3591398370 --> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ae0074b10 con 0x7f7ae0072010 2023-11-28T13:00:40.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.870+0000 7f7af77fe700 1 -- 172.21.15.6:0/3591398370 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7af0088a30 con 0x7f7b000b48d0 2023-11-28T13:00:40.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.870+0000 7f7af5d9b700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7ae0068340 0x7f7ae006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.870+0000 7f7af5d9b700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7ae0068340 0x7f7ae006a7f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b000710d0 tx=0x7f7ae8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.871+0000 7f7afed9d700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] conn(0x7f7ae0072010 0x7f7ae0074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.871+0000 7f7441ed7700 1 -- 172.21.15.6:0/2120726838 shutdown_connections 2023-11-28T13:00:40.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.871+0000 7f7441ed7700 1 --2- 172.21.15.6:0/2120726838 >> v2:172.21.15.6:3300/0 conn(0x7f743c0c2630 0x7f743c0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.871+0000 7f7441ed7700 1 --2- 172.21.15.6:0/2120726838 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce230 0x7f743c06a480 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.871+0000 7f7441ed7700 1 -- 172.21.15.6:0/2120726838 >> 172.21.15.6:0/2120726838 conn(0x7f743c0b0dd0 msgr2=0x7f743c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.871+0000 7f7441ed7700 1 -- 172.21.15.6:0/2120726838 shutdown_connections 2023-11-28T13:00:40.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.871+0000 7f7441ed7700 1 -- 172.21.15.6:0/2120726838 wait complete. 2023-11-28T13:00:40.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.873+0000 7f7441ed7700 1 Processor -- start 2023-11-28T13:00:40.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.873+0000 7f7441ed7700 1 -- start start 2023-11-28T13:00:40.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.874+0000 7f7afed9d700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] conn(0x7f7ae0072010 0x7f7ae0074440 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.874+0000 7f7441ed7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f743c0c2630 0x7f743c155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.874+0000 7f7441ed7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f743c0ce230 0x7f743c0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f7441ed7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce6a0 0x7f743c0c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f7441ed7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0c3c20 con 0x7f743c0ce230 2023-11-28T13:00:40.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f7441ed7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0c3f00 con 0x7f743c0c2630 2023-11-28T13:00:40.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f7441ed7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c162dc0 con 0x7f743c0ce6a0 2023-11-28T13:00:40.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f743bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce6a0 0x7f743c0c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f743bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce6a0 0x7f743c0c2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:47932/0 (socket says 172.21.15.6:47932) 2023-11-28T13:00:40.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f743bfff700 1 -- 172.21.15.6:0/436869894 learned_addr learned my addr 172.21.15.6:0/436869894 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:40.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f743affd700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f743c0ce230 0x7f743c0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f743affd700 1 -- 172.21.15.6:0/436869894 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce6a0 msgr2=0x7f743c0c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f743affd700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f743c0ce6a0 0x7f743c0c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f743affd700 1 -- 172.21.15.6:0/436869894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f743c0c2630 msgr2=0x7f743c155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f743affd700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f743c0c2630 0x7f743c155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.875+0000 7f743affd700 1 -- 172.21.15.6:0/436869894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7430009550 con 0x7f743c0ce230 2023-11-28T13:00:40.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.876+0000 7f7af77fe700 1 -- 172.21.15.6:0/3591398370 <== osd.4 v2:172.21.15.61:6816/103782 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7ae0074b10 con 0x7f7ae0072010 2023-11-28T13:00:40.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.876+0000 7f743affd700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f743c0ce230 0x7f743c0c8a90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f743c0aeb70 tx=0x7f7430004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.879+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 --> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea0c1618c0 con 0x7fe9f8071fc0 2023-11-28T13:00:40.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.881+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 --> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1478002d40 con 0x7f1474072010 2023-11-28T13:00:40.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.881+0000 7f7423fff700 1 -- 172.21.15.6:0/436869894 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430011830 con 0x7f743c0ce230 2023-11-28T13:00:40.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.881+0000 7f7423fff700 1 -- 172.21.15.6:0/436869894 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74300119b0 con 0x7f743c0ce230 2023-11-28T13:00:40.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.881+0000 7f7423fff700 1 -- 172.21.15.6:0/436869894 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430011d60 con 0x7f743c0ce230 2023-11-28T13:00:40.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.883+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c0cda30 con 0x7f743c0ce230 2023-11-28T13:00:40.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.883+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c0cd750 con 0x7f743c0ce230 2023-11-28T13:00:40.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.884+0000 7f14867fc700 1 -- 172.21.15.6:0/3774654808 <== osd.0 v2:172.21.15.6:6809/103502 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1478002d40 con 0x7f1474072010 2023-11-28T13:00:40.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.884+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f743c0cdd10 con 0x7f743c0ce230 2023-11-28T13:00:40.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.885+0000 7fea08ff9700 1 -- 172.21.15.6:0/2106906386 <== osd.1 v2:172.21.15.6:6800/103501 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fea0c1618c0 con 0x7fe9f8071fc0 2023-11-28T13:00:40.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.885+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x7f1474072010 msgr2=0x7f1474074440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.885+0000 7f1497c59700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x7f1474072010 0x7f1474074440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.885+0000 7f7423fff700 1 -- 172.21.15.6:0/436869894 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7430017020 con 0x7f743c0ce230 2023-11-28T13:00:40.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.886+0000 7f7423fff700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f74280683c0 0x7f742806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.886+0000 7f7423fff700 1 -- 172.21.15.6:0/436869894 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7430088150 con 0x7f743c0ce230 2023-11-28T13:00:40.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.886+0000 7f743b7fe700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f74280683c0 0x7f742806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.886+0000 7f7423fff700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] conn(0x7f7428072090 0x7f74280744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:40.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.886+0000 7f7423fff700 1 -- 172.21.15.6:0/436869894 --> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7428074b90 con 0x7f7428072090 2023-11-28T13:00:40.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.886+0000 7f7423fff700 1 -- 172.21.15.6:0/436869894 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7430088500 con 0x7f743c0ce230 2023-11-28T13:00:40.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.886+0000 7f743bfff700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] conn(0x7f7428072090 0x7f74280744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:40.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.886+0000 7f743b7fe700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f74280683c0 0x7f742806a870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f742c0097f0 tx=0x7f742c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.887+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1474068340 msgr2=0x7f147406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.887+0000 7f1497c59700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1474068340 0x7f147406a7f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1488005f30 tx=0x7f1488007d50 comp rx=0 tx=0).stop 2023-11-28T13:00:40.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.887+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14900b4fe0 msgr2=0x7f149014eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.887+0000 7f1497c59700 1 --2- 172.21.15.6:0/3774654808 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14900b4fe0 0x7f149014eef0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f148000b4a0 tx=0x7f148000b860 comp rx=0 tx=0).stop 2023-11-28T13:00:40.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.887+0000 7f14961f6700 1 -- 172.21.15.6:0/3774654808 reap_dead start 2023-11-28T13:00:40.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.888+0000 7f743bfff700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] conn(0x7f7428072090 0x7f74280744c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:40.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.889+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 shutdown_connections 2023-11-28T13:00:40.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.889+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 >> 172.21.15.6:0/3774654808 conn(0x7f14900cf8d0 msgr2=0x7f1490157ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.889+0000 7f7423fff700 1 -- 172.21.15.6:0/436869894 <== osd.7 v2:172.21.15.61:6824/103806 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7428074b90 con 0x7f7428072090 2023-11-28T13:00:40.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.889+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 --> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab3c0867e0 con 0x7fab20071ec0 2023-11-28T13:00:40.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.889+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 shutdown_connections 2023-11-28T13:00:40.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.889+0000 7f1497c59700 1 -- 172.21.15.6:0/3774654808 wait complete. 2023-11-28T13:00:40.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.890+0000 7fab3a7fc700 1 -- 172.21.15.6:0/1914085364 <== osd.3 v2:172.21.15.6:6808/103508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab3c0867e0 con 0x7fab20071ec0 2023-11-28T13:00:40.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.890+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] conn(0x7fab20071ec0 msgr2=0x7fab200742f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.890+0000 7fab43c11700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] conn(0x7fab20071ec0 0x7fab200742f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.891+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab200682f0 msgr2=0x7fab2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.891+0000 7fab43c11700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab200682f0 0x7fab2006a7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab28009a30 tx=0x7fab28011000 comp rx=0 tx=0).stop 2023-11-28T13:00:40.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.891+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c06a1e0 msgr2=0x7fab3c212950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.891+0000 7fab43c11700 1 --2- 172.21.15.6:0/1914085364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab3c06a1e0 0x7fab3c212950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fab3000c8c0 tx=0x7fab3000cc80 comp rx=0 tx=0).stop 2023-11-28T13:00:40.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.891+0000 7fab421ae700 1 -- 172.21.15.6:0/1914085364 reap_dead start 2023-11-28T13:00:40.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.891+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 shutdown_connections 2023-11-28T13:00:40.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.891+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 >> 172.21.15.6:0/1914085364 conn(0x7fab3c0cf670 msgr2=0x7fab3c06e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.893+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 --> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f34f0002d40 con 0x7f34f80721f0 2023-11-28T13:00:40.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.893+0000 7f3503fff700 1 -- 172.21.15.6:0/93417998 <== osd.2 v2:172.21.15.6:6824/103512 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f34f0002d40 con 0x7f34f80721f0 2023-11-28T13:00:40.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.894+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] conn(0x7f34f80721f0 msgr2=0x7f34f8074620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.894+0000 7f35111da700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] conn(0x7f34f80721f0 0x7f34f8074620 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.894+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34f8068520 msgr2=0x7f34f806a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.894+0000 7f35111da700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34f8068520 0x7f34f806a9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34f40097c0 tx=0x7f34f4008030 comp rx=0 tx=0).stop 2023-11-28T13:00:40.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.894+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f350c070ea0 msgr2=0x7f350c155290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.894+0000 7f35111da700 1 --2- 172.21.15.6:0/93417998 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f350c070ea0 0x7f350c155290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00b550 tx=0x7f34fc00b860 comp rx=0 tx=0).stop 2023-11-28T13:00:40.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.894+0000 7f350b59e700 1 -- 172.21.15.6:0/93417998 reap_dead start 2023-11-28T13:00:40.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 shutdown_connections 2023-11-28T13:00:40.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 >> 172.21.15.6:0/93417998 conn(0x7f350c0ce9e0 msgr2=0x7f350c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] conn(0x7fe9f8071fc0 msgr2=0x7fe9f80743f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7fea129d6700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] conn(0x7fe9f8071fc0 0x7fe9f80743f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe9f80682f0 msgr2=0x7fe9f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7fea129d6700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe9f80682f0 0x7fe9f806a7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea0c164920 tx=0x7fea00010000 comp rx=0 tx=0).stop 2023-11-28T13:00:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea0c0b48d0 msgr2=0x7fea0c1febc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7fea129d6700 1 --2- 172.21.15.6:0/2106906386 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea0c0b48d0 0x7fea0c1febc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00e7e0 tx=0x7fe9fc00eaf0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7fea10f73700 1 -- 172.21.15.6:0/2106906386 reap_dead start 2023-11-28T13:00:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 shutdown_connections 2023-11-28T13:00:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7f35111da700 1 -- 172.21.15.6:0/93417998 wait complete. 2023-11-28T13:00:40.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 shutdown_connections 2023-11-28T13:00:40.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.895+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 >> 172.21.15.6:0/2106906386 conn(0x7fea0c0cf8d0 msgr2=0x7fea0c0b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.896+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 shutdown_connections 2023-11-28T13:00:40.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.896+0000 7fab43c11700 1 -- 172.21.15.6:0/1914085364 wait complete. 2023-11-28T13:00:40.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.897+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 --> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f38b4002d40 con 0x7f38b0072010 2023-11-28T13:00:40.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.898+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9bc8002d40 con 0x7f9bd4071fc0 2023-11-28T13:00:40.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.898+0000 7f9be4ff9700 1 -- 172.21.15.6:0/3445221614 <== osd.5 v2:172.21.15.61:6808/103780 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9bc8002d40 con 0x7f9bd4071fc0 2023-11-28T13:00:40.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x7f9bd4071fc0 msgr2=0x7f9bd40743f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bee28a700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x7f9bd4071fc0 0x7f9bd40743f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9bd40682f0 msgr2=0x7f9bd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bee28a700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9bd40682f0 0x7f9bd406a7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9be80c52d0 tx=0x7f9be000b040 comp rx=0 tx=0).stop 2023-11-28T13:00:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9be81588a0 msgr2=0x7f9be80b2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bee28a700 1 --2- 172.21.15.6:0/3445221614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9be81588a0 0x7f9be80b2d00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9be806bcc0 tx=0x7f9bd800c640 comp rx=0 tx=0).stop 2023-11-28T13:00:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bec827700 1 -- 172.21.15.6:0/3445221614 reap_dead start 2023-11-28T13:00:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 shutdown_connections 2023-11-28T13:00:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 >> 172.21.15.6:0/3445221614 conn(0x7f9be80cf670 msgr2=0x7f9be80cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 shutdown_connections 2023-11-28T13:00:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.899+0000 7f9bee28a700 1 -- 172.21.15.6:0/3445221614 wait complete. 2023-11-28T13:00:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.900+0000 7f38c7fff700 1 -- 172.21.15.6:0/1988193717 <== osd.6 v2:172.21.15.61:6800/103779 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f38b4002d40 con 0x7f38b0072010 2023-11-28T13:00:40.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x7f38b0072010 msgr2=0x7f38b0074440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38d5fb8700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x7f38b0072010 0x7f38b0074440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f38b0068340 msgr2=0x7f38b006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38d5fb8700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f38b0068340 0x7f38b006a7f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f38c8007fd0 tx=0x7f38c8009040 comp rx=0 tx=0).stop 2023-11-28T13:00:40.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 msgr2=0x7f38d00b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38d5fb8700 1 --2- 172.21.15.6:0/1988193717 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f38d00cd120 0x7f38d00b18b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f38c000ce40 tx=0x7f38c000a570 comp rx=0 tx=0).stop 2023-11-28T13:00:40.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38cffff700 1 -- 172.21.15.6:0/1988193717 reap_dead start 2023-11-28T13:00:40.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 shutdown_connections 2023-11-28T13:00:40.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 >> 172.21.15.6:0/1988193717 conn(0x7f38d00cf670 msgr2=0x7f38d00afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 shutdown_connections 2023-11-28T13:00:40.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.901+0000 7f38d5fb8700 1 -- 172.21.15.6:0/1988193717 wait complete. 2023-11-28T13:00:40.956 INFO:teuthology.orchestra.run.smithi006.stdout:55834574857 2023-11-28T13:00:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.898+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 shutdown_connections 2023-11-28T13:00:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.898+0000 7fea129d6700 1 -- 172.21.15.6:0/2106906386 wait complete. 2023-11-28T13:00:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.909+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 --> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7ae4002d40 con 0x7f7ae0072010 2023-11-28T13:00:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.910+0000 7f7af77fe700 1 -- 172.21.15.6:0/3591398370 <== osd.4 v2:172.21.15.61:6816/103782 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7ae4002d40 con 0x7f7ae0072010 2023-11-28T13:00:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.911+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 >> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] conn(0x7f7ae0072010 msgr2=0x7f7ae0074440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.911+0000 7f7b04b6f700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] conn(0x7f7ae0072010 0x7f7ae0074440 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.913+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7ae0068340 msgr2=0x7f7ae006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.913+0000 7f7b04b6f700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7ae0068340 0x7f7ae006a7f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b000710d0 tx=0x7f7ae8009040 comp rx=0 tx=0).stop 2023-11-28T13:00:40.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.913+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 msgr2=0x7f7b0015dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.913+0000 7f7b04b6f700 1 --2- 172.21.15.6:0/3591398370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7b000b48d0 0x7f7b0015dd20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7af000cdc0 tx=0x7f7af000a570 comp rx=0 tx=0).stop 2023-11-28T13:00:40.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.913+0000 7f7afed9d700 1 -- 172.21.15.6:0/3591398370 reap_dead start 2023-11-28T13:00:40.958 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T13:00:40.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.915+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 shutdown_connections 2023-11-28T13:00:40.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.915+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 >> 172.21.15.6:0/3591398370 conn(0x7f7b000cf8d0 msgr2=0x7f7b0016be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.916+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 --> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f743c0675a0 con 0x7f7428072090 2023-11-28T13:00:40.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.916+0000 7f7423fff700 1 -- 172.21.15.6:0/436869894 <== osd.7 v2:172.21.15.61:6824/103806 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f743c0675a0 con 0x7f7428072090 2023-11-28T13:00:40.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.917+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 >> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] conn(0x7f7428072090 msgr2=0x7f74280744c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.917+0000 7f7441ed7700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] conn(0x7f7428072090 0x7f74280744c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:40.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.917+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f74280683c0 msgr2=0x7f742806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.917+0000 7f7441ed7700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f74280683c0 0x7f742806a870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f742c0097f0 tx=0x7f742c008040 comp rx=0 tx=0).stop 2023-11-28T13:00:40.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.917+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f743c0ce230 msgr2=0x7f743c0c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:40.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.917+0000 7f7441ed7700 1 --2- 172.21.15.6:0/436869894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f743c0ce230 0x7f743c0c8a90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f743c0aeb70 tx=0x7f7430004750 comp rx=0 tx=0).stop 2023-11-28T13:00:40.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.917+0000 7f743bfff700 1 -- 172.21.15.6:0/436869894 reap_dead start 2023-11-28T13:00:40.962 INFO:teuthology.orchestra.run.smithi006.stdout:55834574855 2023-11-28T13:00:40.962 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T13:00:40.963 INFO:teuthology.orchestra.run.smithi006.stdout:55834574858 2023-11-28T13:00:40.963 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T13:00:40.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.919+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 shutdown_connections 2023-11-28T13:00:40.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.919+0000 7f7b04b6f700 1 -- 172.21.15.6:0/3591398370 wait complete. 2023-11-28T13:00:40.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.921+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 shutdown_connections 2023-11-28T13:00:40.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.921+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 >> 172.21.15.6:0/436869894 conn(0x7f743c0b0dd0 msgr2=0x7f743c06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:40.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.921+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 shutdown_connections 2023-11-28T13:00:40.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:40.921+0000 7f7441ed7700 1 -- 172.21.15.6:0/436869894 wait complete. 2023-11-28T13:00:40.965 INFO:teuthology.orchestra.run.smithi006.stdout:55834574857 2023-11-28T13:00:40.965 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T13:00:40.966 INFO:teuthology.orchestra.run.smithi006.stdout:55834574857 2023-11-28T13:00:40.967 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T13:00:40.967 INFO:teuthology.orchestra.run.smithi006.stdout:55834574857 2023-11-28T13:00:40.968 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T13:00:40.968 INFO:teuthology.orchestra.run.smithi006.stdout:55834574858 2023-11-28T13:00:40.969 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T13:00:40.969 INFO:teuthology.orchestra.run.smithi006.stdout:55834574858 2023-11-28T13:00:40.970 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T13:00:41.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.213+0000 7fe715441700 1 Processor -- start 2023-11-28T13:00:41.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.213+0000 7fe715441700 1 -- start start 2023-11-28T13:00:41.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.215+0000 7fe715441700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7101588a0 0x7fe7100ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.215+0000 7fe715441700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe7100c9920 0x7fe7100c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.215+0000 7fe715441700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71006a280 con 0x7fe7100cd120 2023-11-28T13:00:41.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.215+0000 7fe715441700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7100ca2e0 con 0x7fe7100c9920 2023-11-28T13:00:41.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.215+0000 7fe715441700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71006a560 con 0x7fe7101588a0 2023-11-28T13:00:41.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.215+0000 7fe70e7fc700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe7100cd120 0x7fe71006b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45558/0 (socket says 172.21.15.6:45558) 2023-11-28T13:00:41.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.215+0000 7fe70e7fc700 1 -- 172.21.15.6:0/4096268644 learned_addr learned my addr 172.21.15.6:0/4096268644 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:41.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.215+0000 7fe70f7fe700 1 --2- 172.21.15.6:0/4096268644 >> v2:172.21.15.6:3300/0 conn(0x7fe7100c9920 0x7fe7100c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.215+0000 7fe70dffb700 1 -- 172.21.15.6:0/4096268644 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478523776 0 0) 0x7fe71006a280 con 0x7fe7100cd120 2023-11-28T13:00:41.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.215+0000 7fe70dffb700 1 -- 172.21.15.6:0/4096268644 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6f80079c0 con 0x7fe7100cd120 2023-11-28T13:00:41.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.216+0000 7fe70f7fe700 1 -- 172.21.15.6:0/4096268644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7101588a0 msgr2=0x7fe7100ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.216+0000 7fe70f7fe700 1 --2- 172.21.15.6:0/4096268644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7101588a0 0x7fe7100ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.216+0000 7fe70f7fe700 1 -- 172.21.15.6:0/4096268644 >> v1:172.21.15.6:6789/0 conn(0x7fe7100cd120 legacy=0x7fe71006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.216+0000 7fe70f7fe700 1 -- 172.21.15.6:0/4096268644 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71006a840 con 0x7fe7100c9920 2023-11-28T13:00:41.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.216+0000 7fe70f7fe700 1 --2- 172.21.15.6:0/4096268644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 0x7fe7100c9da0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe70400a910 tx=0x7fe70400a220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8625ab210d77afaf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.217+0000 7fe70dffb700 1 -- 172.21.15.6:0/4096268644 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7080019f0 con 0x7fe7100c9920 2023-11-28T13:00:41.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.217+0000 7fe70dffb700 1 -- 172.21.15.6:0/4096268644 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70400ac70 con 0x7fe7100c9920 2023-11-28T13:00:41.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.217+0000 7fe70dffb700 1 -- 172.21.15.6:0/4096268644 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704004030 con 0x7fe7100c9920 2023-11-28T13:00:41.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.218+0000 7fe715441700 1 -- 172.21.15.6:0/4096268644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 msgr2=0x7fe7100c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.218+0000 7fe715441700 1 --2- 172.21.15.6:0/4096268644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 0x7fe7100c9da0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe70400a910 tx=0x7fe70400a220 comp rx=0 tx=0).stop 2023-11-28T13:00:41.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.218+0000 7fe715441700 1 -- 172.21.15.6:0/4096268644 shutdown_connections 2023-11-28T13:00:41.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.218+0000 7fe715441700 1 --2- 172.21.15.6:0/4096268644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 0x7fe7100c9da0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.218+0000 7fe715441700 1 --2- 172.21.15.6:0/4096268644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7101588a0 0x7fe7100ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.218+0000 7fe715441700 1 -- 172.21.15.6:0/4096268644 >> 172.21.15.6:0/4096268644 conn(0x7fe7100cf670 msgr2=0x7fe7100cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.218+0000 7fe715441700 1 -- 172.21.15.6:0/4096268644 shutdown_connections 2023-11-28T13:00:41.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.220+0000 7f635be2a700 1 Processor -- start 2023-11-28T13:00:41.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.220+0000 7f635be2a700 1 -- start start 2023-11-28T13:00:41.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.221+0000 7f635be2a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f635406e400 0x7f635415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.221+0000 7f635be2a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635415e0b0 con 0x7f6354159590 2023-11-28T13:00:41.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.221+0000 7f635be2a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635415d120 con 0x7f63540ce890 2023-11-28T13:00:41.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.221+0000 7f635be2a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635415d6e0 con 0x7f635406e400 2023-11-28T13:00:41.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.222+0000 7f6359bc6700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6354159590 0x7f63540ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45564/0 (socket says 172.21.15.6:45564) 2023-11-28T13:00:41.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.222+0000 7f6359bc6700 1 -- 172.21.15.6:0/4232705518 learned_addr learned my addr 172.21.15.6:0/4232705518 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:41.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.222+0000 7fe715441700 1 -- 172.21.15.6:0/4096268644 wait complete. 2023-11-28T13:00:41.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.222+0000 7f63593c5700 1 --2- 172.21.15.6:0/4232705518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f635406e400 0x7f635415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.222+0000 7f63593c5700 1 -- 172.21.15.6:0/4232705518 >> v1:172.21.15.61:6789/0 conn(0x7f63540ce890 legacy=0x7f63540caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.222+0000 7f63593c5700 1 -- 172.21.15.6:0/4232705518 >> v1:172.21.15.6:6789/0 conn(0x7f6354159590 legacy=0x7f63540ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.222+0000 7f63593c5700 1 -- 172.21.15.6:0/4232705518 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63540aef60 con 0x7f635406e400 2023-11-28T13:00:41.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.222+0000 7f63593c5700 1 --2- 172.21.15.6:0/4232705518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f635406e400 0x7f635415db70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6350009870 tx=0x7f6350009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a74d67d2f883b4cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.224+0000 7fe715441700 1 Processor -- start 2023-11-28T13:00:41.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.224+0000 7fe715441700 1 -- start start 2023-11-28T13:00:41.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7fe715441700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 0x7fe71006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7fe715441700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7100cd120 0x7fe71015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7fe715441700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7101588a0 0x7fe710159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7fe715441700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71015af40 con 0x7fe7100c9920 2023-11-28T13:00:41.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7fe715441700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710158c70 con 0x7fe7100cd120 2023-11-28T13:00:41.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7fe715441700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71015ac60 con 0x7fe7101588a0 2023-11-28T13:00:41.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7fe70effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 0x7fe71006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7fe70effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 0x7fe71006d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58250/0 (socket says 172.21.15.6:58250) 2023-11-28T13:00:41.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7fe70effd700 1 -- 172.21.15.6:0/3357957419 learned_addr learned my addr 172.21.15.6:0/3357957419 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7ffb6dfeb700 1 Processor -- start 2023-11-28T13:00:41.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7ffb6dfeb700 1 -- start start 2023-11-28T13:00:41.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.224+0000 7f6358bc4700 1 -- 172.21.15.6:0/4232705518 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350004230 con 0x7f635406e400 2023-11-28T13:00:41.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7f6358bc4700 1 -- 172.21.15.6:0/4232705518 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6350004390 con 0x7f635406e400 2023-11-28T13:00:41.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7f6358bc4700 1 -- 172.21.15.6:0/4232705518 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500036a0 con 0x7f635406e400 2023-11-28T13:00:41.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7f635be2a700 1 -- 172.21.15.6:0/4232705518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f635406e400 msgr2=0x7f635415db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.225+0000 7f635be2a700 1 --2- 172.21.15.6:0/4232705518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f635406e400 0x7f635415db70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6350009870 tx=0x7f6350009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:41.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7f635be2a700 1 -- 172.21.15.6:0/4232705518 shutdown_connections 2023-11-28T13:00:41.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7f635be2a700 1 --2- 172.21.15.6:0/4232705518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f635406e400 0x7f635415db70 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6350009870 tx=0x7f6350009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:41.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7f635be2a700 1 -- 172.21.15.6:0/4232705518 >> 172.21.15.6:0/4232705518 conn(0x7f63540b1650 msgr2=0x7f63540b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7fe70f7fe700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7101588a0 0x7fe710159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7fe70effd700 1 -- 172.21.15.6:0/3357957419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7101588a0 msgr2=0x7fe710159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7fe70effd700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7101588a0 0x7fe710159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7fe70effd700 1 -- 172.21.15.6:0/3357957419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7100cd120 msgr2=0x7fe71015a2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T13:00:41.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7fe70effd700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7100cd120 0x7fe71015a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7fe70effd700 1 -- 172.21.15.6:0/3357957419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70400a540 con 0x7fe7100c9920 2023-11-28T13:00:41.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7f0031874700 1 Processor -- start 2023-11-28T13:00:41.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7f635be2a700 1 -- 172.21.15.6:0/4232705518 shutdown_connections 2023-11-28T13:00:41.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7fe70effd700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 0x7fe71006d360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe70000b4a0 tx=0x7fe70000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7fe6f77fe700 1 -- 172.21.15.6:0/3357957419 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700011830 con 0x7fe7100c9920 2023-11-28T13:00:41.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7fe6f77fe700 1 -- 172.21.15.6:0/3357957419 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe700011990 con 0x7fe7100c9920 2023-11-28T13:00:41.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7fe6f77fe700 1 -- 172.21.15.6:0/3357957419 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700011d40 con 0x7fe7100c9920 2023-11-28T13:00:41.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7fe715441700 1 -- 172.21.15.6:0/3357957419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71006ae00 con 0x7fe7100c9920 2023-11-28T13:00:41.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7fe715441700 1 -- 172.21.15.6:0/3357957419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71006ab20 con 0x7fe7100c9920 2023-11-28T13:00:41.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7ffb6dfeb700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ffb68160c90 0x7ffb680ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7ffb6dfeb700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb680ca3a0 con 0x7ffb680c2b60 2023-11-28T13:00:41.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7ffb6dfeb700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb68161410 con 0x7ffb680cca20 2023-11-28T13:00:41.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7ffb6dfeb700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb681616f0 con 0x7ffb68160c90 2023-11-28T13:00:41.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7ffb677fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ffb68160c90 0x7ffb680ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7ffb677fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ffb68160c90 0x7ffb680ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58258/0 (socket says 172.21.15.6:58258) 2023-11-28T13:00:41.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.227+0000 7ffb677fe700 1 -- 172.21.15.6:0/1352288389 learned_addr learned my addr 172.21.15.6:0/1352288389 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.226+0000 7f0031874700 1 -- start start 2023-11-28T13:00:41.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7f0031874700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f002c160c70 0x7f002c0adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7f0031874700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f002c1646c0 0x7f002c070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7f0031874700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002c071300 con 0x7f002c070ea0 2023-11-28T13:00:41.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7f0031874700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002c161040 con 0x7f002c160c70 2023-11-28T13:00:41.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7f0031874700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002c164b30 con 0x7f002c1646c0 2023-11-28T13:00:41.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7ffb677fe700 1 -- 172.21.15.6:0/1352288389 >> v1:172.21.15.61:6789/0 conn(0x7ffb680cca20 legacy=0x7ffb68164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7ffb677fe700 1 -- 172.21.15.6:0/1352288389 >> v1:172.21.15.6:6789/0 conn(0x7ffb680c2b60 legacy=0x7ffb680cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7ffb677fe700 1 -- 172.21.15.6:0/1352288389 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb681619d0 con 0x7ffb68160c90 2023-11-28T13:00:41.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7f002affd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f002c160c70 0x7f002c0adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7f002affd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f002c160c70 0x7f002c0adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58266/0 (socket says 172.21.15.6:58266) 2023-11-28T13:00:41.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7f002affd700 1 -- 172.21.15.6:0/4115064522 learned_addr learned my addr 172.21.15.6:0/4115064522 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7fe715441700 1 -- 172.21.15.6:0/3357957419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71006ba30 con 0x7fe7100c9920 2023-11-28T13:00:41.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.229+0000 7ffb677fe700 1 --2- 172.21.15.6:0/1352288389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68160c90 0x7ffb680ce320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098f0 tx=0x7ffb58009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85ebe3526bad437 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.229+0000 7f002a7fc700 1 --2- 172.21.15.6:0/4115064522 >> v2:172.21.15.61:3300/0 conn(0x7f002c1646c0 0x7f002c070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.229+0000 7f002a7fc700 1 -- 172.21.15.6:0/4115064522 >> v2:172.21.15.6:3300/0 conn(0x7f002c160c70 msgr2=0x7f002c0adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.229+0000 7f002a7fc700 1 --2- 172.21.15.6:0/4115064522 >> v2:172.21.15.6:3300/0 conn(0x7f002c160c70 0x7f002c0adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.229+0000 7f002a7fc700 1 -- 172.21.15.6:0/4115064522 >> v1:172.21.15.61:6789/0 conn(0x7f002c070ea0 legacy=0x7f002c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.229+0000 7f002a7fc700 1 -- 172.21.15.6:0/4115064522 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c164e10 con 0x7f002c1646c0 2023-11-28T13:00:41.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7f002a7fc700 1 --2- 172.21.15.6:0/4115064522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 0x7f002c070960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f002c0b4b70 tx=0x7f001c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e37e7f6b7de8e2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.228+0000 7f635be2a700 1 -- 172.21.15.6:0/4232705518 wait complete. 2023-11-28T13:00:41.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7f635be2a700 1 Processor -- start 2023-11-28T13:00:41.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.229+0000 7fa5261fb700 1 Processor -- start 2023-11-28T13:00:41.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.229+0000 7fa5261fb700 1 -- start start 2023-11-28T13:00:41.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7fa5261fb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa520089580 0x7fa5200c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7fa5261fb700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa52006b5c0 0x7fa52006d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7fa5261fb700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa520169750 con 0x7fa52006a1e0 2023-11-28T13:00:41.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7fa5261fb700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5200ca0d0 con 0x7fa52006b5c0 2023-11-28T13:00:41.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7fa5261fb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200cd9d0 con 0x7fa520089580 2023-11-28T13:00:41.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7fa51f7fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fa52006a1e0 0x7fa52006cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:35622/0 (socket says 172.21.15.6:35622) 2023-11-28T13:00:41.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7fa51f7fe700 1 -- 172.21.15.6:0/3716149115 learned_addr learned my addr 172.21.15.6:0/3716149115 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:41.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7f635be2a700 1 -- start start 2023-11-28T13:00:41.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7f0029ffb700 1 -- 172.21.15.6:0/4115064522 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c004230 con 0x7f002c1646c0 2023-11-28T13:00:41.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7f0029ffb700 1 -- 172.21.15.6:0/4115064522 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c004390 con 0x7f002c1646c0 2023-11-28T13:00:41.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7f0029ffb700 1 -- 172.21.15.6:0/4115064522 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c0036c0 con 0x7f002c1646c0 2023-11-28T13:00:41.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7ffb667fc700 1 -- 172.21.15.6:0/1352288389 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58004230 con 0x7ffb68160c90 2023-11-28T13:00:41.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7ffb667fc700 1 -- 172.21.15.6:0/1352288389 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb58004390 con 0x7ffb68160c90 2023-11-28T13:00:41.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7ffb667fc700 1 -- 172.21.15.6:0/1352288389 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb580036a0 con 0x7ffb68160c90 2023-11-28T13:00:41.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/1352288389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68160c90 msgr2=0x7ffb680ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7ffb6dfeb700 1 --2- 172.21.15.6:0/1352288389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68160c90 0x7ffb680ce320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098f0 tx=0x7ffb58009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:41.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7fa51ffff700 1 --2- 172.21.15.6:0/3716149115 >> v2:172.21.15.6:3300/0 conn(0x7fa52006b5c0 0x7fa52006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f0031874700 1 -- 172.21.15.6:0/4115064522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 msgr2=0x7f002c070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f0031874700 1 --2- 172.21.15.6:0/4115064522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 0x7f002c070960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f002c0b4b70 tx=0x7f001c009b20 comp rx=0 tx=0).stop 2023-11-28T13:00:41.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7feb4ec34700 1 Processor -- start 2023-11-28T13:00:41.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.230+0000 7feb4ec34700 1 -- start start 2023-11-28T13:00:41.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7feb4ec34700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48160be0 0x7feb480c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7feb4ec34700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb480cdb60 con 0x7feb480adec0 2023-11-28T13:00:41.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7feb4ec34700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb480cde40 con 0x7feb480c9df0 2023-11-28T13:00:41.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7feb4ec34700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb480ce120 con 0x7feb48160be0 2023-11-28T13:00:41.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/1352288389 shutdown_connections 2023-11-28T13:00:41.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7ffb6dfeb700 1 --2- 172.21.15.6:0/1352288389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68160c90 0x7ffb680ce320 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098f0 tx=0x7ffb58009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:41.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/1352288389 >> 172.21.15.6:0/1352288389 conn(0x7ffb680b1650 msgr2=0x7ffb680b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7feb47fff700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7feb480c9df0 0x7feb480ad7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:35632/0 (socket says 172.21.15.6:35632) 2023-11-28T13:00:41.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7feb47fff700 1 -- 172.21.15.6:0/1754627246 learned_addr learned my addr 172.21.15.6:0/1754627246 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:41.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f0031874700 1 -- 172.21.15.6:0/4115064522 shutdown_connections 2023-11-28T13:00:41.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f0031874700 1 --2- 172.21.15.6:0/4115064522 >> v2:172.21.15.6:3300/0 conn(0x7f002c160c70 0x7f002c0adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f0031874700 1 --2- 172.21.15.6:0/4115064522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 0x7f002c070960 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f0031874700 1 -- 172.21.15.6:0/4115064522 >> 172.21.15.6:0/4115064522 conn(0x7f002c0ce9e0 msgr2=0x7f002c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/1352288389 shutdown_connections 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7f635be2a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63540ce890 0x7f6354151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7f635be2a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6354159590 0x7f63540c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7f635be2a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63540c5b00 0x7f63540b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7f635be2a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63540c5f80 con 0x7f6354159590 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7f635be2a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6354166a60 con 0x7f63540ce890 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.231+0000 7f635be2a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63541675e0 con 0x7f63540c5b00 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f6359bc6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63540ce890 0x7f6354151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f63593c5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6354159590 0x7f63540c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f63593c5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6354159590 0x7f63540c55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58274/0 (socket says 172.21.15.6:58274) 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f63593c5700 1 -- 172.21.15.6:0/1423059518 learned_addr learned my addr 172.21.15.6:0/1423059518 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7f635a3c7700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63540c5b00 0x7f63540b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7f63593c5700 1 -- 172.21.15.6:0/1423059518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63540c5b00 msgr2=0x7f63540b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7f63593c5700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63540c5b00 0x7f63540b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7f63593c5700 1 -- 172.21.15.6:0/1423059518 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63540ce890 msgr2=0x7f6354151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7f63593c5700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63540ce890 0x7f6354151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7f63593c5700 1 -- 172.21.15.6:0/1423059518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63500094a0 con 0x7f6354159590 2023-11-28T13:00:41.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7f63593c5700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6354159590 0x7f63540c55c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6350009bc0 tx=0x7f6350004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7fa51ffff700 1 -- 172.21.15.6:0/3716149115 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa520089580 msgr2=0x7fa5200c9a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T13:00:41.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7fa51ffff700 1 --2- 172.21.15.6:0/3716149115 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa520089580 0x7fa5200c9a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7fa51ffff700 1 -- 172.21.15.6:0/3716149115 >> v1:172.21.15.61:6789/0 conn(0x7fa52006a1e0 legacy=0x7fa52006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7fa51ffff700 1 -- 172.21.15.6:0/3716149115 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200cdcb0 con 0x7fa52006b5c0 2023-11-28T13:00:41.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7fe6f77fe700 1 -- 172.21.15.6:0/3357957419 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe700021070 con 0x7fe7100c9920 2023-11-28T13:00:41.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7fe6f77fe700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe6fc0683c0 0x7fe6fc06a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7fe6f77fe700 1 -- 172.21.15.6:0/3357957419 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe700085fc0 con 0x7fe7100c9920 2023-11-28T13:00:41.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7fa51ffff700 1 --2- 172.21.15.6:0/3716149115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa52006b5c0 0x7fa52006d460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa510009840 tx=0x7fa510009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288d5fb410d8fce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7fe70e7fc700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe6fc0683c0 0x7fe6fc06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7feb4c9d0700 1 --2- 172.21.15.6:0/1754627246 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48160be0 0x7feb480c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340480607 0 0) 0x7feb480cde40 con 0x7feb480c9df0 2023-11-28T13:00:41.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.233+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb340035b0 con 0x7feb480c9df0 2023-11-28T13:00:41.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 <== mon.1 v1:172.21.15.61:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2775471690 0 0) 0x7feb340035b0 con 0x7feb480c9df0 2023-11-28T13:00:41.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7fe70e7fc700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe6fc0683c0 0x7fe6fc06a870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe708001be0 tx=0x7fe7080017b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48160be0 msgr2=0x7feb480c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 --2- 172.21.15.6:0/1754627246 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48160be0 0x7feb480c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 >> v1:172.21.15.6:6789/0 conn(0x7feb480adec0 legacy=0x7feb4816a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 --> v1:172.21.15.61:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4806a1e0 con 0x7feb480c9df0 2023-11-28T13:00:41.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 <== mon.1 v1:172.21.15.61:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7feb30001670 con 0x7feb480c9df0 2023-11-28T13:00:41.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7f0031874700 1 -- 172.21.15.6:0/4115064522 shutdown_connections 2023-11-28T13:00:41.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.232+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/1352288389 wait complete. 2023-11-28T13:00:41.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7ffb6dfeb700 1 Processor -- start 2023-11-28T13:00:41.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 >> v1:172.21.15.61:6789/0 conn(0x7feb480c9df0 legacy=0x7feb480ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 --2- 172.21.15.6:0/1754627246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb34003a00 0x7feb34003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 --2- 172.21.15.6:0/1754627246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb34004430 0x7feb340048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 --2- 172.21.15.6:0/1754627246 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48160be0 0x7feb34005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb480ce120 con 0x7feb34003a00 2023-11-28T13:00:41.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3400ba40 con 0x7feb34004430 2023-11-28T13:00:41.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3400bc00 con 0x7feb48160be0 2023-11-28T13:00:41.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7feb4c9d0700 1 --2- 172.21.15.6:0/1754627246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb34003a00 0x7feb34003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.234+0000 7f0031874700 1 -- 172.21.15.6:0/4115064522 wait complete. 2023-11-28T13:00:41.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.235+0000 7feb4c9d0700 1 -- 172.21.15.6:0/1754627246 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48160be0 msgr2=0x7feb34005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.235+0000 7feb4c9d0700 1 --2- 172.21.15.6:0/1754627246 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48160be0 0x7feb34005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.235+0000 7feb4c9d0700 1 -- 172.21.15.6:0/1754627246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb34004430 msgr2=0x7feb340048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.235+0000 7feb4c9d0700 1 --2- 172.21.15.6:0/1754627246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb34004430 0x7feb340048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.235+0000 7feb4c9d0700 1 -- 172.21.15.6:0/1754627246 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4806a1e0 con 0x7feb34003a00 2023-11-28T13:00:41.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.235+0000 7feb4c9d0700 1 --2- 172.21.15.6:0/1754627246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb34003a00 0x7feb34003e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb38009b60 tx=0x7feb34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca8a08cf2e7e4fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.235+0000 7ffb6dfeb700 1 -- start start 2023-11-28T13:00:41.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ff9b4762700 1 Processor -- start 2023-11-28T13:00:41.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ff9b4762700 1 -- start start 2023-11-28T13:00:41.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7f0031874700 1 Processor -- start 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ffb6dfeb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb680c2b60 0x7ffb680cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ffb6dfeb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb680cca20 0x7ffb681ff340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ffb6dfeb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68202f30 0x7ffb68159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ffb6dfeb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68203f00 con 0x7ffb68202f30 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ffb6dfeb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb682033d0 con 0x7ffb680c2b60 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ffb6dfeb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68203660 con 0x7ffb680cca20 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ffb67fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68202f30 0x7ffb68159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ffb67fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68202f30 0x7ffb68159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58294/0 (socket says 172.21.15.6:58294) 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7ffb67fff700 1 -- 172.21.15.6:0/2775564458 learned_addr learned my addr 172.21.15.6:0/2775564458 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ffb677fe700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb680c2b60 0x7ffb680cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ffb67fff700 1 -- 172.21.15.6:0/2775564458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb680cca20 msgr2=0x7ffb681ff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ffb67fff700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb680cca20 0x7ffb681ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ffb67fff700 1 -- 172.21.15.6:0/2775564458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb680c2b60 msgr2=0x7ffb680cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ffb67fff700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb680c2b60 0x7ffb680cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ffb67fff700 1 -- 172.21.15.6:0/2775564458 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58009550 con 0x7ffb68202f30 2023-11-28T13:00:41.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ffb67fff700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68202f30 0x7ffb68159360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000cc10 tx=0x7ffb6000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ff9b4762700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac0d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ff9b4762700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff9ac0b48d0 0x7ff9ac0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ff9b4762700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac06a1e0 con 0x7ff9ac164510 2023-11-28T13:00:41.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ff9b4762700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac0b64b0 con 0x7ff9ac0b48d0 2023-11-28T13:00:41.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ff9b4762700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac160e90 con 0x7ff9ac160ac0 2023-11-28T13:00:41.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ff9b24fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac0d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ff9b24fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac0d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:48004/0 (socket says 172.21.15.6:48004) 2023-11-28T13:00:41.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ff9b24fe700 1 -- 172.21.15.6:0/594657530 learned_addr learned my addr 172.21.15.6:0/594657530 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7ff9b2cff700 1 --2- 172.21.15.6:0/594657530 >> v2:172.21.15.61:3300/0 conn(0x7ff9ac0b48d0 0x7ff9ac0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.236+0000 7f0031874700 1 -- start start 2023-11-28T13:00:41.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7f0031874700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f002c070ea0 0x7f002c150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.237+0000 7f0031874700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 0x7f002c145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f0031874700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f002c1634b0 0x7f002c14bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f0031874700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c151d10 con 0x7f002c1634b0 2023-11-28T13:00:41.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f0031874700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c145cb0 con 0x7f002c1646c0 2023-11-28T13:00:41.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f0031874700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c14b790 con 0x7f002c070ea0 2023-11-28T13:00:41.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7ff9b24fe700 1 -- 172.21.15.6:0/594657530 >> v2:172.21.15.61:3300/0 conn(0x7ff9ac0b48d0 msgr2=0x7ff9ac0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7ff9b24fe700 1 --2- 172.21.15.6:0/594657530 >> v2:172.21.15.61:3300/0 conn(0x7ff9ac0b48d0 0x7ff9ac0b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7ff9b24fe700 1 -- 172.21.15.6:0/594657530 >> v1:172.21.15.61:6789/0 conn(0x7ff9ac164510 legacy=0x7ff9ac0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7ff9b14fc700 1 -- 172.21.15.6:0/594657530 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772298008 0 0) 0x7ff9ac06a1e0 con 0x7ff9ac164510 2023-11-28T13:00:41.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f002c1634b0 0x7f002c14bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002a7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 0x7f002c145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f002c1634b0 0x7f002c14bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58310/0 (socket says 172.21.15.6:58310) 2023-11-28T13:00:41.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002a7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 0x7f002c145770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:33794/0 (socket says 172.21.15.6:33794) 2023-11-28T13:00:41.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002b7fe700 1 -- 172.21.15.6:0/1953083221 learned_addr learned my addr 172.21.15.6:0/1953083221 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7ff9b24fe700 1 -- 172.21.15.6:0/594657530 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0b5270 con 0x7ff9ac160ac0 2023-11-28T13:00:41.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7ff9b24fe700 1 --2- 172.21.15.6:0/594657530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac0d2510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80098f0 tx=0x7ff9a8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41bcc28c181f2a7f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002a7fc700 1 -- 172.21.15.6:0/1953083221 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f002c070ea0 msgr2=0x7f002c150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002a7fc700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f002c070ea0 0x7f002c150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002a7fc700 1 -- 172.21.15.6:0/1953083221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f002c1634b0 msgr2=0x7f002c14bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002a7fc700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f002c1634b0 0x7f002c14bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002a7fc700 1 -- 172.21.15.6:0/1953083221 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c0094a0 con 0x7f002c1646c0 2023-11-28T13:00:41.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002b7fe700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f002c1634b0 0x7f002c14bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:41.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7ff9b14fc700 1 -- 172.21.15.6:0/594657530 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8011830 con 0x7ff9ac160ac0 2023-11-28T13:00:41.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7ff9b14fc700 1 -- 172.21.15.6:0/594657530 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a8011990 con 0x7ff9ac160ac0 2023-11-28T13:00:41.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7ff9b14fc700 1 -- 172.21.15.6:0/594657530 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8011d20 con 0x7ff9ac160ac0 2023-11-28T13:00:41.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.238+0000 7f002a7fc700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 0x7f002c145770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f001c013000 tx=0x7f001c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7f0013fff700 1 -- 172.21.15.6:0/1953083221 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c011630 con 0x7f002c1646c0 2023-11-28T13:00:41.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7f0013fff700 1 -- 172.21.15.6:0/1953083221 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c011790 con 0x7f002c1646c0 2023-11-28T13:00:41.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7f0031874700 1 -- 172.21.15.6:0/1953083221 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c1653d0 con 0x7f002c1646c0 2023-11-28T13:00:41.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7f0013fff700 1 -- 172.21.15.6:0/1953083221 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c011b00 con 0x7f002c1646c0 2023-11-28T13:00:41.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7ff9b4762700 1 -- 172.21.15.6:0/594657530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 msgr2=0x7ff9ac0d2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7ff9b4762700 1 --2- 172.21.15.6:0/594657530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac0d2510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80098f0 tx=0x7ff9a8009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:41.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800e6c0 con 0x7feb34003a00 2023-11-28T13:00:41.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7fe6f77fe700 1 -- 172.21.15.6:0/3357957419 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe700052440 con 0x7fe7100c9920 2023-11-28T13:00:41.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7ff9b4762700 1 -- 172.21.15.6:0/594657530 shutdown_connections 2023-11-28T13:00:41.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7ff9b4762700 1 --2- 172.21.15.6:0/594657530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac0d2510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7ff9b4762700 1 --2- 172.21.15.6:0/594657530 >> v2:172.21.15.61:3300/0 conn(0x7ff9ac0b48d0 0x7ff9ac0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7ff9b4762700 1 -- 172.21.15.6:0/594657530 >> 172.21.15.6:0/594657530 conn(0x7ff9ac0cf8d0 msgr2=0x7ff9ac0cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.240+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3800e820 con 0x7feb34003a00 2023-11-28T13:00:41.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.240+0000 7feb477fe700 1 -- 172.21.15.6:0/1754627246 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3800eb90 con 0x7feb34003a00 2023-11-28T13:00:41.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.239+0000 7f0031874700 1 -- 172.21.15.6:0/1953083221 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c1650f0 con 0x7f002c1646c0 2023-11-28T13:00:41.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.240+0000 7ff9b4762700 1 -- 172.21.15.6:0/594657530 shutdown_connections 2023-11-28T13:00:41.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.240+0000 7ff9b4762700 1 -- 172.21.15.6:0/594657530 wait complete. 2023-11-28T13:00:41.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.240+0000 7f0031874700 1 -- 172.21.15.6:0/1953083221 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c0ae040 con 0x7f002c1646c0 2023-11-28T13:00:41.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.240+0000 7feb4ec34700 1 -- 172.21.15.6:0/1754627246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb34003a00 msgr2=0x7feb34003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.240+0000 7feb4ec34700 1 --2- 172.21.15.6:0/1754627246 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb34003a00 0x7feb34003e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb38009b60 tx=0x7feb34005880 comp rx=0 tx=0).stop 2023-11-28T13:00:41.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.240+0000 7feb4d1d1700 1 -- 172.21.15.6:0/1754627246 reap_dead start 2023-11-28T13:00:41.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7f634a7fc700 1 -- 172.21.15.6:0/1423059518 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350003d70 con 0x7f6354159590 2023-11-28T13:00:41.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7f634a7fc700 1 -- 172.21.15.6:0/1423059518 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6350020070 con 0x7f6354159590 2023-11-28T13:00:41.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7f634a7fc700 1 -- 172.21.15.6:0/1423059518 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350011850 con 0x7f6354159590 2023-11-28T13:00:41.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7f635be2a700 1 -- 172.21.15.6:0/1423059518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635415d370 con 0x7f6354159590 2023-11-28T13:00:41.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7f635be2a700 1 -- 172.21.15.6:0/1423059518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63540af240 con 0x7f6354159590 2023-11-28T13:00:41.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7fa51e7fc700 1 -- 172.21.15.6:0/3716149115 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510004230 con 0x7fa52006b5c0 2023-11-28T13:00:41.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7fa51e7fc700 1 -- 172.21.15.6:0/3716149115 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa510004390 con 0x7fa52006b5c0 2023-11-28T13:00:41.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7fa51e7fc700 1 -- 172.21.15.6:0/3716149115 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51000c700 con 0x7fa52006b5c0 2023-11-28T13:00:41.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7fa5261fb700 1 -- 172.21.15.6:0/3716149115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa52006b5c0 msgr2=0x7fa52006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7fa5261fb700 1 --2- 172.21.15.6:0/3716149115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa52006b5c0 0x7fa52006d460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa510009840 tx=0x7fa510009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:41.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7fa5261fb700 1 -- 172.21.15.6:0/3716149115 shutdown_connections 2023-11-28T13:00:41.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7fa5261fb700 1 --2- 172.21.15.6:0/3716149115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa52006b5c0 0x7fa52006d460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7fa5261fb700 1 --2- 172.21.15.6:0/3716149115 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa520089580 0x7fa5200c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7fa5261fb700 1 -- 172.21.15.6:0/3716149115 >> 172.21.15.6:0/3716149115 conn(0x7fa5200cf670 msgr2=0x7fa5200cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7fa5261fb700 1 -- 172.21.15.6:0/3716149115 shutdown_connections 2023-11-28T13:00:41.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7fa5261fb700 1 -- 172.21.15.6:0/3716149115 wait complete. 2023-11-28T13:00:41.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.240+0000 7feb4ec34700 1 -- 172.21.15.6:0/1754627246 shutdown_connections 2023-11-28T13:00:41.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.240+0000 7feb4ec34700 1 -- 172.21.15.6:0/1754627246 >> 172.21.15.6:0/1754627246 conn(0x7feb480cf670 msgr2=0x7feb480cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7feb4ec34700 1 -- 172.21.15.6:0/1754627246 shutdown_connections 2023-11-28T13:00:41.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7feb4ec34700 1 -- 172.21.15.6:0/1754627246 wait complete. 2023-11-28T13:00:41.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7feb4ec34700 1 Processor -- start 2023-11-28T13:00:41.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7ffb4ffff700 1 -- 172.21.15.6:0/2775564458 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000ab10 con 0x7ffb68202f30 2023-11-28T13:00:41.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7ffb4ffff700 1 -- 172.21.15.6:0/2775564458 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6000ac70 con 0x7ffb68202f30 2023-11-28T13:00:41.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7ffb4ffff700 1 -- 172.21.15.6:0/2775564458 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60012420 con 0x7ffb68202f30 2023-11-28T13:00:41.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/2775564458 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb68161f90 con 0x7ffb68202f30 2023-11-28T13:00:41.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/2775564458 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb68161cb0 con 0x7ffb68202f30 2023-11-28T13:00:41.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7f0013fff700 1 -- 172.21.15.6:0/1953083221 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f001c00f050 con 0x7f002c1646c0 2023-11-28T13:00:41.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7f0013fff700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00180682f0 0x7f001806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.241+0000 7f0013fff700 1 -- 172.21.15.6:0/1953083221 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f001c087530 con 0x7f002c1646c0 2023-11-28T13:00:41.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7ff9b4762700 1 Processor -- start 2023-11-28T13:00:41.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.243+0000 7f002affd700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00180682f0 0x7f001806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.243+0000 7f002affd700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00180682f0 0x7f001806a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f002c0cf7e0 tx=0x7f0020008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7fa5261fb700 1 Processor -- start 2023-11-28T13:00:41.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.242+0000 7ff9b4762700 1 -- start start 2023-11-28T13:00:41.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b4762700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9ac0b48d0 0x7ff9ac150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b4762700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac14d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b4762700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac164510 0x7ff9ac06f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b4762700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac14f0d0 con 0x7ff9ac164510 2023-11-28T13:00:41.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b4762700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac151130 con 0x7ff9ac0b48d0 2023-11-28T13:00:41.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b4762700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac14e4c0 con 0x7ff9ac160ac0 2023-11-28T13:00:41.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b1cfd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac14d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b24fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9ac0b48d0 0x7ff9ac150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b1cfd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac14d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:48020/0 (socket says 172.21.15.6:48020) 2023-11-28T13:00:41.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b1cfd700 1 -- 172.21.15.6:0/856651994 learned_addr learned my addr 172.21.15.6:0/856651994 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b1cfd700 1 -- 172.21.15.6:0/856651994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9ac0b48d0 msgr2=0x7ff9ac150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b1cfd700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9ac0b48d0 0x7ff9ac150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b1cfd700 1 -- 172.21.15.6:0/856651994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac164510 msgr2=0x7ff9ac06f400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T13:00:41.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b1cfd700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac164510 0x7ff9ac06f400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b1cfd700 1 -- 172.21.15.6:0/856651994 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a8009550 con 0x7ff9ac160ac0 2023-11-28T13:00:41.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7ff9b1cfd700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac14d800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c480 tx=0x7ff99c00c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7f634a7fc700 1 -- 172.21.15.6:0/1423059518 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f635002b020 con 0x7f6354159590 2023-11-28T13:00:41.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7fa5261fb700 1 -- start start 2023-11-28T13:00:41.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7fa5261fb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa52006a1e0 0x7fa520208aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7fa5261fb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa52006b5c0 0x7fa52020a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7fa5261fb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa520089580 0x7fa5200c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7fa5261fb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa520201e30 con 0x7fa520089580 2023-11-28T13:00:41.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7fa5261fb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa520208fe0 con 0x7fa52006a1e0 2023-11-28T13:00:41.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7fa5261fb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52020ac80 con 0x7fa52006b5c0 2023-11-28T13:00:41.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7fa51f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa52006a1e0 0x7fa520208aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7fa51f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa52006a1e0 0x7fa520208aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:33810/0 (socket says 172.21.15.6:33810) 2023-11-28T13:00:41.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7fa51f7fe700 1 -- 172.21.15.6:0/113181079 learned_addr learned my addr 172.21.15.6:0/113181079 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.244+0000 7feb4ec34700 1 -- start start 2023-11-28T13:00:41.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7feb4ec34700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb34003a00 0x7feb4814e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7feb4ec34700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48151b40 0x7feb48151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7feb4ec34700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb480b38c0 0x7feb480b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7feb4ec34700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb480b42a0 con 0x7feb480b38c0 2023-11-28T13:00:41.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7feb4ec34700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4814ed40 con 0x7feb34003a00 2023-11-28T13:00:41.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7feb4ec34700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb481525c0 con 0x7feb48151b40 2023-11-28T13:00:41.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7feb4c9d0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb34003a00 0x7feb4814e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7feb4c9d0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb34003a00 0x7feb4814e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:33804/0 (socket says 172.21.15.6:33804) 2023-11-28T13:00:41.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7feb4c9d0700 1 -- 172.21.15.6:0/1565356910 learned_addr learned my addr 172.21.15.6:0/1565356910 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.245+0000 7feb4d1d1700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb480b38c0 0x7feb480b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7feb4d1d1700 1 -- 172.21.15.6:0/1565356910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48151b40 msgr2=0x7feb48151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7feb4d1d1700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48151b40 0x7feb48151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7feb4d1d1700 1 -- 172.21.15.6:0/1565356910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb34003a00 msgr2=0x7feb4814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7feb4d1d1700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb34003a00 0x7feb4814e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7feb4d1d1700 1 -- 172.21.15.6:0/1565356910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4806cd50 con 0x7feb480b38c0 2023-11-28T13:00:41.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7ffb4ffff700 1 -- 172.21.15.6:0/2775564458 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffb60010050 con 0x7ffb68202f30 2023-11-28T13:00:41.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7f634a7fc700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f63380682f0 0x7f633806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7fa51ffff700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa520089580 0x7fa5200c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7feb4d1d1700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb480b38c0 0x7feb480b3d60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb480673f0 tx=0x7feb4000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.246+0000 7f634a7fc700 1 -- 172.21.15.6:0/1423059518 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f63500879c0 con 0x7f6354159590 2023-11-28T13:00:41.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7feb457fa700 1 -- 172.21.15.6:0/1565356910 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4000a520 con 0x7feb480b38c0 2023-11-28T13:00:41.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7feb457fa700 1 -- 172.21.15.6:0/1565356910 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4000a6a0 con 0x7feb480b38c0 2023-11-28T13:00:41.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7feb457fa700 1 -- 172.21.15.6:0/1565356910 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4000aa50 con 0x7feb480b38c0 2023-11-28T13:00:41.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7feb4ec34700 1 -- 172.21.15.6:0/1565356910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4806a4c0 con 0x7feb480b38c0 2023-11-28T13:00:41.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7fa51ffff700 1 -- 172.21.15.6:0/113181079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa52006b5c0 msgr2=0x7fa52020a740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:00:41.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7fa51ffff700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa52006b5c0 0x7fa52020a740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7fa51ffff700 1 -- 172.21.15.6:0/113181079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa52006a1e0 msgr2=0x7fa520208aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7feb4ec34700 1 -- 172.21.15.6:0/1565356910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb480cd830 con 0x7feb480b38c0 2023-11-28T13:00:41.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7fa51ffff700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa52006a1e0 0x7fa520208aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7fa51ffff700 1 -- 172.21.15.6:0/113181079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100094a0 con 0x7fa520089580 2023-11-28T13:00:41.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7fa51ffff700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa520089580 0x7fa5200c7d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa510016000 tx=0x7fa510004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7ffb4ffff700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffb540682f0 0x7ffb5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7ffb4ffff700 1 -- 172.21.15.6:0/2775564458 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffb600862e0 con 0x7ffb68202f30 2023-11-28T13:00:41.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7ffb677fe700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffb540682f0 0x7ffb5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.247+0000 7fa507fff700 1 -- 172.21.15.6:0/113181079 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51001d070 con 0x7fa520089580 2023-11-28T13:00:41.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.248+0000 7f635be2a700 1 -- 172.21.15.6:0/1423059518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c0052f0 con 0x7f6354159590 2023-11-28T13:00:41.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.248+0000 7f0013fff700 1 -- 172.21.15.6:0/1953083221 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f001c0539b0 con 0x7f002c1646c0 2023-11-28T13:00:41.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.248+0000 7ffb677fe700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffb540682f0 0x7ffb5406a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb680cd980 tx=0x7ffb58008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.249+0000 7feb457fa700 1 -- 172.21.15.6:0/1565356910 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb40010070 con 0x7feb480b38c0 2023-11-28T13:00:41.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.248+0000 7fa507fff700 1 -- 172.21.15.6:0/113181079 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51000cec0 con 0x7fa520089580 2023-11-28T13:00:41.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.248+0000 7fa507fff700 1 -- 172.21.15.6:0/113181079 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100038c0 con 0x7fa520089580 2023-11-28T13:00:41.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.248+0000 7fa5261fb700 1 -- 172.21.15.6:0/113181079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5200ce270 con 0x7fa520089580 2023-11-28T13:00:41.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.248+0000 7fa5261fb700 1 -- 172.21.15.6:0/113181079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5200cdf90 con 0x7fa520089580 2023-11-28T13:00:41.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.249+0000 7ff9a2ffd700 1 -- 172.21.15.6:0/856651994 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0054e0 con 0x7ff9ac160ac0 2023-11-28T13:00:41.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.249+0000 7fa5261fb700 1 -- 172.21.15.6:0/113181079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5000052f0 con 0x7fa520089580 2023-11-28T13:00:41.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.249+0000 7feb457fa700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb280682f0 0x7feb2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.249+0000 7feb457fa700 1 -- 172.21.15.6:0/1565356910 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7feb40085030 con 0x7feb480b38c0 2023-11-28T13:00:41.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.249+0000 7feb4c9d0700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb280682f0 0x7feb2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.250+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/2775564458 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb68161060 con 0x7ffb68202f30 2023-11-28T13:00:41.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.249+0000 7ff9a2ffd700 1 -- 172.21.15.6:0/856651994 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c003680 con 0x7ff9ac160ac0 2023-11-28T13:00:41.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.249+0000 7ff9a2ffd700 1 -- 172.21.15.6:0/856651994 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c003a10 con 0x7ff9ac160ac0 2023-11-28T13:00:41.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.250+0000 7ff9b4762700 1 -- 172.21.15.6:0/856651994 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac164cd0 con 0x7ff9ac160ac0 2023-11-28T13:00:41.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.250+0000 7feb4c9d0700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb280682f0 0x7feb2806a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feb3800a010 tx=0x7feb3801f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.250+0000 7ff9b4762700 1 -- 172.21.15.6:0/856651994 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac1649f0 con 0x7ff9ac160ac0 2023-11-28T13:00:41.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.250+0000 7f6359bc6700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f63380682f0 0x7f633806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.251+0000 7f6359bc6700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f63380682f0 0x7f633806a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6344007c20 tx=0x7f6344007d70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.251+0000 7ff9b4762700 1 -- 172.21.15.6:0/856651994 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac164fb0 con 0x7ff9ac160ac0 2023-11-28T13:00:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.251+0000 7feb4ec34700 1 -- 172.21.15.6:0/1565356910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb48068570 con 0x7feb480b38c0 2023-11-28T13:00:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.252+0000 7ff9a2ffd700 1 -- 172.21.15.6:0/856651994 <== mon.2 v2:172.21.15.61:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff99c019030 con 0x7ff9ac160ac0 2023-11-28T13:00:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.253+0000 7ff9a2ffd700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff990068340 0x7ff99006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.253+0000 7ff9a2ffd700 1 -- 172.21.15.6:0/856651994 <== mon.2 v2:172.21.15.61:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff99c085340 con 0x7ff9ac160ac0 2023-11-28T13:00:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.253+0000 7ff9b24fe700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff990068340 0x7ff99006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.253+0000 7ff9b24fe700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff990068340 0x7ff99006a7f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8000c00 tx=0x7ff9a8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.254+0000 7f634a7fc700 1 -- 172.21.15.6:0/1423059518 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6350085e60 con 0x7f6354159590 2023-11-28T13:00:41.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.256+0000 7fa507fff700 1 -- 172.21.15.6:0/113181079 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa51002a020 con 0x7fa520089580 2023-11-28T13:00:41.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.256+0000 7fa507fff700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa50c068340 0x7fa50c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.256+0000 7fa51f7fe700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa50c068340 0x7fa50c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.257+0000 7fa51f7fe700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa50c068340 0x7fa50c06a7f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa518004bd0 tx=0x7fa5180074c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.258+0000 7ff9a2ffd700 1 -- 172.21.15.6:0/856651994 <== mon.2 v2:172.21.15.61:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff99c051840 con 0x7ff9ac160ac0 2023-11-28T13:00:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.260+0000 7f8841431700 1 Processor -- start 2023-11-28T13:00:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.260+0000 7f8841431700 1 -- start start 2023-11-28T13:00:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.261+0000 7f8841431700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f883c0ce320 0x7f883c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.261+0000 7f8841431700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f883c0cc850 0x7f883c164960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.261+0000 7f8841431700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883c0ca3a0 con 0x7f883c160c90 2023-11-28T13:00:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.261+0000 7f8841431700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883c161410 con 0x7f883c0cc850 2023-11-28T13:00:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.261+0000 7f8841431700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c1616f0 con 0x7f883c0ce320 2023-11-28T13:00:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.261+0000 7f883b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f883c0cc850 0x7f883c164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.261+0000 7f883b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f883c0cc850 0x7f883c164960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58342/0 (socket says 172.21.15.6:58342) 2023-11-28T13:00:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.261+0000 7f883b7fe700 1 -- 172.21.15.6:0/239235403 learned_addr learned my addr 172.21.15.6:0/239235403 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.262+0000 7f883a7fc700 1 --2- 172.21.15.6:0/239235403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f883c0ce320 0x7f883c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.262+0000 7f883b7fe700 1 -- 172.21.15.6:0/239235403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f883c0ce320 msgr2=0x7f883c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.262+0000 7f883b7fe700 1 --2- 172.21.15.6:0/239235403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f883c0ce320 0x7f883c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.262+0000 7f883b7fe700 1 -- 172.21.15.6:0/239235403 >> v1:172.21.15.61:6789/0 conn(0x7f883c160c90 legacy=0x7f883c0c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.262+0000 7f883b7fe700 1 -- 172.21.15.6:0/239235403 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c1619d0 con 0x7f883c0cc850 2023-11-28T13:00:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.262+0000 7f883b7fe700 1 --2- 172.21.15.6:0/239235403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 0x7f883c164960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8834009e50 tx=0x7f883400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773f556561036267 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.263+0000 7ffb4ffff700 1 -- 172.21.15.6:0/2775564458 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffb60052760 con 0x7ffb68202f30 2023-11-28T13:00:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.272+0000 7feb457fa700 1 -- 172.21.15.6:0/1565356910 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb4004a5e0 con 0x7feb480b38c0 2023-11-28T13:00:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.272+0000 7f8839ffb700 1 -- 172.21.15.6:0/239235403 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883400e070 con 0x7f883c0cc850 2023-11-28T13:00:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.275+0000 7fa507fff700 1 -- 172.21.15.6:0/113181079 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa51005fd80 con 0x7fa520089580 2023-11-28T13:00:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.281+0000 7fa507fff700 1 -- 172.21.15.6:0/113181079 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa510054030 con 0x7fa520089580 2023-11-28T13:00:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.281+0000 7f8839ffb700 1 -- 172.21.15.6:0/239235403 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883400b910 con 0x7f883c0cc850 2023-11-28T13:00:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.281+0000 7f8839ffb700 1 -- 172.21.15.6:0/239235403 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883400bcc0 con 0x7f883c0cc850 2023-11-28T13:00:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.282+0000 7f8841431700 1 -- 172.21.15.6:0/239235403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 msgr2=0x7f883c164960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.282+0000 7f8841431700 1 --2- 172.21.15.6:0/239235403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 0x7f883c164960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8834009e50 tx=0x7f883400b330 comp rx=0 tx=0).stop 2023-11-28T13:00:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.282+0000 7f8841431700 1 -- 172.21.15.6:0/239235403 shutdown_connections 2023-11-28T13:00:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.282+0000 7f8841431700 1 --2- 172.21.15.6:0/239235403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 0x7f883c164960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.282+0000 7f8841431700 1 --2- 172.21.15.6:0/239235403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f883c0ce320 0x7f883c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.282+0000 7f8841431700 1 -- 172.21.15.6:0/239235403 >> 172.21.15.6:0/239235403 conn(0x7f883c0b1650 msgr2=0x7f883c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.284+0000 7f8841431700 1 -- 172.21.15.6:0/239235403 shutdown_connections 2023-11-28T13:00:41.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.284+0000 7f8841431700 1 -- 172.21.15.6:0/239235403 wait complete. 2023-11-28T13:00:41.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.286+0000 7f8841431700 1 Processor -- start 2023-11-28T13:00:41.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.286+0000 7f8841431700 1 -- start start 2023-11-28T13:00:41.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.288+0000 7f8841431700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 0x7f883c06cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.288+0000 7f8841431700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f883c0ce320 0x7f883c14c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.288+0000 7f8841431700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f883c160c90 0x7f883c14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.288+0000 7f8841431700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c06d460 con 0x7f883c0cc850 2023-11-28T13:00:41.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.288+0000 7f8841431700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c0c7a40 con 0x7f883c160c90 2023-11-28T13:00:41.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.288+0000 7f8841431700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c0c7d20 con 0x7f883c0ce320 2023-11-28T13:00:41.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.290+0000 7f883affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 0x7f883c06cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.290+0000 7f883affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 0x7f883c06cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58346/0 (socket says 172.21.15.6:58346) 2023-11-28T13:00:41.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.290+0000 7f883affd700 1 -- 172.21.15.6:0/2594633554 learned_addr learned my addr 172.21.15.6:0/2594633554 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.291+0000 7f883affd700 1 -- 172.21.15.6:0/2594633554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f883c0ce320 msgr2=0x7f883c14c1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:00:41.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.291+0000 7f883affd700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f883c0ce320 0x7f883c14c1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.291+0000 7f883affd700 1 -- 172.21.15.6:0/2594633554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f883c160c90 msgr2=0x7f883c14aac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T13:00:41.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.291+0000 7f883affd700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f883c160c90 0x7f883c14aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.291+0000 7f883affd700 1 -- 172.21.15.6:0/2594633554 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8834009b30 con 0x7f883c0cc850 2023-11-28T13:00:41.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.292+0000 7f883affd700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 0x7f883c06cf20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f882c00c450 tx=0x7f882c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.292+0000 7f88237fe700 1 -- 172.21.15.6:0/2594633554 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c0052d0 con 0x7f883c0cc850 2023-11-28T13:00:41.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.292+0000 7f88237fe700 1 -- 172.21.15.6:0/2594633554 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c005430 con 0x7f883c0cc850 2023-11-28T13:00:41.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.292+0000 7f88237fe700 1 -- 172.21.15.6:0/2594633554 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c0038c0 con 0x7f883c0cc850 2023-11-28T13:00:41.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.292+0000 7f8841431700 1 -- 172.21.15.6:0/2594633554 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883c161f90 con 0x7f883c0cc850 2023-11-28T13:00:41.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.292+0000 7f8841431700 1 -- 172.21.15.6:0/2594633554 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c161cb0 con 0x7f883c0cc850 2023-11-28T13:00:41.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.296+0000 7f88237fe700 1 -- 172.21.15.6:0/2594633554 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f882c019030 con 0x7f883c0cc850 2023-11-28T13:00:41.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.296+0000 7f88237fe700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88280683c0 0x7f882806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.296+0000 7f88237fe700 1 -- 172.21.15.6:0/2594633554 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f882c085320 con 0x7f883c0cc850 2023-11-28T13:00:41.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.298+0000 7f8841431700 1 -- 172.21.15.6:0/2594633554 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881c0052f0 con 0x7f883c0cc850 2023-11-28T13:00:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.305+0000 7f883a7fc700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88280683c0 0x7f882806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.306+0000 7f883a7fc700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88280683c0 0x7f882806a870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f883c14bfb0 tx=0x7f8830009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.307+0000 7f88237fe700 1 -- 172.21.15.6:0/2594633554 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f882c051820 con 0x7f883c0cc850 2023-11-28T13:00:41.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.649+0000 7fe715441700 1 -- 172.21.15.6:0/3357957419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe710068570 con 0x7fe7100c9920 2023-11-28T13:00:41.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.649+0000 7fe6f77fe700 1 -- 172.21.15.6:0/3357957419 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe700015020 con 0x7fe7100c9920 2023-11-28T13:00:41.654 INFO:teuthology.orchestra.run.smithi006.stdout:55834574857 2023-11-28T13:00:41.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.653+0000 7fe715441700 1 -- 172.21.15.6:0/3357957419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe6fc0683c0 msgr2=0x7fe6fc06a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.653+0000 7fe715441700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe6fc0683c0 0x7fe6fc06a870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe708001be0 tx=0x7fe7080017b0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.653+0000 7fe715441700 1 -- 172.21.15.6:0/3357957419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 msgr2=0x7fe71006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.653+0000 7fe715441700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 0x7fe71006d360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe70000b4a0 tx=0x7fe70000b860 comp rx=0 tx=0).stop 2023-11-28T13:00:41.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.653+0000 7fe715441700 1 -- 172.21.15.6:0/3357957419 shutdown_connections 2023-11-28T13:00:41.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.653+0000 7fe715441700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe6fc0683c0 0x7fe6fc06a870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.653+0000 7fe715441700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe7101588a0 0x7fe710159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.653+0000 7fe715441700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe7100c9920 0x7fe71006d360 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.653+0000 7fe715441700 1 --2- 172.21.15.6:0/3357957419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7100cd120 0x7fe71015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.653+0000 7fe715441700 1 -- 172.21.15.6:0/3357957419 >> 172.21.15.6:0/3357957419 conn(0x7fe7100cf670 msgr2=0x7fe71006b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.654+0000 7fe715441700 1 -- 172.21.15.6:0/3357957419 shutdown_connections 2023-11-28T13:00:41.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.654+0000 7fe715441700 1 -- 172.21.15.6:0/3357957419 wait complete. 2023-11-28T13:00:41.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.654+0000 7f635be2a700 1 -- 172.21.15.6:0/1423059518 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f633c0055a0 con 0x7f6354159590 2023-11-28T13:00:41.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.655+0000 7f634a7fc700 1 -- 172.21.15.6:0/1423059518 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6350053ed0 con 0x7f6354159590 2023-11-28T13:00:41.663 INFO:teuthology.orchestra.run.smithi006.stdout:55834574857 2023-11-28T13:00:41.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.655+0000 7f0031874700 1 -- 172.21.15.6:0/1953083221 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f002c068570 con 0x7f002c1646c0 2023-11-28T13:00:41.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.657+0000 7f0013fff700 1 -- 172.21.15.6:0/1953083221 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f001c056fd0 con 0x7f002c1646c0 2023-11-28T13:00:41.666 INFO:teuthology.orchestra.run.smithi006.stdout:55834574857 2023-11-28T13:00:41.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.660+0000 7ff9b4762700 1 -- 172.21.15.6:0/856651994 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff9ac0675a0 con 0x7ff9ac160ac0 2023-11-28T13:00:41.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f635be2a700 1 -- 172.21.15.6:0/1423059518 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f63380682f0 msgr2=0x7f633806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f635be2a700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f63380682f0 0x7f633806a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6344007c20 tx=0x7f6344007d70 comp rx=0 tx=0).stop 2023-11-28T13:00:41.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f635be2a700 1 -- 172.21.15.6:0/1423059518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6354159590 msgr2=0x7f63540c55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f635be2a700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6354159590 0x7f63540c55c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6350009bc0 tx=0x7f6350004750 comp rx=0 tx=0).stop 2023-11-28T13:00:41.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 -- 172.21.15.6:0/1953083221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00180682f0 msgr2=0x7f001806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00180682f0 0x7f001806a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f002c0cf7e0 tx=0x7f0020008040 comp rx=0 tx=0).stop 2023-11-28T13:00:41.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 -- 172.21.15.6:0/1953083221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 msgr2=0x7f002c145770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 0x7f002c145770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f001c013000 tx=0x7f001c004750 comp rx=0 tx=0).stop 2023-11-28T13:00:41.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 -- 172.21.15.6:0/1953083221 shutdown_connections 2023-11-28T13:00:41.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f002c1634b0 0x7f002c14bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00180682f0 0x7f001806a7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f002c070ea0 0x7f002c150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 --2- 172.21.15.6:0/1953083221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f002c1646c0 0x7f002c145770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 -- 172.21.15.6:0/1953083221 >> 172.21.15.6:0/1953083221 conn(0x7f002c0ce9e0 msgr2=0x7f002c0b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 -- 172.21.15.6:0/1953083221 shutdown_connections 2023-11-28T13:00:41.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.661+0000 7f0031874700 1 -- 172.21.15.6:0/1953083221 wait complete. 2023-11-28T13:00:41.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.662+0000 7f635be2a700 1 -- 172.21.15.6:0/1423059518 shutdown_connections 2023-11-28T13:00:41.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.662+0000 7f635be2a700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f63380682f0 0x7f633806a7a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.662+0000 7f635be2a700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63540c5b00 0x7f63540b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.662+0000 7f635be2a700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f63540ce890 0x7f6354151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.662+0000 7f635be2a700 1 --2- 172.21.15.6:0/1423059518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6354159590 0x7f63540c55c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.662+0000 7f635be2a700 1 -- 172.21.15.6:0/1423059518 >> 172.21.15.6:0/1423059518 conn(0x7f63540b1650 msgr2=0x7f63540b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.663+0000 7ff9a2ffd700 1 -- 172.21.15.6:0/856651994 <== mon.2 v2:172.21.15.61:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff99c017020 con 0x7ff9ac160ac0 2023-11-28T13:00:41.678 INFO:teuthology.orchestra.run.smithi006.stdout:55834574858 2023-11-28T13:00:41.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.667+0000 7feb4ec34700 1 -- 172.21.15.6:0/1565356910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7feb4806bd20 con 0x7feb480b38c0 2023-11-28T13:00:41.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.667+0000 7f635be2a700 1 -- 172.21.15.6:0/1423059518 shutdown_connections 2023-11-28T13:00:41.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.667+0000 7f635be2a700 1 -- 172.21.15.6:0/1423059518 wait complete. 2023-11-28T13:00:41.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.668+0000 7feb457fa700 1 -- 172.21.15.6:0/1565356910 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feb40017020 con 0x7feb480b38c0 2023-11-28T13:00:41.679 INFO:teuthology.orchestra.run.smithi006.stdout:55834574858 2023-11-28T13:00:41.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.668+0000 7ff9b4762700 1 -- 172.21.15.6:0/856651994 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff990068340 msgr2=0x7ff99006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.668+0000 7ff9b4762700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff990068340 0x7ff99006a7f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8000c00 tx=0x7ff9a8008030 comp rx=0 tx=0).stop 2023-11-28T13:00:41.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.669+0000 7ff9b4762700 1 -- 172.21.15.6:0/856651994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 msgr2=0x7ff9ac14d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.669+0000 7ff9b4762700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac14d800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00c480 tx=0x7ff99c00c840 comp rx=0 tx=0).stop 2023-11-28T13:00:41.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7feb4ec34700 1 -- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb280682f0 msgr2=0x7feb2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7feb4ec34700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb280682f0 0x7feb2806a7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feb3800a010 tx=0x7feb3801f040 comp rx=0 tx=0).stop 2023-11-28T13:00:41.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7feb4ec34700 1 -- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb480b38c0 msgr2=0x7feb480b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7feb4ec34700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb480b38c0 0x7feb480b3d60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb480673f0 tx=0x7feb4000cd50 comp rx=0 tx=0).stop 2023-11-28T13:00:41.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/2775564458 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffb680675a0 con 0x7ffb68202f30 2023-11-28T13:00:41.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7ffb4ffff700 1 -- 172.21.15.6:0/2775564458 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb60055d80 con 0x7ffb68202f30 2023-11-28T13:00:41.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7ff9b4762700 1 -- 172.21.15.6:0/856651994 shutdown_connections 2023-11-28T13:00:41.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7ff9b4762700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac164510 0x7ff9ac06f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7ff9b4762700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9ac0b48d0 0x7ff9ac150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7ff9b4762700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff990068340 0x7ff99006a7f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7ff9b4762700 1 --2- 172.21.15.6:0/856651994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff9ac160ac0 0x7ff9ac14d800 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.672+0000 7ff9b4762700 1 -- 172.21.15.6:0/856651994 >> 172.21.15.6:0/856651994 conn(0x7ff9ac0cf8d0 msgr2=0x7ff9ac0b7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.682 INFO:teuthology.orchestra.run.smithi006.stdout:55834574857 2023-11-28T13:00:41.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.673+0000 7ff9b4762700 1 -- 172.21.15.6:0/856651994 shutdown_connections 2023-11-28T13:00:41.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.673+0000 7ff9b4762700 1 -- 172.21.15.6:0/856651994 wait complete. 2023-11-28T13:00:41.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.673+0000 7feb4ec34700 1 -- 172.21.15.6:0/1565356910 shutdown_connections 2023-11-28T13:00:41.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.673+0000 7feb4ec34700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb480b38c0 0x7feb480b3d60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.673+0000 7feb4ec34700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feb34003a00 0x7feb4814e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.673+0000 7feb4ec34700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb280682f0 0x7feb2806a7a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.673+0000 7feb4ec34700 1 --2- 172.21.15.6:0/1565356910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feb48151b40 0x7feb48151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.673+0000 7feb4ec34700 1 -- 172.21.15.6:0/1565356910 >> 172.21.15.6:0/1565356910 conn(0x7feb480cf670 msgr2=0x7feb480afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.676+0000 7feb4ec34700 1 -- 172.21.15.6:0/1565356910 shutdown_connections 2023-11-28T13:00:41.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7feb4ec34700 1 -- 172.21.15.6:0/1565356910 wait complete. 2023-11-28T13:00:41.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/2775564458 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffb540682f0 msgr2=0x7ffb5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7ffb6dfeb700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffb540682f0 0x7ffb5406a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffb680cd980 tx=0x7ffb58008030 comp rx=0 tx=0).stop 2023-11-28T13:00:41.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/2775564458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68202f30 msgr2=0x7ffb68159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7ffb6dfeb700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68202f30 0x7ffb68159360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000cc10 tx=0x7ffb6000cf20 comp rx=0 tx=0).stop 2023-11-28T13:00:41.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2023-11-28T13:00:41.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/2775564458 shutdown_connections 2023-11-28T13:00:41.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7ffb6dfeb700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffb68202f30 0x7ffb68159360 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7ffb6dfeb700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffb680c2b60 0x7ffb680cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7ffb6dfeb700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffb540682f0 0x7ffb5406a7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7ffb6dfeb700 1 --2- 172.21.15.6:0/2775564458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffb680cca20 0x7ffb681ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.677+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/2775564458 >> 172.21.15.6:0/2775564458 conn(0x7ffb680b1650 msgr2=0x7ffb68162360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.687 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:41.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.683+0000 7fa5261fb700 1 -- 172.21.15.6:0/113181079 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa500005870 con 0x7fa520089580 2023-11-28T13:00:41.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.684+0000 7fa507fff700 1 -- 172.21.15.6:0/113181079 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa510006cc0 con 0x7fa520089580 2023-11-28T13:00:41.687 INFO:teuthology.orchestra.run.smithi006.stdout:55834574855 2023-11-28T13:00:41.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2023-11-28T13:00:41.688 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:41.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.687+0000 7fa5261fb700 1 -- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa50c068340 msgr2=0x7fa50c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.687+0000 7fa5261fb700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa50c068340 0x7fa50c06a7f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa518004bd0 tx=0x7fa5180074c0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.687+0000 7fa5261fb700 1 -- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa520089580 msgr2=0x7fa5200c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.687+0000 7fa5261fb700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa520089580 0x7fa5200c7d50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa510016000 tx=0x7fa510004750 comp rx=0 tx=0).stop 2023-11-28T13:00:41.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.686+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/2775564458 shutdown_connections 2023-11-28T13:00:41.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.686+0000 7ffb6dfeb700 1 -- 172.21.15.6:0/2775564458 wait complete. 2023-11-28T13:00:41.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.691+0000 7fa5261fb700 1 -- 172.21.15.6:0/113181079 shutdown_connections 2023-11-28T13:00:41.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.691+0000 7fa5261fb700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa520089580 0x7fa5200c7d50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.691+0000 7fa5261fb700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa52006a1e0 0x7fa520208aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.691+0000 7fa5261fb700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa50c068340 0x7fa50c06a7f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.691+0000 7fa5261fb700 1 --2- 172.21.15.6:0/113181079 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa52006b5c0 0x7fa52020a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.691+0000 7fa5261fb700 1 -- 172.21.15.6:0/113181079 >> 172.21.15.6:0/113181079 conn(0x7fa5200cf670 msgr2=0x7fa52006d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2023-11-28T13:00:41.693 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:41.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-11-28T13:00:41.694 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:41.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.695+0000 7fa5261fb700 1 -- 172.21.15.6:0/113181079 shutdown_connections 2023-11-28T13:00:41.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.695+0000 7fa5261fb700 1 -- 172.21.15.6:0/113181079 wait complete. 2023-11-28T13:00:41.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2023-11-28T13:00:41.697 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:41.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.701+0000 7f8841431700 1 -- 172.21.15.6:0/2594633554 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f881c0055a0 con 0x7f883c0cc850 2023-11-28T13:00:41.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.701+0000 7f88237fe700 1 -- 172.21.15.6:0/2594633554 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f882c017090 con 0x7f883c0cc850 2023-11-28T13:00:41.702 INFO:teuthology.orchestra.run.smithi006.stdout:55834574858 2023-11-28T13:00:41.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2023-11-28T13:00:41.705 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:41.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 -- 172.21.15.6:0/2594633554 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88280683c0 msgr2=0x7f882806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88280683c0 0x7f882806a870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f883c14bfb0 tx=0x7f8830009210 comp rx=0 tx=0).stop 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 -- 172.21.15.6:0/2594633554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 msgr2=0x7f883c06cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 0x7f883c06cf20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f882c00c450 tx=0x7f882c00c810 comp rx=0 tx=0).stop 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 -- 172.21.15.6:0/2594633554 shutdown_connections 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f883c160c90 0x7f883c14aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f883c0cc850 0x7f883c06cf20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88280683c0 0x7f882806a870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 --2- 172.21.15.6:0/2594633554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f883c0ce320 0x7f883c14c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 -- 172.21.15.6:0/2594633554 >> 172.21.15.6:0/2594633554 conn(0x7f883c0b1650 msgr2=0x7f883c152c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 -- 172.21.15.6:0/2594633554 shutdown_connections 2023-11-28T13:00:41.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.705+0000 7f8841431700 1 -- 172.21.15.6:0/2594633554 wait complete. 2023-11-28T13:00:41.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-11-28T13:00:41.715 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:41.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2023-11-28T13:00:41.721 DEBUG:teuthology.parallel:result is None 2023-11-28T13:00:41.721 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T13:00:41.721 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:41.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.885+0000 7fb9c7a3f700 1 Processor -- start 2023-11-28T13:00:41.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.885+0000 7fb9c7a3f700 1 -- start start 2023-11-28T13:00:41.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.885+0000 7fb9c7a3f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb9c0161090 0x7fb9c00ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9c7a3f700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c00ce340 con 0x7fb9c015c990 2023-11-28T13:00:41.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9c7a3f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c00cebe0 con 0x7fb9c00caeb0 2023-11-28T13:00:41.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9c7a3f700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c00cf780 con 0x7fb9c0161090 2023-11-28T13:00:41.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9c4fda700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb9c0161090 0x7fb9c00ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9c4fda700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb9c0161090 0x7fb9c00ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58354/0 (socket says 172.21.15.6:58354) 2023-11-28T13:00:41.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9c4fda700 1 -- 172.21.15.6:0/551610222 learned_addr learned my addr 172.21.15.6:0/551610222 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9c4fda700 1 -- 172.21.15.6:0/551610222 >> v1:172.21.15.61:6789/0 conn(0x7fb9c00caeb0 legacy=0x7fb9c00cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9c4fda700 1 -- 172.21.15.6:0/551610222 >> v1:172.21.15.6:6789/0 conn(0x7fb9c015c990 legacy=0x7fb9c00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9b7fff700 1 -- 172.21.15.6:0/551610222 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009118825 0 0) 0x7fb9c00ce340 con 0x7fb9c015c990 2023-11-28T13:00:41.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9c4fda700 1 -- 172.21.15.6:0/551610222 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c00cb3f0 con 0x7fb9c0161090 2023-11-28T13:00:41.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9c4fda700 1 --2- 172.21.15.6:0/551610222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c0161090 0x7fb9c00ca970 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0098f0 tx=0x7fb9bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e686019737d3c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9b7fff700 1 -- 172.21.15.6:0/551610222 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc00b8a0 con 0x7fb9c0161090 2023-11-28T13:00:41.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9b7fff700 1 -- 172.21.15.6:0/551610222 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc00ba00 con 0x7fb9c0161090 2023-11-28T13:00:41.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.886+0000 7fb9b7fff700 1 -- 172.21.15.6:0/551610222 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc00bd90 con 0x7fb9c0161090 2023-11-28T13:00:41.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.887+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/551610222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c0161090 msgr2=0x7fb9c00ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.887+0000 7fb9c7a3f700 1 --2- 172.21.15.6:0/551610222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c0161090 0x7fb9c00ca970 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0098f0 tx=0x7fb9bc009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:41.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.887+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/551610222 shutdown_connections 2023-11-28T13:00:41.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.887+0000 7fb9c7a3f700 1 --2- 172.21.15.6:0/551610222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c0161090 0x7fb9c00ca970 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.887+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/551610222 >> 172.21.15.6:0/551610222 conn(0x7fb9c00b1650 msgr2=0x7fb9c00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:41.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.887+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/551610222 shutdown_connections 2023-11-28T13:00:41.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.887+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/551610222 wait complete. 2023-11-28T13:00:41.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.888+0000 7fb9c7a3f700 1 Processor -- start 2023-11-28T13:00:41.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.888+0000 7fb9c7a3f700 1 -- start start 2023-11-28T13:00:41.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c7a3f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9c00caeb0 0x7fb9c015fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c7a3f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c015c990 0x7fb9c00b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c7a3f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9c0161090 0x7fb9c00cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c7a3f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0160120 con 0x7fb9c015c990 2023-11-28T13:00:41.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c7a3f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00b32f0 con 0x7fb9c0161090 2023-11-28T13:00:41.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c7a3f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00b35d0 con 0x7fb9c00caeb0 2023-11-28T13:00:41.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c4fda700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c015c990 0x7fb9c00b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c4fda700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c015c990 0x7fb9c00b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58362/0 (socket says 172.21.15.6:58362) 2023-11-28T13:00:41.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c4fda700 1 -- 172.21.15.6:0/1947015945 learned_addr learned my addr 172.21.15.6:0/1947015945 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:41.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c57db700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9c00caeb0 0x7fb9c015fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.889+0000 7fb9c5fdc700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9c0161090 0x7fb9c00cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9c4fda700 1 -- 172.21.15.6:0/1947015945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9c00caeb0 msgr2=0x7fb9c015fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9c4fda700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9c00caeb0 0x7fb9c015fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9c4fda700 1 -- 172.21.15.6:0/1947015945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9c0161090 msgr2=0x7fb9c00cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9c4fda700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9c0161090 0x7fb9c00cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9c4fda700 1 -- 172.21.15.6:0/1947015945 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc009550 con 0x7fb9c015c990 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9c57db700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9c00caeb0 0x7fb9c015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9c5fdc700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9c0161090 0x7fb9c00cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9c4fda700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c015c990 0x7fb9c00b2950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc014040 tx=0x7fb9bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9b5ffb700 1 -- 172.21.15.6:0/1947015945 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc011a70 con 0x7fb9c015c990 2023-11-28T13:00:41.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9b5ffb700 1 -- 172.21.15.6:0/1947015945 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc01f070 con 0x7fb9c015c990 2023-11-28T13:00:41.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/1947015945 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c00ce870 con 0x7fb9c015c990 2023-11-28T13:00:41.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9b5ffb700 1 -- 172.21.15.6:0/1947015945 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc011dd0 con 0x7fb9c015c990 2023-11-28T13:00:41.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.890+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/1947015945 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c00ce590 con 0x7fb9c015c990 2023-11-28T13:00:41.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.891+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/1947015945 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c00cca20 con 0x7fb9c015c990 2023-11-28T13:00:41.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.892+0000 7fb9b5ffb700 1 -- 172.21.15.6:0/1947015945 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb9bc00f050 con 0x7fb9c015c990 2023-11-28T13:00:41.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.892+0000 7fb9b5ffb700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a40682f0 0x7fb9a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:41.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.892+0000 7fb9b5ffb700 1 -- 172.21.15.6:0/1947015945 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb9bc021070 con 0x7fb9c015c990 2023-11-28T13:00:41.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.892+0000 7fb9c57db700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a40682f0 0x7fb9a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:41.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.893+0000 7fb9c57db700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a40682f0 0x7fb9a406a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0007fd0 tx=0x7fb9b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:41.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:41.896+0000 7fb9b5ffb700 1 -- 172.21.15.6:0/1947015945 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb9bc054340 con 0x7fb9c015c990 2023-11-28T13:00:42.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.086+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/1947015945 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9c00675a0 con 0x7fb9a40682f0 2023-11-28T13:00:42.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.095+0000 7fb9b5ffb700 1 -- 172.21.15.6:0/1947015945 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367535 (secure 0 0 0) 0x7fb9c00675a0 con 0x7fb9a40682f0 2023-11-28T13:00:42.101 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:42.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/1947015945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a40682f0 msgr2=0x7fb9a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a40682f0 0x7fb9a406a7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0007fd0 tx=0x7fb9b0009040 comp rx=0 tx=0).stop 2023-11-28T13:00:42.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/1947015945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c015c990 msgr2=0x7fb9c00b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c015c990 0x7fb9c00b2950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc014040 tx=0x7fb9bc004750 comp rx=0 tx=0).stop 2023-11-28T13:00:42.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/1947015945 shutdown_connections 2023-11-28T13:00:42.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9c0161090 0x7fb9c00cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a40682f0 0x7fb9a406a7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9c00caeb0 0x7fb9c015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 --2- 172.21.15.6:0/1947015945 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9c015c990 0x7fb9c00b2950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/1947015945 >> 172.21.15.6:0/1947015945 conn(0x7fb9c00b1650 msgr2=0x7fb9c00afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/1947015945 shutdown_connections 2023-11-28T13:00:42.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.099+0000 7fb9c7a3f700 1 -- 172.21.15.6:0/1947015945 wait complete. 2023-11-28T13:00:42.105 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:00:42.112 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-11-28T13:00:40.946475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594896,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67232,"kb_used_data":3296,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907488,"statfs":{"total":773094113280,"available":773025267712,"internally_reserved":0,"allocated":3375104,"data_stored":2065897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2198,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002515"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.456044+0000","last_change":"2023-11-28T13:00:21.430642+0000","last_active":"2023-11-28T13:00:25.456044+0000","last_peered":"2023-11-28T13:00:25.456044+0000","last_clean":"2023-11-28T13:00:25.456044+0000","last_became_active":"2023-11-28T13:00:21.430357+0000","last_became_peered":"2023-11-28T13:00:21.430357+0000","last_unstale":"2023-11-28T13:00:25.456044+0000","last_undegraded":"2023-11-28T13:00:25.456044+0000","last_fullsized":"2023-11-28T13:00:25.456044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:43.433683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452930+0000","last_change":"2023-11-28T13:00:21.432299+0000","last_active":"2023-11-28T13:00:25.452930+0000","last_peered":"2023-11-28T13:00:25.452930+0000","last_clean":"2023-11-28T13:00:25.452930+0000","last_became_active":"2023-11-28T13:00:21.431566+0000","last_became_peered":"2023-11-28T13:00:21.431566+0000","last_unstale":"2023-11-28T13:00:25.452930+0000","last_undegraded":"2023-11-28T13:00:25.452930+0000","last_fullsized":"2023-11-28T13:00:25.452930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:24.765270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.461121+0000","last_change":"2023-11-28T13:00:21.430850+0000","last_active":"2023-11-28T13:00:25.461121+0000","last_peered":"2023-11-28T13:00:25.461121+0000","last_clean":"2023-11-28T13:00:25.461121+0000","last_became_active":"2023-11-28T13:00:21.430557+0000","last_became_peered":"2023-11-28T13:00:21.430557+0000","last_unstale":"2023-11-28T13:00:25.461121+0000","last_undegraded":"2023-11-28T13:00:25.461121+0000","last_fullsized":"2023-11-28T13:00:25.461121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:00.080768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442074+0000","last_change":"2023-11-28T13:00:21.437362+0000","last_active":"2023-11-28T13:00:25.442074+0000","last_peered":"2023-11-28T13:00:25.442074+0000","last_clean":"2023-11-28T13:00:25.442074+0000","last_became_active":"2023-11-28T13:00:21.435046+0000","last_became_peered":"2023-11-28T13:00:21.435046+0000","last_unstale":"2023-11-28T13:00:25.442074+0000","last_undegraded":"2023-11-28T13:00:25.442074+0000","last_fullsized":"2023-11-28T13:00:25.442074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:39.731416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448772+0000","last_change":"2023-11-28T13:00:21.436404+0000","last_active":"2023-11-28T13:00:25.448772+0000","last_peered":"2023-11-28T13:00:25.448772+0000","last_clean":"2023-11-28T13:00:25.448772+0000","last_became_active":"2023-11-28T13:00:21.436158+0000","last_became_peered":"2023-11-28T13:00:21.436158+0000","last_unstale":"2023-11-28T13:00:25.448772+0000","last_undegraded":"2023-11-28T13:00:25.448772+0000","last_fullsized":"2023-11-28T13:00:25.448772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:42.706331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460337+0000","last_change":"2023-11-28T13:00:21.431231+0000","last_active":"2023-11-28T13:00:25.460337+0000","last_peered":"2023-11-28T13:00:25.460337+0000","last_clean":"2023-11-28T13:00:25.460337+0000","last_became_active":"2023-11-28T13:00:21.430602+0000","last_became_peered":"2023-11-28T13:00:21.430602+0000","last_unstale":"2023-11-28T13:00:25.460337+0000","last_undegraded":"2023-11-28T13:00:25.460337+0000","last_fullsized":"2023-11-28T13:00:25.460337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:13.072237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443249+0000","last_change":"2023-11-28T13:00:21.419513+0000","last_active":"2023-11-28T13:00:25.443249+0000","last_peered":"2023-11-28T13:00:25.443249+0000","last_clean":"2023-11-28T13:00:25.443249+0000","last_became_active":"2023-11-28T13:00:21.418668+0000","last_became_peered":"2023-11-28T13:00:21.418668+0000","last_unstale":"2023-11-28T13:00:25.443249+0000","last_undegraded":"2023-11-28T13:00:25.443249+0000","last_fullsized":"2023-11-28T13:00:25.443249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448590+0000","last_change":"2023-11-28T13:00:21.432362+0000","last_active":"2023-11-28T13:00:25.448590+0000","last_peered":"2023-11-28T13:00:25.448590+0000","last_clean":"2023-11-28T13:00:25.448590+0000","last_became_active":"2023-11-28T13:00:21.432056+0000","last_became_peered":"2023-11-28T13:00:21.432056+0000","last_unstale":"2023-11-28T13:00:25.448590+0000","last_undegraded":"2023-11-28T13:00:25.448590+0000","last_fullsized":"2023-11-28T13:00:25.448590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:13.039552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448588+0000","last_change":"2023-11-28T13:00:21.434750+0000","last_active":"2023-11-28T13:00:25.448588+0000","last_peered":"2023-11-28T13:00:25.448588+0000","last_clean":"2023-11-28T13:00:25.448588+0000","last_became_active":"2023-11-28T13:00:21.434060+0000","last_became_peered":"2023-11-28T13:00:21.434060+0000","last_unstale":"2023-11-28T13:00:25.448588+0000","last_undegraded":"2023-11-28T13:00:25.448588+0000","last_fullsized":"2023-11-28T13:00:25.448588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:34.468496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.453105+0000","last_change":"2023-11-28T13:00:21.433386+0000","last_active":"2023-11-28T13:00:25.453105+0000","last_peered":"2023-11-28T13:00:25.453105+0000","last_clean":"2023-11-28T13:00:25.453105+0000","last_became_active":"2023-11-28T13:00:21.432005+0000","last_became_peered":"2023-11-28T13:00:21.432005+0000","last_unstale":"2023-11-28T13:00:25.453105+0000","last_undegraded":"2023-11-28T13:00:25.453105+0000","last_fullsized":"2023-11-28T13:00:25.453105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:11.589800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.461025+0000","last_change":"2023-11-28T13:00:21.427275+0000","last_active":"2023-11-28T13:00:25.461025+0000","last_peered":"2023-11-28T13:00:25.461025+0000","last_clean":"2023-11-28T13:00:25.461025+0000","last_became_active":"2023-11-28T13:00:21.426893+0000","last_became_peered":"2023-11-28T13:00:21.426893+0000","last_unstale":"2023-11-28T13:00:25.461025+0000","last_undegraded":"2023-11-28T13:00:25.461025+0000","last_fullsized":"2023-11-28T13:00:25.461025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:24.765270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455341+0000","last_change":"2023-11-28T13:00:21.436650+0000","last_active":"2023-11-28T13:00:25.455341+0000","last_peered":"2023-11-28T13:00:25.455341+0000","last_clean":"2023-11-28T13:00:25.455341+0000","last_became_active":"2023-11-28T13:00:21.435700+0000","last_became_peered":"2023-11-28T13:00:21.435700+0000","last_unstale":"2023-11-28T13:00:25.455341+0000","last_undegraded":"2023-11-28T13:00:25.455341+0000","last_fullsized":"2023-11-28T13:00:25.455341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:02.740267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.454713+0000","last_change":"2023-11-28T13:00:21.434352+0000","last_active":"2023-11-28T13:00:25.454713+0000","last_peered":"2023-11-28T13:00:25.454713+0000","last_clean":"2023-11-28T13:00:25.454713+0000","last_became_active":"2023-11-28T13:00:21.434067+0000","last_became_peered":"2023-11-28T13:00:21.434067+0000","last_unstale":"2023-11-28T13:00:25.454713+0000","last_undegraded":"2023-11-28T13:00:25.454713+0000","last_fullsized":"2023-11-28T13:00:25.454713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:30.629627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449953+0000","last_change":"2023-11-28T13:00:21.431774+0000","last_active":"2023-11-28T13:00:25.449953+0000","last_peered":"2023-11-28T13:00:25.449953+0000","last_clean":"2023-11-28T13:00:25.449953+0000","last_became_active":"2023-11-28T13:00:21.431445+0000","last_became_peered":"2023-11-28T13:00:21.431445+0000","last_unstale":"2023-11-28T13:00:25.449953+0000","last_undegraded":"2023-11-28T13:00:25.449953+0000","last_fullsized":"2023-11-28T13:00:25.449953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:24.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443018+0000","last_change":"2023-11-28T13:00:21.420404+0000","last_active":"2023-11-28T13:00:25.443018+0000","last_peered":"2023-11-28T13:00:25.443018+0000","last_clean":"2023-11-28T13:00:25.443018+0000","last_became_active":"2023-11-28T13:00:21.419228+0000","last_became_peered":"2023-11-28T13:00:21.419228+0000","last_unstale":"2023-11-28T13:00:25.443018+0000","last_undegraded":"2023-11-28T13:00:25.443018+0000","last_fullsized":"2023-11-28T13:00:25.443018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:40:24.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458890+0000","last_change":"2023-11-28T13:00:21.430943+0000","last_active":"2023-11-28T13:00:25.458890+0000","last_peered":"2023-11-28T13:00:25.458890+0000","last_clean":"2023-11-28T13:00:25.458890+0000","last_became_active":"2023-11-28T13:00:21.430664+0000","last_became_peered":"2023-11-28T13:00:21.430664+0000","last_unstale":"2023-11-28T13:00:25.458890+0000","last_undegraded":"2023-11-28T13:00:25.458890+0000","last_fullsized":"2023-11-28T13:00:25.458890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:12:14.186445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448670+0000","last_change":"2023-11-28T13:00:21.425335+0000","last_active":"2023-11-28T13:00:25.448670+0000","last_peered":"2023-11-28T13:00:25.448670+0000","last_clean":"2023-11-28T13:00:25.448670+0000","last_became_active":"2023-11-28T13:00:21.425073+0000","last_became_peered":"2023-11-28T13:00:21.425073+0000","last_unstale":"2023-11-28T13:00:25.448670+0000","last_undegraded":"2023-11-28T13:00:25.448670+0000","last_fullsized":"2023-11-28T13:00:25.448670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:45.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460831+0000","last_change":"2023-11-28T13:00:21.426161+0000","last_active":"2023-11-28T13:00:25.460831+0000","last_peered":"2023-11-28T13:00:25.460831+0000","last_clean":"2023-11-28T13:00:25.460831+0000","last_became_active":"2023-11-28T13:00:21.425796+0000","last_became_peered":"2023-11-28T13:00:21.425796+0000","last_unstale":"2023-11-28T13:00:25.460831+0000","last_undegraded":"2023-11-28T13:00:25.460831+0000","last_fullsized":"2023-11-28T13:00:25.460831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:17.422401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455990+0000","last_change":"2023-11-28T13:00:21.423565+0000","last_active":"2023-11-28T13:00:25.455990+0000","last_peered":"2023-11-28T13:00:25.455990+0000","last_clean":"2023-11-28T13:00:25.455990+0000","last_became_active":"2023-11-28T13:00:21.423270+0000","last_became_peered":"2023-11-28T13:00:21.423270+0000","last_unstale":"2023-11-28T13:00:25.455990+0000","last_undegraded":"2023-11-28T13:00:25.455990+0000","last_fullsized":"2023-11-28T13:00:25.455990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:31.139532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460193+0000","last_change":"2023-11-28T13:00:21.426452+0000","last_active":"2023-11-28T13:00:25.460193+0000","last_peered":"2023-11-28T13:00:25.460193+0000","last_clean":"2023-11-28T13:00:25.460193+0000","last_became_active":"2023-11-28T13:00:21.426096+0000","last_became_peered":"2023-11-28T13:00:21.426096+0000","last_unstale":"2023-11-28T13:00:25.460193+0000","last_undegraded":"2023-11-28T13:00:25.460193+0000","last_fullsized":"2023-11-28T13:00:25.460193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:33.439707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443668+0000","last_change":"2023-11-28T13:00:21.435483+0000","last_active":"2023-11-28T13:00:25.443668+0000","last_peered":"2023-11-28T13:00:25.443668+0000","last_clean":"2023-11-28T13:00:25.443668+0000","last_became_active":"2023-11-28T13:00:21.435162+0000","last_became_peered":"2023-11-28T13:00:21.435162+0000","last_unstale":"2023-11-28T13:00:25.443668+0000","last_undegraded":"2023-11-28T13:00:25.443668+0000","last_fullsized":"2023-11-28T13:00:25.443668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:13.039552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448596+0000","last_change":"2023-11-28T13:00:21.417958+0000","last_active":"2023-11-28T13:00:25.448596+0000","last_peered":"2023-11-28T13:00:25.448596+0000","last_clean":"2023-11-28T13:00:25.448596+0000","last_became_active":"2023-11-28T13:00:21.417664+0000","last_became_peered":"2023-11-28T13:00:21.417664+0000","last_unstale":"2023-11-28T13:00:25.448596+0000","last_undegraded":"2023-11-28T13:00:25.448596+0000","last_fullsized":"2023-11-28T13:00:25.448596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:35.062024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441395+0000","last_change":"2023-11-28T13:00:21.432315+0000","last_active":"2023-11-28T13:00:25.441395+0000","last_peered":"2023-11-28T13:00:25.441395+0000","last_clean":"2023-11-28T13:00:25.441395+0000","last_became_active":"2023-11-28T13:00:21.431878+0000","last_became_peered":"2023-11-28T13:00:21.431878+0000","last_unstale":"2023-11-28T13:00:25.441395+0000","last_undegraded":"2023-11-28T13:00:25.441395+0000","last_fullsized":"2023-11-28T13:00:25.441395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:26:43.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446220+0000","last_change":"2023-11-28T13:00:21.429436+0000","last_active":"2023-11-28T13:00:25.446220+0000","last_peered":"2023-11-28T13:00:25.446220+0000","last_clean":"2023-11-28T13:00:25.446220+0000","last_became_active":"2023-11-28T13:00:21.429051+0000","last_became_peered":"2023-11-28T13:00:21.429051+0000","last_unstale":"2023-11-28T13:00:25.446220+0000","last_undegraded":"2023-11-28T13:00:25.446220+0000","last_fullsized":"2023-11-28T13:00:25.446220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:11.589800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458565+0000","last_change":"2023-11-28T13:00:21.423512+0000","last_active":"2023-11-28T13:00:25.458565+0000","last_peered":"2023-11-28T13:00:25.458565+0000","last_clean":"2023-11-28T13:00:25.458565+0000","last_became_active":"2023-11-28T13:00:21.423223+0000","last_became_peered":"2023-11-28T13:00:21.423223+0000","last_unstale":"2023-11-28T13:00:25.458565+0000","last_undegraded":"2023-11-28T13:00:25.458565+0000","last_fullsized":"2023-11-28T13:00:25.458565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:35.062024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457987+0000","last_change":"2023-11-28T13:00:21.426702+0000","last_active":"2023-11-28T13:00:25.457987+0000","last_peered":"2023-11-28T13:00:25.457987+0000","last_clean":"2023-11-28T13:00:25.457987+0000","last_became_active":"2023-11-28T13:00:21.425807+0000","last_became_peered":"2023-11-28T13:00:21.425807+0000","last_unstale":"2023-11-28T13:00:25.457987+0000","last_undegraded":"2023-11-28T13:00:25.457987+0000","last_fullsized":"2023-11-28T13:00:25.457987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:04.077060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452451+0000","last_change":"2023-11-28T13:00:21.434848+0000","last_active":"2023-11-28T13:00:25.452451+0000","last_peered":"2023-11-28T13:00:25.452451+0000","last_clean":"2023-11-28T13:00:25.452451+0000","last_became_active":"2023-11-28T13:00:21.432193+0000","last_became_peered":"2023-11-28T13:00:21.432193+0000","last_unstale":"2023-11-28T13:00:25.452451+0000","last_undegraded":"2023-11-28T13:00:25.452451+0000","last_fullsized":"2023-11-28T13:00:25.452451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:43.321870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871441+0000","last_change":"2023-11-28T13:00:25.470736+0000","last_active":"2023-11-28T13:00:25.871441+0000","last_peered":"2023-11-28T13:00:25.871441+0000","last_clean":"2023-11-28T13:00:25.871441+0000","last_became_active":"2023-11-28T13:00:25.470009+0000","last_became_peered":"2023-11-28T13:00:25.470009+0000","last_unstale":"2023-11-28T13:00:25.871441+0000","last_undegraded":"2023-11-28T13:00:25.871441+0000","last_fullsized":"2023-11-28T13:00:25.871441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:05.706754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459220+0000","last_change":"2023-11-28T13:00:21.006957+0000","last_active":"2023-11-28T13:00:25.459220+0000","last_peered":"2023-11-28T13:00:25.459220+0000","last_clean":"2023-11-28T13:00:25.459220+0000","last_became_active":"2023-11-28T13:00:21.006690+0000","last_became_peered":"2023-11-28T13:00:21.006690+0000","last_unstale":"2023-11-28T13:00:25.459220+0000","last_undegraded":"2023-11-28T13:00:25.459220+0000","last_fullsized":"2023-11-28T13:00:25.459220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:30.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441650+0000","last_change":"2023-11-28T13:00:21.430560+0000","last_active":"2023-11-28T13:00:25.441650+0000","last_peered":"2023-11-28T13:00:25.441650+0000","last_clean":"2023-11-28T13:00:25.441650+0000","last_became_active":"2023-11-28T13:00:21.430312+0000","last_became_peered":"2023-11-28T13:00:21.430312+0000","last_unstale":"2023-11-28T13:00:25.441650+0000","last_undegraded":"2023-11-28T13:00:25.441650+0000","last_fullsized":"2023-11-28T13:00:25.441650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347196+0000","last_change":"2023-11-28T13:00:25.460012+0000","last_active":"2023-11-28T13:00:26.347196+0000","last_peered":"2023-11-28T13:00:26.347196+0000","last_clean":"2023-11-28T13:00:26.347196+0000","last_became_active":"2023-11-28T13:00:25.459532+0000","last_became_peered":"2023-11-28T13:00:25.459532+0000","last_unstale":"2023-11-28T13:00:26.347196+0000","last_undegraded":"2023-11-28T13:00:26.347196+0000","last_fullsized":"2023-11-28T13:00:26.347196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:36.890229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448628+0000","last_change":"2023-11-28T13:00:21.005058+0000","last_active":"2023-11-28T13:00:25.448628+0000","last_peered":"2023-11-28T13:00:25.448628+0000","last_clean":"2023-11-28T13:00:25.448628+0000","last_became_active":"2023-11-28T13:00:21.004749+0000","last_became_peered":"2023-11-28T13:00:21.004749+0000","last_unstale":"2023-11-28T13:00:25.448628+0000","last_undegraded":"2023-11-28T13:00:25.448628+0000","last_fullsized":"2023-11-28T13:00:25.448628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:22.962275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448569+0000","last_change":"2023-11-28T13:00:21.416645+0000","last_active":"2023-11-28T13:00:25.448569+0000","last_peered":"2023-11-28T13:00:25.448569+0000","last_clean":"2023-11-28T13:00:25.448569+0000","last_became_active":"2023-11-28T13:00:21.416365+0000","last_became_peered":"2023-11-28T13:00:21.416365+0000","last_unstale":"2023-11-28T13:00:25.448569+0000","last_undegraded":"2023-11-28T13:00:25.448569+0000","last_fullsized":"2023-11-28T13:00:25.448569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:14.429821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073812+0000","last_change":"2023-11-28T13:00:25.468574+0000","last_active":"2023-11-28T13:00:26.073812+0000","last_peered":"2023-11-28T13:00:26.073812+0000","last_clean":"2023-11-28T13:00:26.073812+0000","last_became_active":"2023-11-28T13:00:25.468177+0000","last_became_peered":"2023-11-28T13:00:25.468177+0000","last_unstale":"2023-11-28T13:00:26.073812+0000","last_undegraded":"2023-11-28T13:00:26.073812+0000","last_fullsized":"2023-11-28T13:00:26.073812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:13:55.892226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441558+0000","last_change":"2023-11-28T13:00:21.005438+0000","last_active":"2023-11-28T13:00:25.441558+0000","last_peered":"2023-11-28T13:00:25.441558+0000","last_clean":"2023-11-28T13:00:25.441558+0000","last_became_active":"2023-11-28T13:00:21.004885+0000","last_became_peered":"2023-11-28T13:00:21.004885+0000","last_unstale":"2023-11-28T13:00:25.441558+0000","last_undegraded":"2023-11-28T13:00:25.441558+0000","last_fullsized":"2023-11-28T13:00:25.441558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:47.845351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442760+0000","last_change":"2023-11-28T13:00:21.437422+0000","last_active":"2023-11-28T13:00:25.442760+0000","last_peered":"2023-11-28T13:00:25.442760+0000","last_clean":"2023-11-28T13:00:25.442760+0000","last_became_active":"2023-11-28T13:00:21.437095+0000","last_became_peered":"2023-11-28T13:00:21.437095+0000","last_unstale":"2023-11-28T13:00:25.442760+0000","last_undegraded":"2023-11-28T13:00:25.442760+0000","last_fullsized":"2023-11-28T13:00:25.442760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:52.972220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346252+0000","last_change":"2023-11-28T13:00:25.468548+0000","last_active":"2023-11-28T13:00:26.346252+0000","last_peered":"2023-11-28T13:00:26.346252+0000","last_clean":"2023-11-28T13:00:26.346252+0000","last_became_active":"2023-11-28T13:00:25.468071+0000","last_became_peered":"2023-11-28T13:00:25.468071+0000","last_unstale":"2023-11-28T13:00:26.346252+0000","last_undegraded":"2023-11-28T13:00:26.346252+0000","last_fullsized":"2023-11-28T13:00:26.346252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:20.872923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448774+0000","last_change":"2023-11-28T13:00:21.003619+0000","last_active":"2023-11-28T13:00:25.448774+0000","last_peered":"2023-11-28T13:00:25.448774+0000","last_clean":"2023-11-28T13:00:25.448774+0000","last_became_active":"2023-11-28T13:00:21.003189+0000","last_became_peered":"2023-11-28T13:00:21.003189+0000","last_unstale":"2023-11-28T13:00:25.448774+0000","last_undegraded":"2023-11-28T13:00:25.448774+0000","last_fullsized":"2023-11-28T13:00:25.448774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:53.009716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441443+0000","last_change":"2023-11-28T13:00:21.428000+0000","last_active":"2023-11-28T13:00:25.441443+0000","last_peered":"2023-11-28T13:00:25.441443+0000","last_clean":"2023-11-28T13:00:25.441443+0000","last_became_active":"2023-11-28T13:00:21.427738+0000","last_became_peered":"2023-11-28T13:00:21.427738+0000","last_unstale":"2023-11-28T13:00:25.441443+0000","last_undegraded":"2023-11-28T13:00:25.441443+0000","last_fullsized":"2023-11-28T13:00:25.441443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:24.529486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871318+0000","last_change":"2023-11-28T13:00:25.463375+0000","last_active":"2023-11-28T13:00:25.871318+0000","last_peered":"2023-11-28T13:00:25.871318+0000","last_clean":"2023-11-28T13:00:25.871318+0000","last_became_active":"2023-11-28T13:00:25.462962+0000","last_became_peered":"2023-11-28T13:00:25.462962+0000","last_unstale":"2023-11-28T13:00:25.871318+0000","last_undegraded":"2023-11-28T13:00:25.871318+0000","last_fullsized":"2023-11-28T13:00:25.871318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:16.522760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452987+0000","last_change":"2023-11-28T13:00:21.011149+0000","last_active":"2023-11-28T13:00:25.452987+0000","last_peered":"2023-11-28T13:00:25.452987+0000","last_clean":"2023-11-28T13:00:25.452987+0000","last_became_active":"2023-11-28T13:00:21.010813+0000","last_became_peered":"2023-11-28T13:00:21.010813+0000","last_unstale":"2023-11-28T13:00:25.452987+0000","last_undegraded":"2023-11-28T13:00:25.452987+0000","last_fullsized":"2023-11-28T13:00:25.452987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441165+0000","last_change":"2023-11-28T13:00:21.428947+0000","last_active":"2023-11-28T13:00:25.441165+0000","last_peered":"2023-11-28T13:00:25.441165+0000","last_clean":"2023-11-28T13:00:25.441165+0000","last_became_active":"2023-11-28T13:00:21.428690+0000","last_became_peered":"2023-11-28T13:00:21.428690+0000","last_unstale":"2023-11-28T13:00:25.441165+0000","last_undegraded":"2023-11-28T13:00:25.441165+0000","last_fullsized":"2023-11-28T13:00:25.441165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:55:41.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869911+0000","last_change":"2023-11-28T13:00:25.468634+0000","last_active":"2023-11-28T13:00:25.869911+0000","last_peered":"2023-11-28T13:00:25.869911+0000","last_clean":"2023-11-28T13:00:25.869911+0000","last_became_active":"2023-11-28T13:00:25.468224+0000","last_became_peered":"2023-11-28T13:00:25.468224+0000","last_unstale":"2023-11-28T13:00:25.869911+0000","last_undegraded":"2023-11-28T13:00:25.869911+0000","last_fullsized":"2023-11-28T13:00:25.869911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:21.591804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446244+0000","last_change":"2023-11-28T13:00:21.006053+0000","last_active":"2023-11-28T13:00:25.446244+0000","last_peered":"2023-11-28T13:00:25.446244+0000","last_clean":"2023-11-28T13:00:25.446244+0000","last_became_active":"2023-11-28T13:00:21.005693+0000","last_became_peered":"2023-11-28T13:00:21.005693+0000","last_unstale":"2023-11-28T13:00:25.446244+0000","last_undegraded":"2023-11-28T13:00:25.446244+0000","last_fullsized":"2023-11-28T13:00:25.446244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:38.720815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457494+0000","last_change":"2023-11-28T13:00:21.426158+0000","last_active":"2023-11-28T13:00:25.457494+0000","last_peered":"2023-11-28T13:00:25.457494+0000","last_clean":"2023-11-28T13:00:25.457494+0000","last_became_active":"2023-11-28T13:00:21.425771+0000","last_became_peered":"2023-11-28T13:00:21.425771+0000","last_unstale":"2023-11-28T13:00:25.457494+0000","last_undegraded":"2023-11-28T13:00:25.457494+0000","last_fullsized":"2023-11-28T13:00:25.457494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:48.996841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076004+0000","last_change":"2023-11-28T13:00:25.467575+0000","last_active":"2023-11-28T13:00:26.076004+0000","last_peered":"2023-11-28T13:00:26.076004+0000","last_clean":"2023-11-28T13:00:26.076004+0000","last_became_active":"2023-11-28T13:00:25.466169+0000","last_became_peered":"2023-11-28T13:00:25.466169+0000","last_unstale":"2023-11-28T13:00:26.076004+0000","last_undegraded":"2023-11-28T13:00:26.076004+0000","last_fullsized":"2023-11-28T13:00:26.076004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:57.102759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:30.521611+0000","last_change":"2023-11-28T13:00:21.000813+0000","last_active":"2023-11-28T13:00:30.521611+0000","last_peered":"2023-11-28T13:00:30.521611+0000","last_clean":"2023-11-28T13:00:30.521611+0000","last_became_active":"2023-11-28T13:00:20.999079+0000","last_became_peered":"2023-11-28T13:00:20.999079+0000","last_unstale":"2023-11-28T13:00:30.521611+0000","last_undegraded":"2023-11-28T13:00:30.521611+0000","last_fullsized":"2023-11-28T13:00:30.521611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:04.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460800+0000","last_change":"2023-11-28T13:00:21.422098+0000","last_active":"2023-11-28T13:00:25.460800+0000","last_peered":"2023-11-28T13:00:25.460800+0000","last_clean":"2023-11-28T13:00:25.460800+0000","last_became_active":"2023-11-28T13:00:21.421774+0000","last_became_peered":"2023-11-28T13:00:21.421774+0000","last_unstale":"2023-11-28T13:00:25.460800+0000","last_undegraded":"2023-11-28T13:00:25.460800+0000","last_fullsized":"2023-11-28T13:00:25.460800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:08.151957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347474+0000","last_change":"2023-11-28T13:00:25.468792+0000","last_active":"2023-11-28T13:00:26.347474+0000","last_peered":"2023-11-28T13:00:26.347474+0000","last_clean":"2023-11-28T13:00:26.347474+0000","last_became_active":"2023-11-28T13:00:25.468376+0000","last_became_peered":"2023-11-28T13:00:25.468376+0000","last_unstale":"2023-11-28T13:00:26.347474+0000","last_undegraded":"2023-11-28T13:00:26.347474+0000","last_fullsized":"2023-11-28T13:00:26.347474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:51.024234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443541+0000","last_change":"2023-11-28T13:00:21.003466+0000","last_active":"2023-11-28T13:00:25.443541+0000","last_peered":"2023-11-28T13:00:25.443541+0000","last_clean":"2023-11-28T13:00:25.443541+0000","last_became_active":"2023-11-28T13:00:21.003127+0000","last_became_peered":"2023-11-28T13:00:21.003127+0000","last_unstale":"2023-11-28T13:00:25.443541+0000","last_undegraded":"2023-11-28T13:00:25.443541+0000","last_fullsized":"2023-11-28T13:00:25.443541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441650+0000","last_change":"2023-11-28T13:00:21.429933+0000","last_active":"2023-11-28T13:00:25.441650+0000","last_peered":"2023-11-28T13:00:25.441650+0000","last_clean":"2023-11-28T13:00:25.441650+0000","last_became_active":"2023-11-28T13:00:21.429681+0000","last_became_peered":"2023-11-28T13:00:21.429681+0000","last_unstale":"2023-11-28T13:00:25.441650+0000","last_undegraded":"2023-11-28T13:00:25.441650+0000","last_fullsized":"2023-11-28T13:00:25.441650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:45.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869412+0000","last_change":"2023-11-28T13:00:25.468518+0000","last_active":"2023-11-28T13:00:25.869412+0000","last_peered":"2023-11-28T13:00:25.869412+0000","last_clean":"2023-11-28T13:00:25.869412+0000","last_became_active":"2023-11-28T13:00:25.468083+0000","last_became_peered":"2023-11-28T13:00:25.468083+0000","last_unstale":"2023-11-28T13:00:25.869412+0000","last_undegraded":"2023-11-28T13:00:25.869412+0000","last_fullsized":"2023-11-28T13:00:25.869412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:53:56.156971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448763+0000","last_change":"2023-11-28T13:00:20.997868+0000","last_active":"2023-11-28T13:00:25.448763+0000","last_peered":"2023-11-28T13:00:25.448763+0000","last_clean":"2023-11-28T13:00:25.448763+0000","last_became_active":"2023-11-28T13:00:20.997565+0000","last_became_peered":"2023-11-28T13:00:20.997565+0000","last_unstale":"2023-11-28T13:00:25.448763+0000","last_undegraded":"2023-11-28T13:00:25.448763+0000","last_fullsized":"2023-11-28T13:00:25.448763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:37.465748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.451059+0000","last_change":"2023-11-28T13:00:21.431844+0000","last_active":"2023-11-28T13:00:25.451059+0000","last_peered":"2023-11-28T13:00:25.451059+0000","last_clean":"2023-11-28T13:00:25.451059+0000","last_became_active":"2023-11-28T13:00:21.431496+0000","last_became_peered":"2023-11-28T13:00:21.431496+0000","last_unstale":"2023-11-28T13:00:25.451059+0000","last_undegraded":"2023-11-28T13:00:25.451059+0000","last_fullsized":"2023-11-28T13:00:25.451059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.154908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869150+0000","last_change":"2023-11-28T13:00:25.469324+0000","last_active":"2023-11-28T13:00:25.869150+0000","last_peered":"2023-11-28T13:00:25.869150+0000","last_clean":"2023-11-28T13:00:25.869150+0000","last_became_active":"2023-11-28T13:00:25.468890+0000","last_became_peered":"2023-11-28T13:00:25.468890+0000","last_unstale":"2023-11-28T13:00:25.869150+0000","last_undegraded":"2023-11-28T13:00:25.869150+0000","last_fullsized":"2023-11-28T13:00:25.869150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:41.487196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443130+0000","last_change":"2023-11-28T13:00:21.002295+0000","last_active":"2023-11-28T13:00:25.443130+0000","last_peered":"2023-11-28T13:00:25.443130+0000","last_clean":"2023-11-28T13:00:25.443130+0000","last_became_active":"2023-11-28T13:00:21.001997+0000","last_became_peered":"2023-11-28T13:00:21.001997+0000","last_unstale":"2023-11-28T13:00:25.443130+0000","last_undegraded":"2023-11-28T13:00:25.443130+0000","last_fullsized":"2023-11-28T13:00:25.443130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449818+0000","last_change":"2023-11-28T13:00:21.431066+0000","last_active":"2023-11-28T13:00:25.449818+0000","last_peered":"2023-11-28T13:00:25.449818+0000","last_clean":"2023-11-28T13:00:25.449818+0000","last_became_active":"2023-11-28T13:00:21.430741+0000","last_became_peered":"2023-11-28T13:00:21.430741+0000","last_unstale":"2023-11-28T13:00:25.449818+0000","last_undegraded":"2023-11-28T13:00:25.449818+0000","last_fullsized":"2023-11-28T13:00:25.449818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:55.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347017+0000","last_change":"2023-11-28T13:00:25.468987+0000","last_active":"2023-11-28T13:00:26.347017+0000","last_peered":"2023-11-28T13:00:26.347017+0000","last_clean":"2023-11-28T13:00:26.347017+0000","last_became_active":"2023-11-28T13:00:25.468345+0000","last_became_peered":"2023-11-28T13:00:25.468345+0000","last_unstale":"2023-11-28T13:00:26.347017+0000","last_undegraded":"2023-11-28T13:00:26.347017+0000","last_fullsized":"2023-11-28T13:00:26.347017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:16.201562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441951+0000","last_change":"2023-11-28T13:00:20.993647+0000","last_active":"2023-11-28T13:00:25.441951+0000","last_peered":"2023-11-28T13:00:25.441951+0000","last_clean":"2023-11-28T13:00:25.441951+0000","last_became_active":"2023-11-28T13:00:20.993298+0000","last_became_peered":"2023-11-28T13:00:20.993298+0000","last_unstale":"2023-11-28T13:00:25.441951+0000","last_undegraded":"2023-11-28T13:00:25.441951+0000","last_fullsized":"2023-11-28T13:00:25.441951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:14:48.024917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442632+0000","last_change":"2023-11-28T13:00:21.429127+0000","last_active":"2023-11-28T13:00:25.442632+0000","last_peered":"2023-11-28T13:00:25.442632+0000","last_clean":"2023-11-28T13:00:25.442632+0000","last_became_active":"2023-11-28T13:00:21.428830+0000","last_became_peered":"2023-11-28T13:00:21.428830+0000","last_unstale":"2023-11-28T13:00:25.442632+0000","last_undegraded":"2023-11-28T13:00:25.442632+0000","last_fullsized":"2023-11-28T13:00:25.442632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:55.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871898+0000","last_change":"2023-11-28T13:00:25.464255+0000","last_active":"2023-11-28T13:00:25.871898+0000","last_peered":"2023-11-28T13:00:25.871898+0000","last_clean":"2023-11-28T13:00:25.871898+0000","last_became_active":"2023-11-28T13:00:25.463731+0000","last_became_peered":"2023-11-28T13:00:25.463731+0000","last_unstale":"2023-11-28T13:00:25.871898+0000","last_undegraded":"2023-11-28T13:00:25.871898+0000","last_fullsized":"2023-11-28T13:00:25.871898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:23:35.278662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.447931+0000","last_change":"2023-11-28T13:00:21.002097+0000","last_active":"2023-11-28T13:00:25.447931+0000","last_peered":"2023-11-28T13:00:25.447931+0000","last_clean":"2023-11-28T13:00:25.447931+0000","last_became_active":"2023-11-28T13:00:21.001830+0000","last_became_peered":"2023-11-28T13:00:21.001830+0000","last_unstale":"2023-11-28T13:00:25.447931+0000","last_undegraded":"2023-11-28T13:00:25.447931+0000","last_fullsized":"2023-11-28T13:00:25.447931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:18.850685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448366+0000","last_change":"2023-11-28T13:00:21.426617+0000","last_active":"2023-11-28T13:00:25.448366+0000","last_peered":"2023-11-28T13:00:25.448366+0000","last_clean":"2023-11-28T13:00:25.448366+0000","last_became_active":"2023-11-28T13:00:21.426326+0000","last_became_peered":"2023-11-28T13:00:21.426326+0000","last_unstale":"2023-11-28T13:00:25.448366+0000","last_undegraded":"2023-11-28T13:00:25.448366+0000","last_fullsized":"2023-11-28T13:00:25.448366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231013+0000","last_change":"2023-11-28T13:00:25.476625+0000","last_active":"2023-11-28T13:00:26.231013+0000","last_peered":"2023-11-28T13:00:26.231013+0000","last_clean":"2023-11-28T13:00:26.231013+0000","last_became_active":"2023-11-28T13:00:25.476137+0000","last_became_peered":"2023-11-28T13:00:25.476137+0000","last_unstale":"2023-11-28T13:00:26.231013+0000","last_undegraded":"2023-11-28T13:00:26.231013+0000","last_fullsized":"2023-11-28T13:00:26.231013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:28.631772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446518+0000","last_change":"2023-11-28T13:00:21.005624+0000","last_active":"2023-11-28T13:00:25.446518+0000","last_peered":"2023-11-28T13:00:25.446518+0000","last_clean":"2023-11-28T13:00:25.446518+0000","last_became_active":"2023-11-28T13:00:21.005092+0000","last_became_peered":"2023-11-28T13:00:21.005092+0000","last_unstale":"2023-11-28T13:00:25.446518+0000","last_undegraded":"2023-11-28T13:00:25.446518+0000","last_fullsized":"2023-11-28T13:00:25.446518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:37.060637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459224+0000","last_change":"2023-11-28T13:00:21.439380+0000","last_active":"2023-11-28T13:00:25.459224+0000","last_peered":"2023-11-28T13:00:25.459224+0000","last_clean":"2023-11-28T13:00:25.459224+0000","last_became_active":"2023-11-28T13:00:21.439094+0000","last_became_peered":"2023-11-28T13:00:21.439094+0000","last_unstale":"2023-11-28T13:00:25.459224+0000","last_undegraded":"2023-11-28T13:00:25.459224+0000","last_fullsized":"2023-11-28T13:00:25.459224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:02.248744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231207+0000","last_change":"2023-11-28T13:00:25.464879+0000","last_active":"2023-11-28T13:00:26.231207+0000","last_peered":"2023-11-28T13:00:26.231207+0000","last_clean":"2023-11-28T13:00:26.231207+0000","last_became_active":"2023-11-28T13:00:25.464486+0000","last_became_peered":"2023-11-28T13:00:25.464486+0000","last_unstale":"2023-11-28T13:00:26.231207+0000","last_undegraded":"2023-11-28T13:00:26.231207+0000","last_fullsized":"2023-11-28T13:00:26.231207+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:39.922810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446121+0000","last_change":"2023-11-28T13:00:21.003453+0000","last_active":"2023-11-28T13:00:25.446121+0000","last_peered":"2023-11-28T13:00:25.446121+0000","last_clean":"2023-11-28T13:00:25.446121+0000","last_became_active":"2023-11-28T13:00:21.003010+0000","last_became_peered":"2023-11-28T13:00:21.003010+0000","last_unstale":"2023-11-28T13:00:25.446121+0000","last_undegraded":"2023-11-28T13:00:25.446121+0000","last_fullsized":"2023-11-28T13:00:25.446121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:50.427721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442176+0000","last_change":"2023-11-28T13:00:21.428448+0000","last_active":"2023-11-28T13:00:25.442176+0000","last_peered":"2023-11-28T13:00:25.442176+0000","last_clean":"2023-11-28T13:00:25.442176+0000","last_became_active":"2023-11-28T13:00:21.427997+0000","last_became_peered":"2023-11-28T13:00:21.427997+0000","last_unstale":"2023-11-28T13:00:25.442176+0000","last_undegraded":"2023-11-28T13:00:25.442176+0000","last_fullsized":"2023-11-28T13:00:25.442176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:41.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231241+0000","last_change":"2023-11-28T13:00:25.465904+0000","last_active":"2023-11-28T13:00:26.231241+0000","last_peered":"2023-11-28T13:00:26.231241+0000","last_clean":"2023-11-28T13:00:26.231241+0000","last_became_active":"2023-11-28T13:00:25.465285+0000","last_became_peered":"2023-11-28T13:00:25.465285+0000","last_unstale":"2023-11-28T13:00:26.231241+0000","last_undegraded":"2023-11-28T13:00:26.231241+0000","last_fullsized":"2023-11-28T13:00:26.231241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:36.890229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450628+0000","last_change":"2023-11-28T13:00:21.010220+0000","last_active":"2023-11-28T13:00:25.450628+0000","last_peered":"2023-11-28T13:00:25.450628+0000","last_clean":"2023-11-28T13:00:25.450628+0000","last_became_active":"2023-11-28T13:00:21.009892+0000","last_became_peered":"2023-11-28T13:00:21.009892+0000","last_unstale":"2023-11-28T13:00:25.450628+0000","last_undegraded":"2023-11-28T13:00:25.450628+0000","last_fullsized":"2023-11-28T13:00:25.450628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448937+0000","last_change":"2023-11-28T13:00:21.421459+0000","last_active":"2023-11-28T13:00:25.448937+0000","last_peered":"2023-11-28T13:00:25.448937+0000","last_clean":"2023-11-28T13:00:25.448937+0000","last_became_active":"2023-11-28T13:00:21.420822+0000","last_became_peered":"2023-11-28T13:00:21.420822+0000","last_unstale":"2023-11-28T13:00:25.448937+0000","last_undegraded":"2023-11-28T13:00:25.448937+0000","last_fullsized":"2023-11-28T13:00:25.448937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:32:01.515518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231366+0000","last_change":"2023-11-28T13:00:25.464941+0000","last_active":"2023-11-28T13:00:26.231366+0000","last_peered":"2023-11-28T13:00:26.231366+0000","last_clean":"2023-11-28T13:00:26.231366+0000","last_became_active":"2023-11-28T13:00:25.464504+0000","last_became_peered":"2023-11-28T13:00:25.464504+0000","last_unstale":"2023-11-28T13:00:26.231366+0000","last_undegraded":"2023-11-28T13:00:26.231366+0000","last_fullsized":"2023-11-28T13:00:26.231366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:11.602479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441429+0000","last_change":"2023-11-28T13:00:21.008391+0000","last_active":"2023-11-28T13:00:25.441429+0000","last_peered":"2023-11-28T13:00:25.441429+0000","last_clean":"2023-11-28T13:00:25.441429+0000","last_became_active":"2023-11-28T13:00:21.006513+0000","last_became_peered":"2023-11-28T13:00:21.006513+0000","last_unstale":"2023-11-28T13:00:25.441429+0000","last_undegraded":"2023-11-28T13:00:25.441429+0000","last_fullsized":"2023-11-28T13:00:25.441429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:58.676237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460081+0000","last_change":"2023-11-28T13:00:21.431780+0000","last_active":"2023-11-28T13:00:25.460081+0000","last_peered":"2023-11-28T13:00:25.460081+0000","last_clean":"2023-11-28T13:00:25.460081+0000","last_became_active":"2023-11-28T13:00:21.430937+0000","last_became_peered":"2023-11-28T13:00:21.430937+0000","last_unstale":"2023-11-28T13:00:25.460081+0000","last_undegraded":"2023-11-28T13:00:25.460081+0000","last_fullsized":"2023-11-28T13:00:25.460081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:12:13.983682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076305+0000","last_change":"2023-11-28T13:00:25.466505+0000","last_active":"2023-11-28T13:00:26.076305+0000","last_peered":"2023-11-28T13:00:26.076305+0000","last_clean":"2023-11-28T13:00:26.076305+0000","last_became_active":"2023-11-28T13:00:25.466059+0000","last_became_peered":"2023-11-28T13:00:25.466059+0000","last_unstale":"2023-11-28T13:00:26.076305+0000","last_undegraded":"2023-11-28T13:00:26.076305+0000","last_fullsized":"2023-11-28T13:00:26.076305+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:12.856136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442306+0000","last_change":"2023-11-28T13:00:21.003064+0000","last_active":"2023-11-28T13:00:25.442306+0000","last_peered":"2023-11-28T13:00:25.442306+0000","last_clean":"2023-11-28T13:00:25.442306+0000","last_became_active":"2023-11-28T13:00:21.002742+0000","last_became_peered":"2023-11-28T13:00:21.002742+0000","last_unstale":"2023-11-28T13:00:25.442306+0000","last_undegraded":"2023-11-28T13:00:25.442306+0000","last_fullsized":"2023-11-28T13:00:25.442306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:04.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460629+0000","last_change":"2023-11-28T13:00:21.427213+0000","last_active":"2023-11-28T13:00:25.460629+0000","last_peered":"2023-11-28T13:00:25.460629+0000","last_clean":"2023-11-28T13:00:25.460629+0000","last_became_active":"2023-11-28T13:00:21.426737+0000","last_became_peered":"2023-11-28T13:00:21.426737+0000","last_unstale":"2023-11-28T13:00:25.460629+0000","last_undegraded":"2023-11-28T13:00:25.460629+0000","last_fullsized":"2023-11-28T13:00:25.460629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:43.321870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347427+0000","last_change":"2023-11-28T13:00:25.466146+0000","last_active":"2023-11-28T13:00:26.347427+0000","last_peered":"2023-11-28T13:00:26.347427+0000","last_clean":"2023-11-28T13:00:26.347427+0000","last_became_active":"2023-11-28T13:00:25.465652+0000","last_became_peered":"2023-11-28T13:00:25.465652+0000","last_unstale":"2023-11-28T13:00:26.347427+0000","last_undegraded":"2023-11-28T13:00:26.347427+0000","last_fullsized":"2023-11-28T13:00:26.347427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:14.273148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459071+0000","last_change":"2023-11-28T13:00:21.001745+0000","last_active":"2023-11-28T13:00:25.459071+0000","last_peered":"2023-11-28T13:00:25.459071+0000","last_clean":"2023-11-28T13:00:25.459071+0000","last_became_active":"2023-11-28T13:00:21.001473+0000","last_became_peered":"2023-11-28T13:00:21.001473+0000","last_unstale":"2023-11-28T13:00:25.459071+0000","last_undegraded":"2023-11-28T13:00:25.459071+0000","last_fullsized":"2023-11-28T13:00:25.459071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.462841+0000","last_change":"2023-11-28T13:00:21.422938+0000","last_active":"2023-11-28T13:00:25.462841+0000","last_peered":"2023-11-28T13:00:25.462841+0000","last_clean":"2023-11-28T13:00:25.462841+0000","last_became_active":"2023-11-28T13:00:21.422654+0000","last_became_peered":"2023-11-28T13:00:21.422654+0000","last_unstale":"2023-11-28T13:00:25.462841+0000","last_undegraded":"2023-11-28T13:00:25.462841+0000","last_fullsized":"2023-11-28T13:00:25.462841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:28:59.158475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.870965+0000","last_change":"2023-11-28T13:00:25.463080+0000","last_active":"2023-11-28T13:00:25.870965+0000","last_peered":"2023-11-28T13:00:25.870965+0000","last_clean":"2023-11-28T13:00:25.870965+0000","last_became_active":"2023-11-28T13:00:25.462460+0000","last_became_peered":"2023-11-28T13:00:25.462460+0000","last_unstale":"2023-11-28T13:00:25.870965+0000","last_undegraded":"2023-11-28T13:00:25.870965+0000","last_fullsized":"2023-11-28T13:00:25.870965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:45.428316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441326+0000","last_change":"2023-11-28T13:00:21.009988+0000","last_active":"2023-11-28T13:00:25.441326+0000","last_peered":"2023-11-28T13:00:25.441326+0000","last_clean":"2023-11-28T13:00:25.441326+0000","last_became_active":"2023-11-28T13:00:21.009637+0000","last_became_peered":"2023-11-28T13:00:21.009637+0000","last_unstale":"2023-11-28T13:00:25.441326+0000","last_undegraded":"2023-11-28T13:00:25.441326+0000","last_fullsized":"2023-11-28T13:00:25.441326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:53.009716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458749+0000","last_change":"2023-11-28T13:00:21.433621+0000","last_active":"2023-11-28T13:00:25.458749+0000","last_peered":"2023-11-28T13:00:25.458749+0000","last_clean":"2023-11-28T13:00:25.458749+0000","last_became_active":"2023-11-28T13:00:21.433293+0000","last_became_peered":"2023-11-28T13:00:21.433293+0000","last_unstale":"2023-11-28T13:00:25.458749+0000","last_undegraded":"2023-11-28T13:00:25.458749+0000","last_fullsized":"2023-11-28T13:00:25.458749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:18.248057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.870341+0000","last_change":"2023-11-28T13:00:25.460697+0000","last_active":"2023-11-28T13:00:25.870341+0000","last_peered":"2023-11-28T13:00:25.870341+0000","last_clean":"2023-11-28T13:00:25.870341+0000","last_became_active":"2023-11-28T13:00:25.460230+0000","last_became_peered":"2023-11-28T13:00:25.460230+0000","last_unstale":"2023-11-28T13:00:25.870341+0000","last_undegraded":"2023-11-28T13:00:25.870341+0000","last_fullsized":"2023-11-28T13:00:25.870341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:25.897486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442429+0000","last_change":"2023-11-28T13:00:21.001229+0000","last_active":"2023-11-28T13:00:25.442429+0000","last_peered":"2023-11-28T13:00:25.442429+0000","last_clean":"2023-11-28T13:00:25.442429+0000","last_became_active":"2023-11-28T13:00:21.000871+0000","last_became_peered":"2023-11-28T13:00:21.000871+0000","last_unstale":"2023-11-28T13:00:25.442429+0000","last_undegraded":"2023-11-28T13:00:25.442429+0000","last_fullsized":"2023-11-28T13:00:25.442429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:49.179565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460903+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:00:25.460903+0000","last_peered":"2023-11-28T13:00:25.460903+0000","last_clean":"2023-11-28T13:00:25.460903+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:00:25.460903+0000","last_undegraded":"2023-11-28T13:00:25.460903+0000","last_fullsized":"2023-11-28T13:00:25.460903+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441199+0000","last_change":"2023-11-28T13:00:21.429135+0000","last_active":"2023-11-28T13:00:25.441199+0000","last_peered":"2023-11-28T13:00:25.441199+0000","last_clean":"2023-11-28T13:00:25.441199+0000","last_became_active":"2023-11-28T13:00:21.428808+0000","last_became_peered":"2023-11-28T13:00:21.428808+0000","last_unstale":"2023-11-28T13:00:25.441199+0000","last_undegraded":"2023-11-28T13:00:25.441199+0000","last_fullsized":"2023-11-28T13:00:25.441199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:28:36.151382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.868731+0000","last_change":"2023-11-28T13:00:25.472188+0000","last_active":"2023-11-28T13:00:25.868731+0000","last_peered":"2023-11-28T13:00:25.868731+0000","last_clean":"2023-11-28T13:00:25.868731+0000","last_became_active":"2023-11-28T13:00:25.471768+0000","last_became_peered":"2023-11-28T13:00:25.471768+0000","last_unstale":"2023-11-28T13:00:25.868731+0000","last_undegraded":"2023-11-28T13:00:25.868731+0000","last_fullsized":"2023-11-28T13:00:25.868731+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:17:47.454963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441547+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:00:25.441547+0000","last_peered":"2023-11-28T13:00:25.441547+0000","last_clean":"2023-11-28T13:00:25.441547+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:00:25.441547+0000","last_undegraded":"2023-11-28T13:00:25.441547+0000","last_fullsized":"2023-11-28T13:00:25.441547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073655+0000","last_change":"2023-11-28T13:00:20.998799+0000","last_active":"2023-11-28T13:00:26.073655+0000","last_peered":"2023-11-28T13:00:26.073655+0000","last_clean":"2023-11-28T13:00:26.073655+0000","last_became_active":"2023-11-28T13:00:20.998482+0000","last_became_peered":"2023-11-28T13:00:20.998482+0000","last_unstale":"2023-11-28T13:00:26.073655+0000","last_undegraded":"2023-11-28T13:00:26.073655+0000","last_fullsized":"2023-11-28T13:00:26.073655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:28.673649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459183+0000","last_change":"2023-11-28T13:00:21.440454+0000","last_active":"2023-11-28T13:00:25.459183+0000","last_peered":"2023-11-28T13:00:25.459183+0000","last_clean":"2023-11-28T13:00:25.459183+0000","last_became_active":"2023-11-28T13:00:21.439616+0000","last_became_peered":"2023-11-28T13:00:21.439616+0000","last_unstale":"2023-11-28T13:00:25.459183+0000","last_undegraded":"2023-11-28T13:00:25.459183+0000","last_fullsized":"2023-11-28T13:00:25.459183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:36.472288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347208+0000","last_change":"2023-11-28T13:00:25.468244+0000","last_active":"2023-11-28T13:00:26.347208+0000","last_peered":"2023-11-28T13:00:26.347208+0000","last_clean":"2023-11-28T13:00:26.347208+0000","last_became_active":"2023-11-28T13:00:25.467850+0000","last_became_peered":"2023-11-28T13:00:25.467850+0000","last_unstale":"2023-11-28T13:00:26.347208+0000","last_undegraded":"2023-11-28T13:00:26.347208+0000","last_fullsized":"2023-11-28T13:00:26.347208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:47.712713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.456847+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:00:25.456847+0000","last_peered":"2023-11-28T13:00:25.456847+0000","last_clean":"2023-11-28T13:00:25.456847+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:00:25.456847+0000","last_undegraded":"2023-11-28T13:00:25.456847+0000","last_fullsized":"2023-11-28T13:00:25.456847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446771+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:00:25.446771+0000","last_peered":"2023-11-28T13:00:25.446771+0000","last_clean":"2023-11-28T13:00:25.446771+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:00:25.446771+0000","last_undegraded":"2023-11-28T13:00:25.446771+0000","last_fullsized":"2023-11-28T13:00:25.446771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.447961+0000","last_change":"2023-11-28T13:00:21.005940+0000","last_active":"2023-11-28T13:00:25.447961+0000","last_peered":"2023-11-28T13:00:25.447961+0000","last_clean":"2023-11-28T13:00:25.447961+0000","last_became_active":"2023-11-28T13:00:21.005231+0000","last_became_peered":"2023-11-28T13:00:21.005231+0000","last_unstale":"2023-11-28T13:00:25.447961+0000","last_undegraded":"2023-11-28T13:00:25.447961+0000","last_fullsized":"2023-11-28T13:00:25.447961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:49.179565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448989+0000","last_change":"2023-11-28T13:00:21.426195+0000","last_active":"2023-11-28T13:00:25.448989+0000","last_peered":"2023-11-28T13:00:25.448989+0000","last_clean":"2023-11-28T13:00:25.448989+0000","last_became_active":"2023-11-28T13:00:21.423397+0000","last_became_peered":"2023-11-28T13:00:21.423397+0000","last_unstale":"2023-11-28T13:00:25.448989+0000","last_undegraded":"2023-11-28T13:00:25.448989+0000","last_fullsized":"2023-11-28T13:00:25.448989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231539+0000","last_change":"2023-11-28T13:00:25.468335+0000","last_active":"2023-11-28T13:00:26.231539+0000","last_peered":"2023-11-28T13:00:26.231539+0000","last_clean":"2023-11-28T13:00:26.231539+0000","last_became_active":"2023-11-28T13:00:25.467898+0000","last_became_peered":"2023-11-28T13:00:25.467898+0000","last_unstale":"2023-11-28T13:00:26.231539+0000","last_undegraded":"2023-11-28T13:00:26.231539+0000","last_fullsized":"2023-11-28T13:00:26.231539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:20.872923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443294+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:00:25.443294+0000","last_peered":"2023-11-28T13:00:25.443294+0000","last_clean":"2023-11-28T13:00:25.443294+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:00:25.443294+0000","last_undegraded":"2023-11-28T13:00:25.443294+0000","last_fullsized":"2023-11-28T13:00:25.443294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460654+0000","last_change":"2023-11-28T13:00:21.007444+0000","last_active":"2023-11-28T13:00:25.460654+0000","last_peered":"2023-11-28T13:00:25.460654+0000","last_clean":"2023-11-28T13:00:25.460654+0000","last_became_active":"2023-11-28T13:00:21.007159+0000","last_became_peered":"2023-11-28T13:00:21.007159+0000","last_unstale":"2023-11-28T13:00:25.460654+0000","last_undegraded":"2023-11-28T13:00:25.460654+0000","last_fullsized":"2023-11-28T13:00:25.460654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:55.415887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458654+0000","last_change":"2023-11-28T13:00:21.437327+0000","last_active":"2023-11-28T13:00:25.458654+0000","last_peered":"2023-11-28T13:00:25.458654+0000","last_clean":"2023-11-28T13:00:25.458654+0000","last_became_active":"2023-11-28T13:00:21.437058+0000","last_became_peered":"2023-11-28T13:00:21.437058+0000","last_unstale":"2023-11-28T13:00:25.458654+0000","last_undegraded":"2023-11-28T13:00:25.458654+0000","last_fullsized":"2023-11-28T13:00:25.458654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871438+0000","last_change":"2023-11-28T13:00:25.462821+0000","last_active":"2023-11-28T13:00:25.871438+0000","last_peered":"2023-11-28T13:00:25.871438+0000","last_clean":"2023-11-28T13:00:25.871438+0000","last_became_active":"2023-11-28T13:00:25.462378+0000","last_became_peered":"2023-11-28T13:00:25.462378+0000","last_unstale":"2023-11-28T13:00:25.871438+0000","last_undegraded":"2023-11-28T13:00:25.871438+0000","last_fullsized":"2023-11-28T13:00:25.871438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:06:52.582450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446128+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:00:25.446128+0000","last_peered":"2023-11-28T13:00:25.446128+0000","last_clean":"2023-11-28T13:00:25.446128+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:00:25.446128+0000","last_undegraded":"2023-11-28T13:00:25.446128+0000","last_fullsized":"2023-11-28T13:00:25.446128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441243+0000","last_change":"2023-11-28T13:00:21.005259+0000","last_active":"2023-11-28T13:00:25.441243+0000","last_peered":"2023-11-28T13:00:25.441243+0000","last_clean":"2023-11-28T13:00:25.441243+0000","last_became_active":"2023-11-28T13:00:21.004973+0000","last_became_peered":"2023-11-28T13:00:21.004973+0000","last_unstale":"2023-11-28T13:00:25.441243+0000","last_undegraded":"2023-11-28T13:00:25.441243+0000","last_fullsized":"2023-11-28T13:00:25.441243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:18.754380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442055+0000","last_change":"2023-11-28T13:00:21.428421+0000","last_active":"2023-11-28T13:00:25.442055+0000","last_peered":"2023-11-28T13:00:25.442055+0000","last_clean":"2023-11-28T13:00:25.442055+0000","last_became_active":"2023-11-28T13:00:21.427914+0000","last_became_peered":"2023-11-28T13:00:21.427914+0000","last_unstale":"2023-11-28T13:00:25.442055+0000","last_undegraded":"2023-11-28T13:00:25.442055+0000","last_fullsized":"2023-11-28T13:00:25.442055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:13.464393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073795+0000","last_change":"2023-11-28T13:00:25.468907+0000","last_active":"2023-11-28T13:00:26.073795+0000","last_peered":"2023-11-28T13:00:26.073795+0000","last_clean":"2023-11-28T13:00:26.073795+0000","last_became_active":"2023-11-28T13:00:25.468477+0000","last_became_peered":"2023-11-28T13:00:25.468477+0000","last_unstale":"2023-11-28T13:00:26.073795+0000","last_undegraded":"2023-11-28T13:00:26.073795+0000","last_fullsized":"2023-11-28T13:00:26.073795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:36:04.014440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448786+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:00:25.448786+0000","last_peered":"2023-11-28T13:00:25.448786+0000","last_clean":"2023-11-28T13:00:25.448786+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:00:25.448786+0000","last_undegraded":"2023-11-28T13:00:25.448786+0000","last_fullsized":"2023-11-28T13:00:25.448786+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460768+0000","last_change":"2023-11-28T13:00:21.008135+0000","last_active":"2023-11-28T13:00:25.460768+0000","last_peered":"2023-11-28T13:00:25.460768+0000","last_clean":"2023-11-28T13:00:25.460768+0000","last_became_active":"2023-11-28T13:00:21.007883+0000","last_became_peered":"2023-11-28T13:00:21.007883+0000","last_unstale":"2023-11-28T13:00:25.460768+0000","last_undegraded":"2023-11-28T13:00:25.460768+0000","last_fullsized":"2023-11-28T13:00:25.460768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455821+0000","last_change":"2023-11-28T13:00:21.425530+0000","last_active":"2023-11-28T13:00:25.455821+0000","last_peered":"2023-11-28T13:00:25.455821+0000","last_clean":"2023-11-28T13:00:25.455821+0000","last_became_active":"2023-11-28T13:00:21.425325+0000","last_became_peered":"2023-11-28T13:00:21.425325+0000","last_unstale":"2023-11-28T13:00:25.455821+0000","last_undegraded":"2023-11-28T13:00:25.455821+0000","last_fullsized":"2023-11-28T13:00:25.455821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:43.401826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346874+0000","last_change":"2023-11-28T13:00:25.457191+0000","last_active":"2023-11-28T13:00:26.346874+0000","last_peered":"2023-11-28T13:00:26.346874+0000","last_clean":"2023-11-28T13:00:26.346874+0000","last_became_active":"2023-11-28T13:00:25.456577+0000","last_became_peered":"2023-11-28T13:00:25.456577+0000","last_unstale":"2023-11-28T13:00:26.346874+0000","last_undegraded":"2023-11-28T13:00:26.346874+0000","last_fullsized":"2023-11-28T13:00:26.346874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:11.602479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442371+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:00:25.442371+0000","last_peered":"2023-11-28T13:00:25.442371+0000","last_clean":"2023-11-28T13:00:25.442371+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:00:25.442371+0000","last_undegraded":"2023-11-28T13:00:25.442371+0000","last_fullsized":"2023-11-28T13:00:25.442371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455999+0000","last_change":"2023-11-28T13:00:21.006815+0000","last_active":"2023-11-28T13:00:25.455999+0000","last_peered":"2023-11-28T13:00:25.455999+0000","last_clean":"2023-11-28T13:00:25.455999+0000","last_became_active":"2023-11-28T13:00:21.006608+0000","last_became_peered":"2023-11-28T13:00:21.006608+0000","last_unstale":"2023-11-28T13:00:25.455999+0000","last_undegraded":"2023-11-28T13:00:25.455999+0000","last_fullsized":"2023-11-28T13:00:25.455999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:28.673649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448670+0000","last_change":"2023-11-28T13:00:21.421641+0000","last_active":"2023-11-28T13:00:25.448670+0000","last_peered":"2023-11-28T13:00:25.448670+0000","last_clean":"2023-11-28T13:00:25.448670+0000","last_became_active":"2023-11-28T13:00:21.421196+0000","last_became_peered":"2023-11-28T13:00:21.421196+0000","last_unstale":"2023-11-28T13:00:25.448670+0000","last_undegraded":"2023-11-28T13:00:25.448670+0000","last_fullsized":"2023-11-28T13:00:25.448670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:49.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073549+0000","last_change":"2023-11-28T13:00:25.468640+0000","last_active":"2023-11-28T13:00:26.073549+0000","last_peered":"2023-11-28T13:00:26.073549+0000","last_clean":"2023-11-28T13:00:26.073549+0000","last_became_active":"2023-11-28T13:00:25.467992+0000","last_became_peered":"2023-11-28T13:00:25.467992+0000","last_unstale":"2023-11-28T13:00:26.073549+0000","last_undegraded":"2023-11-28T13:00:26.073549+0000","last_fullsized":"2023-11-28T13:00:26.073549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:45:50.526405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442668+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:00:25.442668+0000","last_peered":"2023-11-28T13:00:25.442668+0000","last_clean":"2023-11-28T13:00:25.442668+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:00:25.442668+0000","last_undegraded":"2023-11-28T13:00:25.442668+0000","last_fullsized":"2023-11-28T13:00:25.442668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450339+0000","last_change":"2023-11-28T13:00:21.010100+0000","last_active":"2023-11-28T13:00:25.450339+0000","last_peered":"2023-11-28T13:00:25.450339+0000","last_clean":"2023-11-28T13:00:25.450339+0000","last_became_active":"2023-11-28T13:00:21.009797+0000","last_became_peered":"2023-11-28T13:00:21.009797+0000","last_unstale":"2023-11-28T13:00:25.450339+0000","last_undegraded":"2023-11-28T13:00:25.450339+0000","last_fullsized":"2023-11-28T13:00:25.450339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:18.850685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457393+0000","last_change":"2023-11-28T13:00:21.433346+0000","last_active":"2023-11-28T13:00:25.457393+0000","last_peered":"2023-11-28T13:00:25.457393+0000","last_clean":"2023-11-28T13:00:25.457393+0000","last_became_active":"2023-11-28T13:00:21.433022+0000","last_became_peered":"2023-11-28T13:00:21.433022+0000","last_unstale":"2023-11-28T13:00:25.457393+0000","last_undegraded":"2023-11-28T13:00:25.457393+0000","last_fullsized":"2023-11-28T13:00:25.457393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:34:14.133792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076291+0000","last_change":"2023-11-28T13:00:25.474463+0000","last_active":"2023-11-28T13:00:26.076291+0000","last_peered":"2023-11-28T13:00:26.076291+0000","last_clean":"2023-11-28T13:00:26.076291+0000","last_became_active":"2023-11-28T13:00:25.473727+0000","last_became_peered":"2023-11-28T13:00:25.473727+0000","last_unstale":"2023-11-28T13:00:26.076291+0000","last_undegraded":"2023-11-28T13:00:26.076291+0000","last_fullsized":"2023-11-28T13:00:26.076291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:35:49.356939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448491+0000","last_change":"2023-11-28T13:00:21.005791+0000","last_active":"2023-11-28T13:00:25.448491+0000","last_peered":"2023-11-28T13:00:25.448491+0000","last_clean":"2023-11-28T13:00:25.448491+0000","last_became_active":"2023-11-28T13:00:21.005474+0000","last_became_peered":"2023-11-28T13:00:21.005474+0000","last_unstale":"2023-11-28T13:00:25.448491+0000","last_undegraded":"2023-11-28T13:00:25.448491+0000","last_fullsized":"2023-11-28T13:00:25.448491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:14.890428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455579+0000","last_change":"2023-11-28T13:00:21.424133+0000","last_active":"2023-11-28T13:00:25.455579+0000","last_peered":"2023-11-28T13:00:25.455579+0000","last_clean":"2023-11-28T13:00:25.455579+0000","last_became_active":"2023-11-28T13:00:21.423849+0000","last_became_peered":"2023-11-28T13:00:21.423849+0000","last_unstale":"2023-11-28T13:00:25.455579+0000","last_undegraded":"2023-11-28T13:00:25.455579+0000","last_fullsized":"2023-11-28T13:00:25.455579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:05.801837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871653+0000","last_change":"2023-11-28T13:00:25.468445+0000","last_active":"2023-11-28T13:00:25.871653+0000","last_peered":"2023-11-28T13:00:25.871653+0000","last_clean":"2023-11-28T13:00:25.871653+0000","last_became_active":"2023-11-28T13:00:25.468042+0000","last_became_peered":"2023-11-28T13:00:25.468042+0000","last_unstale":"2023-11-28T13:00:25.871653+0000","last_undegraded":"2023-11-28T13:00:25.871653+0000","last_fullsized":"2023-11-28T13:00:25.871653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.716007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449852+0000","last_change":"2023-11-28T13:00:21.006819+0000","last_active":"2023-11-28T13:00:25.449852+0000","last_peered":"2023-11-28T13:00:25.449852+0000","last_clean":"2023-11-28T13:00:25.449852+0000","last_became_active":"2023-11-28T13:00:21.006504+0000","last_became_peered":"2023-11-28T13:00:21.006504+0000","last_unstale":"2023-11-28T13:00:25.449852+0000","last_undegraded":"2023-11-28T13:00:25.449852+0000","last_fullsized":"2023-11-28T13:00:25.449852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:37.060637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459182+0000","last_change":"2023-11-28T13:00:21.436419+0000","last_active":"2023-11-28T13:00:25.459182+0000","last_peered":"2023-11-28T13:00:25.459182+0000","last_clean":"2023-11-28T13:00:25.459182+0000","last_became_active":"2023-11-28T13:00:21.436087+0000","last_became_peered":"2023-11-28T13:00:21.436087+0000","last_unstale":"2023-11-28T13:00:25.459182+0000","last_undegraded":"2023-11-28T13:00:25.459182+0000","last_fullsized":"2023-11-28T13:00:25.459182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:18.316015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346972+0000","last_change":"2023-11-28T13:00:25.464176+0000","last_active":"2023-11-28T13:00:26.346972+0000","last_peered":"2023-11-28T13:00:26.346972+0000","last_clean":"2023-11-28T13:00:26.346972+0000","last_became_active":"2023-11-28T13:00:25.463779+0000","last_became_peered":"2023-11-28T13:00:25.463779+0000","last_unstale":"2023-11-28T13:00:26.346972+0000","last_undegraded":"2023-11-28T13:00:26.346972+0000","last_fullsized":"2023-11-28T13:00:26.346972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.716007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450222+0000","last_change":"2023-11-28T13:00:21.006862+0000","last_active":"2023-11-28T13:00:25.450222+0000","last_peered":"2023-11-28T13:00:25.450222+0000","last_clean":"2023-11-28T13:00:25.450222+0000","last_became_active":"2023-11-28T13:00:21.006580+0000","last_became_peered":"2023-11-28T13:00:21.006580+0000","last_unstale":"2023-11-28T13:00:25.450222+0000","last_undegraded":"2023-11-28T13:00:25.450222+0000","last_fullsized":"2023-11-28T13:00:25.450222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:01.729666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460103+0000","last_change":"2023-11-28T13:00:21.427093+0000","last_active":"2023-11-28T13:00:25.460103+0000","last_peered":"2023-11-28T13:00:25.460103+0000","last_clean":"2023-11-28T13:00:25.460103+0000","last_became_active":"2023-11-28T13:00:21.426329+0000","last_became_peered":"2023-11-28T13:00:21.426329+0000","last_unstale":"2023-11-28T13:00:25.460103+0000","last_undegraded":"2023-11-28T13:00:25.460103+0000","last_fullsized":"2023-11-28T13:00:25.460103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:26:04.532312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869619+0000","last_change":"2023-11-28T13:00:25.475157+0000","last_active":"2023-11-28T13:00:25.869619+0000","last_peered":"2023-11-28T13:00:25.869619+0000","last_clean":"2023-11-28T13:00:25.869619+0000","last_became_active":"2023-11-28T13:00:25.474714+0000","last_became_peered":"2023-11-28T13:00:25.474714+0000","last_unstale":"2023-11-28T13:00:25.869619+0000","last_undegraded":"2023-11-28T13:00:25.869619+0000","last_fullsized":"2023-11-28T13:00:25.869619+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:43.383854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441539+0000","last_change":"2023-11-28T13:00:21.004157+0000","last_active":"2023-11-28T13:00:25.441539+0000","last_peered":"2023-11-28T13:00:25.441539+0000","last_clean":"2023-11-28T13:00:25.441539+0000","last_became_active":"2023-11-28T13:00:21.003717+0000","last_became_peered":"2023-11-28T13:00:21.003717+0000","last_unstale":"2023-11-28T13:00:25.441539+0000","last_undegraded":"2023-11-28T13:00:25.441539+0000","last_fullsized":"2023-11-28T13:00:25.441539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:14.599072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441293+0000","last_change":"2023-11-28T13:00:21.427401+0000","last_active":"2023-11-28T13:00:25.441293+0000","last_peered":"2023-11-28T13:00:25.441293+0000","last_clean":"2023-11-28T13:00:25.441293+0000","last_became_active":"2023-11-28T13:00:21.425193+0000","last_became_peered":"2023-11-28T13:00:21.425193+0000","last_unstale":"2023-11-28T13:00:25.441293+0000","last_undegraded":"2023-11-28T13:00:25.441293+0000","last_fullsized":"2023-11-28T13:00:25.441293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:16.691765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076011+0000","last_change":"2023-11-28T13:00:25.466935+0000","last_active":"2023-11-28T13:00:26.076011+0000","last_peered":"2023-11-28T13:00:26.076011+0000","last_clean":"2023-11-28T13:00:26.076011+0000","last_became_active":"2023-11-28T13:00:25.466438+0000","last_became_peered":"2023-11-28T13:00:25.466438+0000","last_unstale":"2023-11-28T13:00:26.076011+0000","last_undegraded":"2023-11-28T13:00:26.076011+0000","last_fullsized":"2023-11-28T13:00:26.076011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:44.975678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448944+0000","last_change":"2023-11-28T13:00:21.003458+0000","last_active":"2023-11-28T13:00:25.448944+0000","last_peered":"2023-11-28T13:00:25.448944+0000","last_clean":"2023-11-28T13:00:25.448944+0000","last_became_active":"2023-11-28T13:00:21.002912+0000","last_became_peered":"2023-11-28T13:00:21.002912+0000","last_unstale":"2023-11-28T13:00:25.448944+0000","last_undegraded":"2023-11-28T13:00:25.448944+0000","last_fullsized":"2023-11-28T13:00:25.448944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:18.754380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458864+0000","last_change":"2023-11-28T13:00:21.422316+0000","last_active":"2023-11-28T13:00:25.458864+0000","last_peered":"2023-11-28T13:00:25.458864+0000","last_clean":"2023-11-28T13:00:25.458864+0000","last_became_active":"2023-11-28T13:00:21.422012+0000","last_became_peered":"2023-11-28T13:00:21.422012+0000","last_unstale":"2023-11-28T13:00:25.458864+0000","last_undegraded":"2023-11-28T13:00:25.458864+0000","last_fullsized":"2023-11-28T13:00:25.458864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442053+0000","last_change":"2023-11-28T13:00:21.423769+0000","last_active":"2023-11-28T13:00:25.442053+0000","last_peered":"2023-11-28T13:00:25.442053+0000","last_clean":"2023-11-28T13:00:25.442053+0000","last_became_active":"2023-11-28T13:00:21.423458+0000","last_became_peered":"2023-11-28T13:00:21.423458+0000","last_unstale":"2023-11-28T13:00:25.442053+0000","last_undegraded":"2023-11-28T13:00:25.442053+0000","last_fullsized":"2023-11-28T13:00:25.442053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:51.438322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441431+0000","last_change":"2023-11-28T13:00:21.432489+0000","last_active":"2023-11-28T13:00:25.441431+0000","last_peered":"2023-11-28T13:00:25.441431+0000","last_clean":"2023-11-28T13:00:25.441431+0000","last_became_active":"2023-11-28T13:00:21.432224+0000","last_became_peered":"2023-11-28T13:00:21.432224+0000","last_unstale":"2023-11-28T13:00:25.441431+0000","last_undegraded":"2023-11-28T13:00:25.441431+0000","last_fullsized":"2023-11-28T13:00:25.441431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:27.442626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460221+0000","last_change":"2023-11-28T13:00:21.427335+0000","last_active":"2023-11-28T13:00:25.460221+0000","last_peered":"2023-11-28T13:00:25.460221+0000","last_clean":"2023-11-28T13:00:25.460221+0000","last_became_active":"2023-11-28T13:00:21.426777+0000","last_became_peered":"2023-11-28T13:00:21.426777+0000","last_unstale":"2023-11-28T13:00:25.460221+0000","last_undegraded":"2023-11-28T13:00:25.460221+0000","last_fullsized":"2023-11-28T13:00:25.460221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:24.706933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441759+0000","last_change":"2023-11-28T13:00:21.436760+0000","last_active":"2023-11-28T13:00:25.441759+0000","last_peered":"2023-11-28T13:00:25.441759+0000","last_clean":"2023-11-28T13:00:25.441759+0000","last_became_active":"2023-11-28T13:00:21.436476+0000","last_became_peered":"2023-11-28T13:00:21.436476+0000","last_unstale":"2023-11-28T13:00:25.441759+0000","last_undegraded":"2023-11-28T13:00:25.441759+0000","last_fullsized":"2023-11-28T13:00:25.441759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:49.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455911+0000","last_change":"2023-11-28T13:00:21.415227+0000","last_active":"2023-11-28T13:00:25.455911+0000","last_peered":"2023-11-28T13:00:25.455911+0000","last_clean":"2023-11-28T13:00:25.455911+0000","last_became_active":"2023-11-28T13:00:21.414918+0000","last_became_peered":"2023-11-28T13:00:21.414918+0000","last_unstale":"2023-11-28T13:00:25.455911+0000","last_undegraded":"2023-11-28T13:00:25.455911+0000","last_fullsized":"2023-11-28T13:00:25.455911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:35:41.989664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4528,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":704092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9228,"kb_used_data":836,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362612,"statfs":{"total":96636764160,"available":96627314688,"internally_reserved":0,"allocated":856064,"data_stored":699125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8140,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363700,"statfs":{"total":96636764160,"available":96628428800,"internally_reserved":0,"allocated":266240,"data_stored":109342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0540000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6919999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8084,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363756,"statfs":{"total":96636764160,"available":96628486144,"internally_reserved":0,"allocated":274432,"data_stored":110398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:42.119 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:00:42.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.276+0000 7f5ca384c700 1 Processor -- start 2023-11-28T13:00:42.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.277+0000 7f5ca384c700 1 -- start start 2023-11-28T13:00:42.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.277+0000 7f5ca384c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c06a560 0x7f5c9c06a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.277+0000 7f5ca384c700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9c0cdec0 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.277+0000 7f5ca384c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9c0c9cd0 con 0x7f5c9c15d1e0 2023-11-28T13:00:42.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.277+0000 7f5ca384c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c06af10 con 0x7f5c9c06a560 2023-11-28T13:00:42.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5ca1de9700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f5c9c0cceb0 0x7f5c9c0cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45692/0 (socket says 172.21.15.6:45692) 2023-11-28T13:00:42.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5ca1de9700 1 -- 172.21.15.6:0/1596727591 learned_addr learned my addr 172.21.15.6:0/1596727591 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:42.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5ca0de7700 1 --2- 172.21.15.6:0/1596727591 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c06a560 0x7f5c9c06a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5c8ffff700 1 -- 172.21.15.6:0/1596727591 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957177183 0 0) 0x7f5c9c0cdec0 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5c8ffff700 1 -- 172.21.15.6:0/1596727591 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c880035b0 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5c8ffff700 1 -- 172.21.15.6:0/1596727591 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070535197 0 0) 0x7f5c9c0c9cd0 con 0x7f5c9c15d1e0 2023-11-28T13:00:42.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5c8ffff700 1 -- 172.21.15.6:0/1596727591 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c9c0cdec0 con 0x7f5c9c15d1e0 2023-11-28T13:00:42.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5ca0de7700 1 -- 172.21.15.6:0/1596727591 >> v1:172.21.15.61:6789/0 conn(0x7f5c9c15d1e0 legacy=0x7f5c9c15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5ca0de7700 1 -- 172.21.15.6:0/1596727591 >> v1:172.21.15.6:6789/0 conn(0x7f5c9c0cceb0 legacy=0x7f5c9c0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5ca0de7700 1 -- 172.21.15.6:0/1596727591 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c0c9fb0 con 0x7f5c9c06a560 2023-11-28T13:00:42.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5ca0de7700 1 --2- 172.21.15.6:0/1596727591 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c06a560 0x7f5c9c06a9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009870 tx=0x7f5c90009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c4200acb16d7a1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5c8ffff700 1 -- 172.21.15.6:0/1596727591 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90011830 con 0x7f5c9c06a560 2023-11-28T13:00:42.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.278+0000 7f5c8ffff700 1 -- 172.21.15.6:0/1596727591 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c90011990 con 0x7f5c9c06a560 2023-11-28T13:00:42.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.279+0000 7f5c8ffff700 1 -- 172.21.15.6:0/1596727591 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90011d20 con 0x7f5c9c06a560 2023-11-28T13:00:42.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.279+0000 7f5ca384c700 1 -- 172.21.15.6:0/1596727591 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c06a560 msgr2=0x7f5c9c06a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.279+0000 7f5ca384c700 1 --2- 172.21.15.6:0/1596727591 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c06a560 0x7f5c9c06a9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c90009870 tx=0x7f5c90009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:42.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.279+0000 7f5ca384c700 1 -- 172.21.15.6:0/1596727591 shutdown_connections 2023-11-28T13:00:42.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.279+0000 7f5ca384c700 1 --2- 172.21.15.6:0/1596727591 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c06a560 0x7f5c9c06a9d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.279+0000 7f5ca384c700 1 -- 172.21.15.6:0/1596727591 >> 172.21.15.6:0/1596727591 conn(0x7f5c9c0b0dd0 msgr2=0x7f5c9c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.279+0000 7f5ca384c700 1 -- 172.21.15.6:0/1596727591 shutdown_connections 2023-11-28T13:00:42.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.279+0000 7f5ca384c700 1 -- 172.21.15.6:0/1596727591 wait complete. 2023-11-28T13:00:42.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.280+0000 7f5ca384c700 1 Processor -- start 2023-11-28T13:00:42.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.280+0000 7f5ca384c700 1 -- start start 2023-11-28T13:00:42.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca384c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c9c06a560 0x7f5c9c1f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca384c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c9c0cceb0 0x7f5c9c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca384c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c15d1e0 0x7f5c9c15b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca384c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c1f5f50 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca384c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c1f5620 con 0x7f5c9c06a560 2023-11-28T13:00:42.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca384c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c1f5900 con 0x7f5c9c15d1e0 2023-11-28T13:00:42.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca0de7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c9c0cceb0 0x7f5c9c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca0de7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c9c0cceb0 0x7f5c9c15b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58376/0 (socket says 172.21.15.6:58376) 2023-11-28T13:00:42.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca0de7700 1 -- 172.21.15.6:0/2741863594 learned_addr learned my addr 172.21.15.6:0/2741863594 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca15e8700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c9c06a560 0x7f5c9c1f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.281+0000 7f5ca1de9700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c15d1e0 0x7f5c9c15b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5ca0de7700 1 -- 172.21.15.6:0/2741863594 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c15d1e0 msgr2=0x7f5c9c15b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5ca0de7700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c15d1e0 0x7f5c9c15b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5ca0de7700 1 -- 172.21.15.6:0/2741863594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c9c06a560 msgr2=0x7f5c9c1f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5ca0de7700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c9c06a560 0x7f5c9c1f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5ca0de7700 1 -- 172.21.15.6:0/2741863594 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900094a0 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5ca0de7700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c9c0cceb0 0x7f5c9c15b430 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c900090b0 tx=0x7f5c90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5c8dffb700 1 -- 172.21.15.6:0/2741863594 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9001a860 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5c8dffb700 1 -- 172.21.15.6:0/2741863594 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9001a9c0 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5ca384c700 1 -- 172.21.15.6:0/2741863594 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c0ca570 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5c8dffb700 1 -- 172.21.15.6:0/2741863594 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9001ad30 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.282+0000 7f5ca384c700 1 -- 172.21.15.6:0/2741863594 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c0ca290 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.283+0000 7f5ca384c700 1 -- 172.21.15.6:0/2741863594 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c0ca850 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.284+0000 7f5c8dffb700 1 -- 172.21.15.6:0/2741863594 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c9002b030 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.284+0000 7f5c8dffb700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c800682f0 0x7f5c8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.284+0000 7f5c8dffb700 1 -- 172.21.15.6:0/2741863594 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5c900871e0 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.284+0000 7f5ca15e8700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c800682f0 0x7f5c8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.284+0000 7f5ca15e8700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c800682f0 0x7f5c8006a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5c980011e0 tx=0x7f5c98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.288+0000 7f5c8dffb700 1 -- 172.21.15.6:0/2741863594 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c90053750 con 0x7f5c9c0cceb0 2023-11-28T13:00:42.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.478+0000 7f5ca384c700 1 -- 172.21.15.6:0/2741863594 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5c9c0675a0 con 0x7f5c800682f0 2023-11-28T13:00:42.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.488+0000 7f5c8dffb700 1 -- 172.21.15.6:0/2741863594 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367535 (secure 0 0 0) 0x7f5c9c0675a0 con 0x7f5c800682f0 2023-11-28T13:00:42.493 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 -- 172.21.15.6:0/2741863594 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c800682f0 msgr2=0x7f5c8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c800682f0 0x7f5c8006a7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5c980011e0 tx=0x7f5c98009040 comp rx=0 tx=0).stop 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 -- 172.21.15.6:0/2741863594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c9c0cceb0 msgr2=0x7f5c9c15b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c9c0cceb0 0x7f5c9c15b430 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5c900090b0 tx=0x7f5c90004750 comp rx=0 tx=0).stop 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 -- 172.21.15.6:0/2741863594 shutdown_connections 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c800682f0 0x7f5c8006a7a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c9c15d1e0 0x7f5c9c15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c9c06a560 0x7f5c9c1f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 --2- 172.21.15.6:0/2741863594 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c9c0cceb0 0x7f5c9c15b430 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 -- 172.21.15.6:0/2741863594 >> 172.21.15.6:0/2741863594 conn(0x7f5c9c0b0dd0 msgr2=0x7f5c9c0cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.491+0000 7f5ca384c700 1 -- 172.21.15.6:0/2741863594 shutdown_connections 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.492+0000 7f5ca384c700 1 -- 172.21.15.6:0/2741863594 wait complete. 2023-11-28T13:00:42.497 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:00:42.505 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-11-28T13:00:40.946475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594896,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67232,"kb_used_data":3296,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907488,"statfs":{"total":773094113280,"available":773025267712,"internally_reserved":0,"allocated":3375104,"data_stored":2065897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2198,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002515"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.456044+0000","last_change":"2023-11-28T13:00:21.430642+0000","last_active":"2023-11-28T13:00:25.456044+0000","last_peered":"2023-11-28T13:00:25.456044+0000","last_clean":"2023-11-28T13:00:25.456044+0000","last_became_active":"2023-11-28T13:00:21.430357+0000","last_became_peered":"2023-11-28T13:00:21.430357+0000","last_unstale":"2023-11-28T13:00:25.456044+0000","last_undegraded":"2023-11-28T13:00:25.456044+0000","last_fullsized":"2023-11-28T13:00:25.456044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:43.433683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452930+0000","last_change":"2023-11-28T13:00:21.432299+0000","last_active":"2023-11-28T13:00:25.452930+0000","last_peered":"2023-11-28T13:00:25.452930+0000","last_clean":"2023-11-28T13:00:25.452930+0000","last_became_active":"2023-11-28T13:00:21.431566+0000","last_became_peered":"2023-11-28T13:00:21.431566+0000","last_unstale":"2023-11-28T13:00:25.452930+0000","last_undegraded":"2023-11-28T13:00:25.452930+0000","last_fullsized":"2023-11-28T13:00:25.452930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:24.765270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.461121+0000","last_change":"2023-11-28T13:00:21.430850+0000","last_active":"2023-11-28T13:00:25.461121+0000","last_peered":"2023-11-28T13:00:25.461121+0000","last_clean":"2023-11-28T13:00:25.461121+0000","last_became_active":"2023-11-28T13:00:21.430557+0000","last_became_peered":"2023-11-28T13:00:21.430557+0000","last_unstale":"2023-11-28T13:00:25.461121+0000","last_undegraded":"2023-11-28T13:00:25.461121+0000","last_fullsized":"2023-11-28T13:00:25.461121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:56:00.080768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442074+0000","last_change":"2023-11-28T13:00:21.437362+0000","last_active":"2023-11-28T13:00:25.442074+0000","last_peered":"2023-11-28T13:00:25.442074+0000","last_clean":"2023-11-28T13:00:25.442074+0000","last_became_active":"2023-11-28T13:00:21.435046+0000","last_became_peered":"2023-11-28T13:00:21.435046+0000","last_unstale":"2023-11-28T13:00:25.442074+0000","last_undegraded":"2023-11-28T13:00:25.442074+0000","last_fullsized":"2023-11-28T13:00:25.442074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:39.731416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448772+0000","last_change":"2023-11-28T13:00:21.436404+0000","last_active":"2023-11-28T13:00:25.448772+0000","last_peered":"2023-11-28T13:00:25.448772+0000","last_clean":"2023-11-28T13:00:25.448772+0000","last_became_active":"2023-11-28T13:00:21.436158+0000","last_became_peered":"2023-11-28T13:00:21.436158+0000","last_unstale":"2023-11-28T13:00:25.448772+0000","last_undegraded":"2023-11-28T13:00:25.448772+0000","last_fullsized":"2023-11-28T13:00:25.448772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:42.706331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460337+0000","last_change":"2023-11-28T13:00:21.431231+0000","last_active":"2023-11-28T13:00:25.460337+0000","last_peered":"2023-11-28T13:00:25.460337+0000","last_clean":"2023-11-28T13:00:25.460337+0000","last_became_active":"2023-11-28T13:00:21.430602+0000","last_became_peered":"2023-11-28T13:00:21.430602+0000","last_unstale":"2023-11-28T13:00:25.460337+0000","last_undegraded":"2023-11-28T13:00:25.460337+0000","last_fullsized":"2023-11-28T13:00:25.460337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:13.072237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443249+0000","last_change":"2023-11-28T13:00:21.419513+0000","last_active":"2023-11-28T13:00:25.443249+0000","last_peered":"2023-11-28T13:00:25.443249+0000","last_clean":"2023-11-28T13:00:25.443249+0000","last_became_active":"2023-11-28T13:00:21.418668+0000","last_became_peered":"2023-11-28T13:00:21.418668+0000","last_unstale":"2023-11-28T13:00:25.443249+0000","last_undegraded":"2023-11-28T13:00:25.443249+0000","last_fullsized":"2023-11-28T13:00:25.443249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448590+0000","last_change":"2023-11-28T13:00:21.432362+0000","last_active":"2023-11-28T13:00:25.448590+0000","last_peered":"2023-11-28T13:00:25.448590+0000","last_clean":"2023-11-28T13:00:25.448590+0000","last_became_active":"2023-11-28T13:00:21.432056+0000","last_became_peered":"2023-11-28T13:00:21.432056+0000","last_unstale":"2023-11-28T13:00:25.448590+0000","last_undegraded":"2023-11-28T13:00:25.448590+0000","last_fullsized":"2023-11-28T13:00:25.448590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:13.039552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448588+0000","last_change":"2023-11-28T13:00:21.434750+0000","last_active":"2023-11-28T13:00:25.448588+0000","last_peered":"2023-11-28T13:00:25.448588+0000","last_clean":"2023-11-28T13:00:25.448588+0000","last_became_active":"2023-11-28T13:00:21.434060+0000","last_became_peered":"2023-11-28T13:00:21.434060+0000","last_unstale":"2023-11-28T13:00:25.448588+0000","last_undegraded":"2023-11-28T13:00:25.448588+0000","last_fullsized":"2023-11-28T13:00:25.448588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:59:34.468496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.453105+0000","last_change":"2023-11-28T13:00:21.433386+0000","last_active":"2023-11-28T13:00:25.453105+0000","last_peered":"2023-11-28T13:00:25.453105+0000","last_clean":"2023-11-28T13:00:25.453105+0000","last_became_active":"2023-11-28T13:00:21.432005+0000","last_became_peered":"2023-11-28T13:00:21.432005+0000","last_unstale":"2023-11-28T13:00:25.453105+0000","last_undegraded":"2023-11-28T13:00:25.453105+0000","last_fullsized":"2023-11-28T13:00:25.453105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:11.589800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.461025+0000","last_change":"2023-11-28T13:00:21.427275+0000","last_active":"2023-11-28T13:00:25.461025+0000","last_peered":"2023-11-28T13:00:25.461025+0000","last_clean":"2023-11-28T13:00:25.461025+0000","last_became_active":"2023-11-28T13:00:21.426893+0000","last_became_peered":"2023-11-28T13:00:21.426893+0000","last_unstale":"2023-11-28T13:00:25.461025+0000","last_undegraded":"2023-11-28T13:00:25.461025+0000","last_fullsized":"2023-11-28T13:00:25.461025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:04:24.765270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455341+0000","last_change":"2023-11-28T13:00:21.436650+0000","last_active":"2023-11-28T13:00:25.455341+0000","last_peered":"2023-11-28T13:00:25.455341+0000","last_clean":"2023-11-28T13:00:25.455341+0000","last_became_active":"2023-11-28T13:00:21.435700+0000","last_became_peered":"2023-11-28T13:00:21.435700+0000","last_unstale":"2023-11-28T13:00:25.455341+0000","last_undegraded":"2023-11-28T13:00:25.455341+0000","last_fullsized":"2023-11-28T13:00:25.455341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:02.740267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.454713+0000","last_change":"2023-11-28T13:00:21.434352+0000","last_active":"2023-11-28T13:00:25.454713+0000","last_peered":"2023-11-28T13:00:25.454713+0000","last_clean":"2023-11-28T13:00:25.454713+0000","last_became_active":"2023-11-28T13:00:21.434067+0000","last_became_peered":"2023-11-28T13:00:21.434067+0000","last_unstale":"2023-11-28T13:00:25.454713+0000","last_undegraded":"2023-11-28T13:00:25.454713+0000","last_fullsized":"2023-11-28T13:00:25.454713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:30.629627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449953+0000","last_change":"2023-11-28T13:00:21.431774+0000","last_active":"2023-11-28T13:00:25.449953+0000","last_peered":"2023-11-28T13:00:25.449953+0000","last_clean":"2023-11-28T13:00:25.449953+0000","last_became_active":"2023-11-28T13:00:21.431445+0000","last_became_peered":"2023-11-28T13:00:21.431445+0000","last_unstale":"2023-11-28T13:00:25.449953+0000","last_undegraded":"2023-11-28T13:00:25.449953+0000","last_fullsized":"2023-11-28T13:00:25.449953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:09:24.029034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443018+0000","last_change":"2023-11-28T13:00:21.420404+0000","last_active":"2023-11-28T13:00:25.443018+0000","last_peered":"2023-11-28T13:00:25.443018+0000","last_clean":"2023-11-28T13:00:25.443018+0000","last_became_active":"2023-11-28T13:00:21.419228+0000","last_became_peered":"2023-11-28T13:00:21.419228+0000","last_unstale":"2023-11-28T13:00:25.443018+0000","last_undegraded":"2023-11-28T13:00:25.443018+0000","last_fullsized":"2023-11-28T13:00:25.443018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:40:24.287015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458890+0000","last_change":"2023-11-28T13:00:21.430943+0000","last_active":"2023-11-28T13:00:25.458890+0000","last_peered":"2023-11-28T13:00:25.458890+0000","last_clean":"2023-11-28T13:00:25.458890+0000","last_became_active":"2023-11-28T13:00:21.430664+0000","last_became_peered":"2023-11-28T13:00:21.430664+0000","last_unstale":"2023-11-28T13:00:25.458890+0000","last_undegraded":"2023-11-28T13:00:25.458890+0000","last_fullsized":"2023-11-28T13:00:25.458890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:12:14.186445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448670+0000","last_change":"2023-11-28T13:00:21.425335+0000","last_active":"2023-11-28T13:00:25.448670+0000","last_peered":"2023-11-28T13:00:25.448670+0000","last_clean":"2023-11-28T13:00:25.448670+0000","last_became_active":"2023-11-28T13:00:21.425073+0000","last_became_peered":"2023-11-28T13:00:21.425073+0000","last_unstale":"2023-11-28T13:00:25.448670+0000","last_undegraded":"2023-11-28T13:00:25.448670+0000","last_fullsized":"2023-11-28T13:00:25.448670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:45.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460831+0000","last_change":"2023-11-28T13:00:21.426161+0000","last_active":"2023-11-28T13:00:25.460831+0000","last_peered":"2023-11-28T13:00:25.460831+0000","last_clean":"2023-11-28T13:00:25.460831+0000","last_became_active":"2023-11-28T13:00:21.425796+0000","last_became_peered":"2023-11-28T13:00:21.425796+0000","last_unstale":"2023-11-28T13:00:25.460831+0000","last_undegraded":"2023-11-28T13:00:25.460831+0000","last_fullsized":"2023-11-28T13:00:25.460831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:17.422401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455990+0000","last_change":"2023-11-28T13:00:21.423565+0000","last_active":"2023-11-28T13:00:25.455990+0000","last_peered":"2023-11-28T13:00:25.455990+0000","last_clean":"2023-11-28T13:00:25.455990+0000","last_became_active":"2023-11-28T13:00:21.423270+0000","last_became_peered":"2023-11-28T13:00:21.423270+0000","last_unstale":"2023-11-28T13:00:25.455990+0000","last_undegraded":"2023-11-28T13:00:25.455990+0000","last_fullsized":"2023-11-28T13:00:25.455990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:31.139532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460193+0000","last_change":"2023-11-28T13:00:21.426452+0000","last_active":"2023-11-28T13:00:25.460193+0000","last_peered":"2023-11-28T13:00:25.460193+0000","last_clean":"2023-11-28T13:00:25.460193+0000","last_became_active":"2023-11-28T13:00:21.426096+0000","last_became_peered":"2023-11-28T13:00:21.426096+0000","last_unstale":"2023-11-28T13:00:25.460193+0000","last_undegraded":"2023-11-28T13:00:25.460193+0000","last_fullsized":"2023-11-28T13:00:25.460193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:33.439707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443668+0000","last_change":"2023-11-28T13:00:21.435483+0000","last_active":"2023-11-28T13:00:25.443668+0000","last_peered":"2023-11-28T13:00:25.443668+0000","last_clean":"2023-11-28T13:00:25.443668+0000","last_became_active":"2023-11-28T13:00:21.435162+0000","last_became_peered":"2023-11-28T13:00:21.435162+0000","last_unstale":"2023-11-28T13:00:25.443668+0000","last_undegraded":"2023-11-28T13:00:25.443668+0000","last_fullsized":"2023-11-28T13:00:25.443668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:47:13.039552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448596+0000","last_change":"2023-11-28T13:00:21.417958+0000","last_active":"2023-11-28T13:00:25.448596+0000","last_peered":"2023-11-28T13:00:25.448596+0000","last_clean":"2023-11-28T13:00:25.448596+0000","last_became_active":"2023-11-28T13:00:21.417664+0000","last_became_peered":"2023-11-28T13:00:21.417664+0000","last_unstale":"2023-11-28T13:00:25.448596+0000","last_undegraded":"2023-11-28T13:00:25.448596+0000","last_fullsized":"2023-11-28T13:00:25.448596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:35.062024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441395+0000","last_change":"2023-11-28T13:00:21.432315+0000","last_active":"2023-11-28T13:00:25.441395+0000","last_peered":"2023-11-28T13:00:25.441395+0000","last_clean":"2023-11-28T13:00:25.441395+0000","last_became_active":"2023-11-28T13:00:21.431878+0000","last_became_peered":"2023-11-28T13:00:21.431878+0000","last_unstale":"2023-11-28T13:00:25.441395+0000","last_undegraded":"2023-11-28T13:00:25.441395+0000","last_fullsized":"2023-11-28T13:00:25.441395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:26:43.602259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446220+0000","last_change":"2023-11-28T13:00:21.429436+0000","last_active":"2023-11-28T13:00:25.446220+0000","last_peered":"2023-11-28T13:00:25.446220+0000","last_clean":"2023-11-28T13:00:25.446220+0000","last_became_active":"2023-11-28T13:00:21.429051+0000","last_became_peered":"2023-11-28T13:00:21.429051+0000","last_unstale":"2023-11-28T13:00:25.446220+0000","last_undegraded":"2023-11-28T13:00:25.446220+0000","last_fullsized":"2023-11-28T13:00:25.446220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:49:11.589800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458565+0000","last_change":"2023-11-28T13:00:21.423512+0000","last_active":"2023-11-28T13:00:25.458565+0000","last_peered":"2023-11-28T13:00:25.458565+0000","last_clean":"2023-11-28T13:00:25.458565+0000","last_became_active":"2023-11-28T13:00:21.423223+0000","last_became_peered":"2023-11-28T13:00:21.423223+0000","last_unstale":"2023-11-28T13:00:25.458565+0000","last_undegraded":"2023-11-28T13:00:25.458565+0000","last_fullsized":"2023-11-28T13:00:25.458565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:21:35.062024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457987+0000","last_change":"2023-11-28T13:00:21.426702+0000","last_active":"2023-11-28T13:00:25.457987+0000","last_peered":"2023-11-28T13:00:25.457987+0000","last_clean":"2023-11-28T13:00:25.457987+0000","last_became_active":"2023-11-28T13:00:21.425807+0000","last_became_peered":"2023-11-28T13:00:21.425807+0000","last_unstale":"2023-11-28T13:00:25.457987+0000","last_undegraded":"2023-11-28T13:00:25.457987+0000","last_fullsized":"2023-11-28T13:00:25.457987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:45:04.077060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452451+0000","last_change":"2023-11-28T13:00:21.434848+0000","last_active":"2023-11-28T13:00:25.452451+0000","last_peered":"2023-11-28T13:00:25.452451+0000","last_clean":"2023-11-28T13:00:25.452451+0000","last_became_active":"2023-11-28T13:00:21.432193+0000","last_became_peered":"2023-11-28T13:00:21.432193+0000","last_unstale":"2023-11-28T13:00:25.452451+0000","last_undegraded":"2023-11-28T13:00:25.452451+0000","last_fullsized":"2023-11-28T13:00:25.452451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:43.321870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871441+0000","last_change":"2023-11-28T13:00:25.470736+0000","last_active":"2023-11-28T13:00:25.871441+0000","last_peered":"2023-11-28T13:00:25.871441+0000","last_clean":"2023-11-28T13:00:25.871441+0000","last_became_active":"2023-11-28T13:00:25.470009+0000","last_became_peered":"2023-11-28T13:00:25.470009+0000","last_unstale":"2023-11-28T13:00:25.871441+0000","last_undegraded":"2023-11-28T13:00:25.871441+0000","last_fullsized":"2023-11-28T13:00:25.871441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:05.706754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459220+0000","last_change":"2023-11-28T13:00:21.006957+0000","last_active":"2023-11-28T13:00:25.459220+0000","last_peered":"2023-11-28T13:00:25.459220+0000","last_clean":"2023-11-28T13:00:25.459220+0000","last_became_active":"2023-11-28T13:00:21.006690+0000","last_became_peered":"2023-11-28T13:00:21.006690+0000","last_unstale":"2023-11-28T13:00:25.459220+0000","last_undegraded":"2023-11-28T13:00:25.459220+0000","last_fullsized":"2023-11-28T13:00:25.459220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:30.128931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441650+0000","last_change":"2023-11-28T13:00:21.430560+0000","last_active":"2023-11-28T13:00:25.441650+0000","last_peered":"2023-11-28T13:00:25.441650+0000","last_clean":"2023-11-28T13:00:25.441650+0000","last_became_active":"2023-11-28T13:00:21.430312+0000","last_became_peered":"2023-11-28T13:00:21.430312+0000","last_unstale":"2023-11-28T13:00:25.441650+0000","last_undegraded":"2023-11-28T13:00:25.441650+0000","last_fullsized":"2023-11-28T13:00:25.441650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347196+0000","last_change":"2023-11-28T13:00:25.460012+0000","last_active":"2023-11-28T13:00:26.347196+0000","last_peered":"2023-11-28T13:00:26.347196+0000","last_clean":"2023-11-28T13:00:26.347196+0000","last_became_active":"2023-11-28T13:00:25.459532+0000","last_became_peered":"2023-11-28T13:00:25.459532+0000","last_unstale":"2023-11-28T13:00:26.347196+0000","last_undegraded":"2023-11-28T13:00:26.347196+0000","last_fullsized":"2023-11-28T13:00:26.347196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:36.890229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448628+0000","last_change":"2023-11-28T13:00:21.005058+0000","last_active":"2023-11-28T13:00:25.448628+0000","last_peered":"2023-11-28T13:00:25.448628+0000","last_clean":"2023-11-28T13:00:25.448628+0000","last_became_active":"2023-11-28T13:00:21.004749+0000","last_became_peered":"2023-11-28T13:00:21.004749+0000","last_unstale":"2023-11-28T13:00:25.448628+0000","last_undegraded":"2023-11-28T13:00:25.448628+0000","last_fullsized":"2023-11-28T13:00:25.448628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:52:22.962275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448569+0000","last_change":"2023-11-28T13:00:21.416645+0000","last_active":"2023-11-28T13:00:25.448569+0000","last_peered":"2023-11-28T13:00:25.448569+0000","last_clean":"2023-11-28T13:00:25.448569+0000","last_became_active":"2023-11-28T13:00:21.416365+0000","last_became_peered":"2023-11-28T13:00:21.416365+0000","last_unstale":"2023-11-28T13:00:25.448569+0000","last_undegraded":"2023-11-28T13:00:25.448569+0000","last_fullsized":"2023-11-28T13:00:25.448569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:19:14.429821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073812+0000","last_change":"2023-11-28T13:00:25.468574+0000","last_active":"2023-11-28T13:00:26.073812+0000","last_peered":"2023-11-28T13:00:26.073812+0000","last_clean":"2023-11-28T13:00:26.073812+0000","last_became_active":"2023-11-28T13:00:25.468177+0000","last_became_peered":"2023-11-28T13:00:25.468177+0000","last_unstale":"2023-11-28T13:00:26.073812+0000","last_undegraded":"2023-11-28T13:00:26.073812+0000","last_fullsized":"2023-11-28T13:00:26.073812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:13:55.892226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441558+0000","last_change":"2023-11-28T13:00:21.005438+0000","last_active":"2023-11-28T13:00:25.441558+0000","last_peered":"2023-11-28T13:00:25.441558+0000","last_clean":"2023-11-28T13:00:25.441558+0000","last_became_active":"2023-11-28T13:00:21.004885+0000","last_became_peered":"2023-11-28T13:00:21.004885+0000","last_unstale":"2023-11-28T13:00:25.441558+0000","last_undegraded":"2023-11-28T13:00:25.441558+0000","last_fullsized":"2023-11-28T13:00:25.441558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:47.845351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442760+0000","last_change":"2023-11-28T13:00:21.437422+0000","last_active":"2023-11-28T13:00:25.442760+0000","last_peered":"2023-11-28T13:00:25.442760+0000","last_clean":"2023-11-28T13:00:25.442760+0000","last_became_active":"2023-11-28T13:00:21.437095+0000","last_became_peered":"2023-11-28T13:00:21.437095+0000","last_unstale":"2023-11-28T13:00:25.442760+0000","last_undegraded":"2023-11-28T13:00:25.442760+0000","last_fullsized":"2023-11-28T13:00:25.442760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:52.972220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346252+0000","last_change":"2023-11-28T13:00:25.468548+0000","last_active":"2023-11-28T13:00:26.346252+0000","last_peered":"2023-11-28T13:00:26.346252+0000","last_clean":"2023-11-28T13:00:26.346252+0000","last_became_active":"2023-11-28T13:00:25.468071+0000","last_became_peered":"2023-11-28T13:00:25.468071+0000","last_unstale":"2023-11-28T13:00:26.346252+0000","last_undegraded":"2023-11-28T13:00:26.346252+0000","last_fullsized":"2023-11-28T13:00:26.346252+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:20.872923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448774+0000","last_change":"2023-11-28T13:00:21.003619+0000","last_active":"2023-11-28T13:00:25.448774+0000","last_peered":"2023-11-28T13:00:25.448774+0000","last_clean":"2023-11-28T13:00:25.448774+0000","last_became_active":"2023-11-28T13:00:21.003189+0000","last_became_peered":"2023-11-28T13:00:21.003189+0000","last_unstale":"2023-11-28T13:00:25.448774+0000","last_undegraded":"2023-11-28T13:00:25.448774+0000","last_fullsized":"2023-11-28T13:00:25.448774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:53.009716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441443+0000","last_change":"2023-11-28T13:00:21.428000+0000","last_active":"2023-11-28T13:00:25.441443+0000","last_peered":"2023-11-28T13:00:25.441443+0000","last_clean":"2023-11-28T13:00:25.441443+0000","last_became_active":"2023-11-28T13:00:21.427738+0000","last_became_peered":"2023-11-28T13:00:21.427738+0000","last_unstale":"2023-11-28T13:00:25.441443+0000","last_undegraded":"2023-11-28T13:00:25.441443+0000","last_fullsized":"2023-11-28T13:00:25.441443+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:08:24.529486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871318+0000","last_change":"2023-11-28T13:00:25.463375+0000","last_active":"2023-11-28T13:00:25.871318+0000","last_peered":"2023-11-28T13:00:25.871318+0000","last_clean":"2023-11-28T13:00:25.871318+0000","last_became_active":"2023-11-28T13:00:25.462962+0000","last_became_peered":"2023-11-28T13:00:25.462962+0000","last_unstale":"2023-11-28T13:00:25.871318+0000","last_undegraded":"2023-11-28T13:00:25.871318+0000","last_fullsized":"2023-11-28T13:00:25.871318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:18:16.522760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.452987+0000","last_change":"2023-11-28T13:00:21.011149+0000","last_active":"2023-11-28T13:00:25.452987+0000","last_peered":"2023-11-28T13:00:25.452987+0000","last_clean":"2023-11-28T13:00:25.452987+0000","last_became_active":"2023-11-28T13:00:21.010813+0000","last_became_peered":"2023-11-28T13:00:21.010813+0000","last_unstale":"2023-11-28T13:00:25.452987+0000","last_undegraded":"2023-11-28T13:00:25.452987+0000","last_fullsized":"2023-11-28T13:00:25.452987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441165+0000","last_change":"2023-11-28T13:00:21.428947+0000","last_active":"2023-11-28T13:00:25.441165+0000","last_peered":"2023-11-28T13:00:25.441165+0000","last_clean":"2023-11-28T13:00:25.441165+0000","last_became_active":"2023-11-28T13:00:21.428690+0000","last_became_peered":"2023-11-28T13:00:21.428690+0000","last_unstale":"2023-11-28T13:00:25.441165+0000","last_undegraded":"2023-11-28T13:00:25.441165+0000","last_fullsized":"2023-11-28T13:00:25.441165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:55:41.240771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869911+0000","last_change":"2023-11-28T13:00:25.468634+0000","last_active":"2023-11-28T13:00:25.869911+0000","last_peered":"2023-11-28T13:00:25.869911+0000","last_clean":"2023-11-28T13:00:25.869911+0000","last_became_active":"2023-11-28T13:00:25.468224+0000","last_became_peered":"2023-11-28T13:00:25.468224+0000","last_unstale":"2023-11-28T13:00:25.869911+0000","last_undegraded":"2023-11-28T13:00:25.869911+0000","last_fullsized":"2023-11-28T13:00:25.869911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:28:21.591804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446244+0000","last_change":"2023-11-28T13:00:21.006053+0000","last_active":"2023-11-28T13:00:25.446244+0000","last_peered":"2023-11-28T13:00:25.446244+0000","last_clean":"2023-11-28T13:00:25.446244+0000","last_became_active":"2023-11-28T13:00:21.005693+0000","last_became_peered":"2023-11-28T13:00:21.005693+0000","last_unstale":"2023-11-28T13:00:25.446244+0000","last_undegraded":"2023-11-28T13:00:25.446244+0000","last_fullsized":"2023-11-28T13:00:25.446244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:38.720815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457494+0000","last_change":"2023-11-28T13:00:21.426158+0000","last_active":"2023-11-28T13:00:25.457494+0000","last_peered":"2023-11-28T13:00:25.457494+0000","last_clean":"2023-11-28T13:00:25.457494+0000","last_became_active":"2023-11-28T13:00:21.425771+0000","last_became_peered":"2023-11-28T13:00:21.425771+0000","last_unstale":"2023-11-28T13:00:25.457494+0000","last_undegraded":"2023-11-28T13:00:25.457494+0000","last_fullsized":"2023-11-28T13:00:25.457494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:21:48.996841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076004+0000","last_change":"2023-11-28T13:00:25.467575+0000","last_active":"2023-11-28T13:00:26.076004+0000","last_peered":"2023-11-28T13:00:26.076004+0000","last_clean":"2023-11-28T13:00:26.076004+0000","last_became_active":"2023-11-28T13:00:25.466169+0000","last_became_peered":"2023-11-28T13:00:25.466169+0000","last_unstale":"2023-11-28T13:00:26.076004+0000","last_undegraded":"2023-11-28T13:00:26.076004+0000","last_fullsized":"2023-11-28T13:00:26.076004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:56:57.102759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:30.521611+0000","last_change":"2023-11-28T13:00:21.000813+0000","last_active":"2023-11-28T13:00:30.521611+0000","last_peered":"2023-11-28T13:00:30.521611+0000","last_clean":"2023-11-28T13:00:30.521611+0000","last_became_active":"2023-11-28T13:00:20.999079+0000","last_became_peered":"2023-11-28T13:00:20.999079+0000","last_unstale":"2023-11-28T13:00:30.521611+0000","last_undegraded":"2023-11-28T13:00:30.521611+0000","last_fullsized":"2023-11-28T13:00:30.521611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:04.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460800+0000","last_change":"2023-11-28T13:00:21.422098+0000","last_active":"2023-11-28T13:00:25.460800+0000","last_peered":"2023-11-28T13:00:25.460800+0000","last_clean":"2023-11-28T13:00:25.460800+0000","last_became_active":"2023-11-28T13:00:21.421774+0000","last_became_peered":"2023-11-28T13:00:21.421774+0000","last_unstale":"2023-11-28T13:00:25.460800+0000","last_undegraded":"2023-11-28T13:00:25.460800+0000","last_fullsized":"2023-11-28T13:00:25.460800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:08.151957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347474+0000","last_change":"2023-11-28T13:00:25.468792+0000","last_active":"2023-11-28T13:00:26.347474+0000","last_peered":"2023-11-28T13:00:26.347474+0000","last_clean":"2023-11-28T13:00:26.347474+0000","last_became_active":"2023-11-28T13:00:25.468376+0000","last_became_peered":"2023-11-28T13:00:25.468376+0000","last_unstale":"2023-11-28T13:00:26.347474+0000","last_undegraded":"2023-11-28T13:00:26.347474+0000","last_fullsized":"2023-11-28T13:00:26.347474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:43:51.024234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443541+0000","last_change":"2023-11-28T13:00:21.003466+0000","last_active":"2023-11-28T13:00:25.443541+0000","last_peered":"2023-11-28T13:00:25.443541+0000","last_clean":"2023-11-28T13:00:25.443541+0000","last_became_active":"2023-11-28T13:00:21.003127+0000","last_became_peered":"2023-11-28T13:00:21.003127+0000","last_unstale":"2023-11-28T13:00:25.443541+0000","last_undegraded":"2023-11-28T13:00:25.443541+0000","last_fullsized":"2023-11-28T13:00:25.443541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441650+0000","last_change":"2023-11-28T13:00:21.429933+0000","last_active":"2023-11-28T13:00:25.441650+0000","last_peered":"2023-11-28T13:00:25.441650+0000","last_clean":"2023-11-28T13:00:25.441650+0000","last_became_active":"2023-11-28T13:00:21.429681+0000","last_became_peered":"2023-11-28T13:00:21.429681+0000","last_unstale":"2023-11-28T13:00:25.441650+0000","last_undegraded":"2023-11-28T13:00:25.441650+0000","last_fullsized":"2023-11-28T13:00:25.441650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:45.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869412+0000","last_change":"2023-11-28T13:00:25.468518+0000","last_active":"2023-11-28T13:00:25.869412+0000","last_peered":"2023-11-28T13:00:25.869412+0000","last_clean":"2023-11-28T13:00:25.869412+0000","last_became_active":"2023-11-28T13:00:25.468083+0000","last_became_peered":"2023-11-28T13:00:25.468083+0000","last_unstale":"2023-11-28T13:00:25.869412+0000","last_undegraded":"2023-11-28T13:00:25.869412+0000","last_fullsized":"2023-11-28T13:00:25.869412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:53:56.156971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448763+0000","last_change":"2023-11-28T13:00:20.997868+0000","last_active":"2023-11-28T13:00:25.448763+0000","last_peered":"2023-11-28T13:00:25.448763+0000","last_clean":"2023-11-28T13:00:25.448763+0000","last_became_active":"2023-11-28T13:00:20.997565+0000","last_became_peered":"2023-11-28T13:00:20.997565+0000","last_unstale":"2023-11-28T13:00:25.448763+0000","last_undegraded":"2023-11-28T13:00:25.448763+0000","last_fullsized":"2023-11-28T13:00:25.448763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:28:37.465748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.451059+0000","last_change":"2023-11-28T13:00:21.431844+0000","last_active":"2023-11-28T13:00:25.451059+0000","last_peered":"2023-11-28T13:00:25.451059+0000","last_clean":"2023-11-28T13:00:25.451059+0000","last_became_active":"2023-11-28T13:00:21.431496+0000","last_became_peered":"2023-11-28T13:00:21.431496+0000","last_unstale":"2023-11-28T13:00:25.451059+0000","last_undegraded":"2023-11-28T13:00:25.451059+0000","last_fullsized":"2023-11-28T13:00:25.451059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.154908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869150+0000","last_change":"2023-11-28T13:00:25.469324+0000","last_active":"2023-11-28T13:00:25.869150+0000","last_peered":"2023-11-28T13:00:25.869150+0000","last_clean":"2023-11-28T13:00:25.869150+0000","last_became_active":"2023-11-28T13:00:25.468890+0000","last_became_peered":"2023-11-28T13:00:25.468890+0000","last_unstale":"2023-11-28T13:00:25.869150+0000","last_undegraded":"2023-11-28T13:00:25.869150+0000","last_fullsized":"2023-11-28T13:00:25.869150+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:41.487196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443130+0000","last_change":"2023-11-28T13:00:21.002295+0000","last_active":"2023-11-28T13:00:25.443130+0000","last_peered":"2023-11-28T13:00:25.443130+0000","last_clean":"2023-11-28T13:00:25.443130+0000","last_became_active":"2023-11-28T13:00:21.001997+0000","last_became_peered":"2023-11-28T13:00:21.001997+0000","last_unstale":"2023-11-28T13:00:25.443130+0000","last_undegraded":"2023-11-28T13:00:25.443130+0000","last_fullsized":"2023-11-28T13:00:25.443130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449818+0000","last_change":"2023-11-28T13:00:21.431066+0000","last_active":"2023-11-28T13:00:25.449818+0000","last_peered":"2023-11-28T13:00:25.449818+0000","last_clean":"2023-11-28T13:00:25.449818+0000","last_became_active":"2023-11-28T13:00:21.430741+0000","last_became_peered":"2023-11-28T13:00:21.430741+0000","last_unstale":"2023-11-28T13:00:25.449818+0000","last_undegraded":"2023-11-28T13:00:25.449818+0000","last_fullsized":"2023-11-28T13:00:25.449818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:55.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347017+0000","last_change":"2023-11-28T13:00:25.468987+0000","last_active":"2023-11-28T13:00:26.347017+0000","last_peered":"2023-11-28T13:00:26.347017+0000","last_clean":"2023-11-28T13:00:26.347017+0000","last_became_active":"2023-11-28T13:00:25.468345+0000","last_became_peered":"2023-11-28T13:00:25.468345+0000","last_unstale":"2023-11-28T13:00:26.347017+0000","last_undegraded":"2023-11-28T13:00:26.347017+0000","last_fullsized":"2023-11-28T13:00:26.347017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:16.201562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441951+0000","last_change":"2023-11-28T13:00:20.993647+0000","last_active":"2023-11-28T13:00:25.441951+0000","last_peered":"2023-11-28T13:00:25.441951+0000","last_clean":"2023-11-28T13:00:25.441951+0000","last_became_active":"2023-11-28T13:00:20.993298+0000","last_became_peered":"2023-11-28T13:00:20.993298+0000","last_unstale":"2023-11-28T13:00:25.441951+0000","last_undegraded":"2023-11-28T13:00:25.441951+0000","last_fullsized":"2023-11-28T13:00:25.441951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:14:48.024917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442632+0000","last_change":"2023-11-28T13:00:21.429127+0000","last_active":"2023-11-28T13:00:25.442632+0000","last_peered":"2023-11-28T13:00:25.442632+0000","last_clean":"2023-11-28T13:00:25.442632+0000","last_became_active":"2023-11-28T13:00:21.428830+0000","last_became_peered":"2023-11-28T13:00:21.428830+0000","last_unstale":"2023-11-28T13:00:25.442632+0000","last_undegraded":"2023-11-28T13:00:25.442632+0000","last_fullsized":"2023-11-28T13:00:25.442632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:55.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871898+0000","last_change":"2023-11-28T13:00:25.464255+0000","last_active":"2023-11-28T13:00:25.871898+0000","last_peered":"2023-11-28T13:00:25.871898+0000","last_clean":"2023-11-28T13:00:25.871898+0000","last_became_active":"2023-11-28T13:00:25.463731+0000","last_became_peered":"2023-11-28T13:00:25.463731+0000","last_unstale":"2023-11-28T13:00:25.871898+0000","last_undegraded":"2023-11-28T13:00:25.871898+0000","last_fullsized":"2023-11-28T13:00:25.871898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:23:35.278662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.447931+0000","last_change":"2023-11-28T13:00:21.002097+0000","last_active":"2023-11-28T13:00:25.447931+0000","last_peered":"2023-11-28T13:00:25.447931+0000","last_clean":"2023-11-28T13:00:25.447931+0000","last_became_active":"2023-11-28T13:00:21.001830+0000","last_became_peered":"2023-11-28T13:00:21.001830+0000","last_unstale":"2023-11-28T13:00:25.447931+0000","last_undegraded":"2023-11-28T13:00:25.447931+0000","last_fullsized":"2023-11-28T13:00:25.447931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:18.850685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448366+0000","last_change":"2023-11-28T13:00:21.426617+0000","last_active":"2023-11-28T13:00:25.448366+0000","last_peered":"2023-11-28T13:00:25.448366+0000","last_clean":"2023-11-28T13:00:25.448366+0000","last_became_active":"2023-11-28T13:00:21.426326+0000","last_became_peered":"2023-11-28T13:00:21.426326+0000","last_unstale":"2023-11-28T13:00:25.448366+0000","last_undegraded":"2023-11-28T13:00:25.448366+0000","last_fullsized":"2023-11-28T13:00:25.448366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231013+0000","last_change":"2023-11-28T13:00:25.476625+0000","last_active":"2023-11-28T13:00:26.231013+0000","last_peered":"2023-11-28T13:00:26.231013+0000","last_clean":"2023-11-28T13:00:26.231013+0000","last_became_active":"2023-11-28T13:00:25.476137+0000","last_became_peered":"2023-11-28T13:00:25.476137+0000","last_unstale":"2023-11-28T13:00:26.231013+0000","last_undegraded":"2023-11-28T13:00:26.231013+0000","last_fullsized":"2023-11-28T13:00:26.231013+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:28.631772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446518+0000","last_change":"2023-11-28T13:00:21.005624+0000","last_active":"2023-11-28T13:00:25.446518+0000","last_peered":"2023-11-28T13:00:25.446518+0000","last_clean":"2023-11-28T13:00:25.446518+0000","last_became_active":"2023-11-28T13:00:21.005092+0000","last_became_peered":"2023-11-28T13:00:21.005092+0000","last_unstale":"2023-11-28T13:00:25.446518+0000","last_undegraded":"2023-11-28T13:00:25.446518+0000","last_fullsized":"2023-11-28T13:00:25.446518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:37.060637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459224+0000","last_change":"2023-11-28T13:00:21.439380+0000","last_active":"2023-11-28T13:00:25.459224+0000","last_peered":"2023-11-28T13:00:25.459224+0000","last_clean":"2023-11-28T13:00:25.459224+0000","last_became_active":"2023-11-28T13:00:21.439094+0000","last_became_peered":"2023-11-28T13:00:21.439094+0000","last_unstale":"2023-11-28T13:00:25.459224+0000","last_undegraded":"2023-11-28T13:00:25.459224+0000","last_fullsized":"2023-11-28T13:00:25.459224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:02.248744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231207+0000","last_change":"2023-11-28T13:00:25.464879+0000","last_active":"2023-11-28T13:00:26.231207+0000","last_peered":"2023-11-28T13:00:26.231207+0000","last_clean":"2023-11-28T13:00:26.231207+0000","last_became_active":"2023-11-28T13:00:25.464486+0000","last_became_peered":"2023-11-28T13:00:25.464486+0000","last_unstale":"2023-11-28T13:00:26.231207+0000","last_undegraded":"2023-11-28T13:00:26.231207+0000","last_fullsized":"2023-11-28T13:00:26.231207+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:39.922810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446121+0000","last_change":"2023-11-28T13:00:21.003453+0000","last_active":"2023-11-28T13:00:25.446121+0000","last_peered":"2023-11-28T13:00:25.446121+0000","last_clean":"2023-11-28T13:00:25.446121+0000","last_became_active":"2023-11-28T13:00:21.003010+0000","last_became_peered":"2023-11-28T13:00:21.003010+0000","last_unstale":"2023-11-28T13:00:25.446121+0000","last_undegraded":"2023-11-28T13:00:25.446121+0000","last_fullsized":"2023-11-28T13:00:25.446121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:50.427721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442176+0000","last_change":"2023-11-28T13:00:21.428448+0000","last_active":"2023-11-28T13:00:25.442176+0000","last_peered":"2023-11-28T13:00:25.442176+0000","last_clean":"2023-11-28T13:00:25.442176+0000","last_became_active":"2023-11-28T13:00:21.427997+0000","last_became_peered":"2023-11-28T13:00:21.427997+0000","last_unstale":"2023-11-28T13:00:25.442176+0000","last_undegraded":"2023-11-28T13:00:25.442176+0000","last_fullsized":"2023-11-28T13:00:25.442176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:41.541085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231241+0000","last_change":"2023-11-28T13:00:25.465904+0000","last_active":"2023-11-28T13:00:26.231241+0000","last_peered":"2023-11-28T13:00:26.231241+0000","last_clean":"2023-11-28T13:00:26.231241+0000","last_became_active":"2023-11-28T13:00:25.465285+0000","last_became_peered":"2023-11-28T13:00:25.465285+0000","last_unstale":"2023-11-28T13:00:26.231241+0000","last_undegraded":"2023-11-28T13:00:26.231241+0000","last_fullsized":"2023-11-28T13:00:26.231241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:46:36.890229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450628+0000","last_change":"2023-11-28T13:00:21.010220+0000","last_active":"2023-11-28T13:00:25.450628+0000","last_peered":"2023-11-28T13:00:25.450628+0000","last_clean":"2023-11-28T13:00:25.450628+0000","last_became_active":"2023-11-28T13:00:21.009892+0000","last_became_peered":"2023-11-28T13:00:21.009892+0000","last_unstale":"2023-11-28T13:00:25.450628+0000","last_undegraded":"2023-11-28T13:00:25.450628+0000","last_fullsized":"2023-11-28T13:00:25.450628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448937+0000","last_change":"2023-11-28T13:00:21.421459+0000","last_active":"2023-11-28T13:00:25.448937+0000","last_peered":"2023-11-28T13:00:25.448937+0000","last_clean":"2023-11-28T13:00:25.448937+0000","last_became_active":"2023-11-28T13:00:21.420822+0000","last_became_peered":"2023-11-28T13:00:21.420822+0000","last_unstale":"2023-11-28T13:00:25.448937+0000","last_undegraded":"2023-11-28T13:00:25.448937+0000","last_fullsized":"2023-11-28T13:00:25.448937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:32:01.515518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231366+0000","last_change":"2023-11-28T13:00:25.464941+0000","last_active":"2023-11-28T13:00:26.231366+0000","last_peered":"2023-11-28T13:00:26.231366+0000","last_clean":"2023-11-28T13:00:26.231366+0000","last_became_active":"2023-11-28T13:00:25.464504+0000","last_became_peered":"2023-11-28T13:00:25.464504+0000","last_unstale":"2023-11-28T13:00:26.231366+0000","last_undegraded":"2023-11-28T13:00:26.231366+0000","last_fullsized":"2023-11-28T13:00:26.231366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:11.602479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441429+0000","last_change":"2023-11-28T13:00:21.008391+0000","last_active":"2023-11-28T13:00:25.441429+0000","last_peered":"2023-11-28T13:00:25.441429+0000","last_clean":"2023-11-28T13:00:25.441429+0000","last_became_active":"2023-11-28T13:00:21.006513+0000","last_became_peered":"2023-11-28T13:00:21.006513+0000","last_unstale":"2023-11-28T13:00:25.441429+0000","last_undegraded":"2023-11-28T13:00:25.441429+0000","last_fullsized":"2023-11-28T13:00:25.441429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:47:58.676237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460081+0000","last_change":"2023-11-28T13:00:21.431780+0000","last_active":"2023-11-28T13:00:25.460081+0000","last_peered":"2023-11-28T13:00:25.460081+0000","last_clean":"2023-11-28T13:00:25.460081+0000","last_became_active":"2023-11-28T13:00:21.430937+0000","last_became_peered":"2023-11-28T13:00:21.430937+0000","last_unstale":"2023-11-28T13:00:25.460081+0000","last_undegraded":"2023-11-28T13:00:25.460081+0000","last_fullsized":"2023-11-28T13:00:25.460081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:12:13.983682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076305+0000","last_change":"2023-11-28T13:00:25.466505+0000","last_active":"2023-11-28T13:00:26.076305+0000","last_peered":"2023-11-28T13:00:26.076305+0000","last_clean":"2023-11-28T13:00:26.076305+0000","last_became_active":"2023-11-28T13:00:25.466059+0000","last_became_peered":"2023-11-28T13:00:25.466059+0000","last_unstale":"2023-11-28T13:00:26.076305+0000","last_undegraded":"2023-11-28T13:00:26.076305+0000","last_fullsized":"2023-11-28T13:00:26.076305+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:01:12.856136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442306+0000","last_change":"2023-11-28T13:00:21.003064+0000","last_active":"2023-11-28T13:00:25.442306+0000","last_peered":"2023-11-28T13:00:25.442306+0000","last_clean":"2023-11-28T13:00:25.442306+0000","last_became_active":"2023-11-28T13:00:21.002742+0000","last_became_peered":"2023-11-28T13:00:21.002742+0000","last_unstale":"2023-11-28T13:00:25.442306+0000","last_undegraded":"2023-11-28T13:00:25.442306+0000","last_fullsized":"2023-11-28T13:00:25.442306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:04.791236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460629+0000","last_change":"2023-11-28T13:00:21.427213+0000","last_active":"2023-11-28T13:00:25.460629+0000","last_peered":"2023-11-28T13:00:25.460629+0000","last_clean":"2023-11-28T13:00:25.460629+0000","last_became_active":"2023-11-28T13:00:21.426737+0000","last_became_peered":"2023-11-28T13:00:21.426737+0000","last_unstale":"2023-11-28T13:00:25.460629+0000","last_undegraded":"2023-11-28T13:00:25.460629+0000","last_fullsized":"2023-11-28T13:00:25.460629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:20:43.321870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347427+0000","last_change":"2023-11-28T13:00:25.466146+0000","last_active":"2023-11-28T13:00:26.347427+0000","last_peered":"2023-11-28T13:00:26.347427+0000","last_clean":"2023-11-28T13:00:26.347427+0000","last_became_active":"2023-11-28T13:00:25.465652+0000","last_became_peered":"2023-11-28T13:00:25.465652+0000","last_unstale":"2023-11-28T13:00:26.347427+0000","last_undegraded":"2023-11-28T13:00:26.347427+0000","last_fullsized":"2023-11-28T13:00:26.347427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:42:14.273148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459071+0000","last_change":"2023-11-28T13:00:21.001745+0000","last_active":"2023-11-28T13:00:25.459071+0000","last_peered":"2023-11-28T13:00:25.459071+0000","last_clean":"2023-11-28T13:00:25.459071+0000","last_became_active":"2023-11-28T13:00:21.001473+0000","last_became_peered":"2023-11-28T13:00:21.001473+0000","last_unstale":"2023-11-28T13:00:25.459071+0000","last_undegraded":"2023-11-28T13:00:25.459071+0000","last_fullsized":"2023-11-28T13:00:25.459071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:46:29.619026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.462841+0000","last_change":"2023-11-28T13:00:21.422938+0000","last_active":"2023-11-28T13:00:25.462841+0000","last_peered":"2023-11-28T13:00:25.462841+0000","last_clean":"2023-11-28T13:00:25.462841+0000","last_became_active":"2023-11-28T13:00:21.422654+0000","last_became_peered":"2023-11-28T13:00:21.422654+0000","last_unstale":"2023-11-28T13:00:25.462841+0000","last_undegraded":"2023-11-28T13:00:25.462841+0000","last_fullsized":"2023-11-28T13:00:25.462841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:28:59.158475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.870965+0000","last_change":"2023-11-28T13:00:25.463080+0000","last_active":"2023-11-28T13:00:25.870965+0000","last_peered":"2023-11-28T13:00:25.870965+0000","last_clean":"2023-11-28T13:00:25.870965+0000","last_became_active":"2023-11-28T13:00:25.462460+0000","last_became_peered":"2023-11-28T13:00:25.462460+0000","last_unstale":"2023-11-28T13:00:25.870965+0000","last_undegraded":"2023-11-28T13:00:25.870965+0000","last_fullsized":"2023-11-28T13:00:25.870965+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:36:45.428316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441326+0000","last_change":"2023-11-28T13:00:21.009988+0000","last_active":"2023-11-28T13:00:25.441326+0000","last_peered":"2023-11-28T13:00:25.441326+0000","last_clean":"2023-11-28T13:00:25.441326+0000","last_became_active":"2023-11-28T13:00:21.009637+0000","last_became_peered":"2023-11-28T13:00:21.009637+0000","last_unstale":"2023-11-28T13:00:25.441326+0000","last_undegraded":"2023-11-28T13:00:25.441326+0000","last_fullsized":"2023-11-28T13:00:25.441326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:03:53.009716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458749+0000","last_change":"2023-11-28T13:00:21.433621+0000","last_active":"2023-11-28T13:00:25.458749+0000","last_peered":"2023-11-28T13:00:25.458749+0000","last_clean":"2023-11-28T13:00:25.458749+0000","last_became_active":"2023-11-28T13:00:21.433293+0000","last_became_peered":"2023-11-28T13:00:21.433293+0000","last_unstale":"2023-11-28T13:00:25.458749+0000","last_undegraded":"2023-11-28T13:00:25.458749+0000","last_fullsized":"2023-11-28T13:00:25.458749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:56:18.248057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.870341+0000","last_change":"2023-11-28T13:00:25.460697+0000","last_active":"2023-11-28T13:00:25.870341+0000","last_peered":"2023-11-28T13:00:25.870341+0000","last_clean":"2023-11-28T13:00:25.870341+0000","last_became_active":"2023-11-28T13:00:25.460230+0000","last_became_peered":"2023-11-28T13:00:25.460230+0000","last_unstale":"2023-11-28T13:00:25.870341+0000","last_undegraded":"2023-11-28T13:00:25.870341+0000","last_fullsized":"2023-11-28T13:00:25.870341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:24:25.897486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442429+0000","last_change":"2023-11-28T13:00:21.001229+0000","last_active":"2023-11-28T13:00:25.442429+0000","last_peered":"2023-11-28T13:00:25.442429+0000","last_clean":"2023-11-28T13:00:25.442429+0000","last_became_active":"2023-11-28T13:00:21.000871+0000","last_became_peered":"2023-11-28T13:00:21.000871+0000","last_unstale":"2023-11-28T13:00:25.442429+0000","last_undegraded":"2023-11-28T13:00:25.442429+0000","last_fullsized":"2023-11-28T13:00:25.442429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:49.179565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460903+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:00:25.460903+0000","last_peered":"2023-11-28T13:00:25.460903+0000","last_clean":"2023-11-28T13:00:25.460903+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:00:25.460903+0000","last_undegraded":"2023-11-28T13:00:25.460903+0000","last_fullsized":"2023-11-28T13:00:25.460903+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441199+0000","last_change":"2023-11-28T13:00:21.429135+0000","last_active":"2023-11-28T13:00:25.441199+0000","last_peered":"2023-11-28T13:00:25.441199+0000","last_clean":"2023-11-28T13:00:25.441199+0000","last_became_active":"2023-11-28T13:00:21.428808+0000","last_became_peered":"2023-11-28T13:00:21.428808+0000","last_unstale":"2023-11-28T13:00:25.441199+0000","last_undegraded":"2023-11-28T13:00:25.441199+0000","last_fullsized":"2023-11-28T13:00:25.441199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:28:36.151382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.868731+0000","last_change":"2023-11-28T13:00:25.472188+0000","last_active":"2023-11-28T13:00:25.868731+0000","last_peered":"2023-11-28T13:00:25.868731+0000","last_clean":"2023-11-28T13:00:25.868731+0000","last_became_active":"2023-11-28T13:00:25.471768+0000","last_became_peered":"2023-11-28T13:00:25.471768+0000","last_unstale":"2023-11-28T13:00:25.868731+0000","last_undegraded":"2023-11-28T13:00:25.868731+0000","last_fullsized":"2023-11-28T13:00:25.868731+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:17:47.454963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441547+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:00:25.441547+0000","last_peered":"2023-11-28T13:00:25.441547+0000","last_clean":"2023-11-28T13:00:25.441547+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:00:25.441547+0000","last_undegraded":"2023-11-28T13:00:25.441547+0000","last_fullsized":"2023-11-28T13:00:25.441547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073655+0000","last_change":"2023-11-28T13:00:20.998799+0000","last_active":"2023-11-28T13:00:26.073655+0000","last_peered":"2023-11-28T13:00:26.073655+0000","last_clean":"2023-11-28T13:00:26.073655+0000","last_became_active":"2023-11-28T13:00:20.998482+0000","last_became_peered":"2023-11-28T13:00:20.998482+0000","last_unstale":"2023-11-28T13:00:26.073655+0000","last_undegraded":"2023-11-28T13:00:26.073655+0000","last_fullsized":"2023-11-28T13:00:26.073655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:28.673649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459183+0000","last_change":"2023-11-28T13:00:21.440454+0000","last_active":"2023-11-28T13:00:25.459183+0000","last_peered":"2023-11-28T13:00:25.459183+0000","last_clean":"2023-11-28T13:00:25.459183+0000","last_became_active":"2023-11-28T13:00:21.439616+0000","last_became_peered":"2023-11-28T13:00:21.439616+0000","last_unstale":"2023-11-28T13:00:25.459183+0000","last_undegraded":"2023-11-28T13:00:25.459183+0000","last_fullsized":"2023-11-28T13:00:25.459183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:36.472288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.347208+0000","last_change":"2023-11-28T13:00:25.468244+0000","last_active":"2023-11-28T13:00:26.347208+0000","last_peered":"2023-11-28T13:00:26.347208+0000","last_clean":"2023-11-28T13:00:26.347208+0000","last_became_active":"2023-11-28T13:00:25.467850+0000","last_became_peered":"2023-11-28T13:00:25.467850+0000","last_unstale":"2023-11-28T13:00:26.347208+0000","last_undegraded":"2023-11-28T13:00:26.347208+0000","last_fullsized":"2023-11-28T13:00:26.347208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:03:47.712713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.456847+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:00:25.456847+0000","last_peered":"2023-11-28T13:00:25.456847+0000","last_clean":"2023-11-28T13:00:25.456847+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:00:25.456847+0000","last_undegraded":"2023-11-28T13:00:25.456847+0000","last_fullsized":"2023-11-28T13:00:25.456847+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446771+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:00:25.446771+0000","last_peered":"2023-11-28T13:00:25.446771+0000","last_clean":"2023-11-28T13:00:25.446771+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:00:25.446771+0000","last_undegraded":"2023-11-28T13:00:25.446771+0000","last_fullsized":"2023-11-28T13:00:25.446771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.447961+0000","last_change":"2023-11-28T13:00:21.005940+0000","last_active":"2023-11-28T13:00:25.447961+0000","last_peered":"2023-11-28T13:00:25.447961+0000","last_clean":"2023-11-28T13:00:25.447961+0000","last_became_active":"2023-11-28T13:00:21.005231+0000","last_became_peered":"2023-11-28T13:00:21.005231+0000","last_unstale":"2023-11-28T13:00:25.447961+0000","last_undegraded":"2023-11-28T13:00:25.447961+0000","last_fullsized":"2023-11-28T13:00:25.447961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:04:49.179565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448989+0000","last_change":"2023-11-28T13:00:21.426195+0000","last_active":"2023-11-28T13:00:25.448989+0000","last_peered":"2023-11-28T13:00:25.448989+0000","last_clean":"2023-11-28T13:00:25.448989+0000","last_became_active":"2023-11-28T13:00:21.423397+0000","last_became_peered":"2023-11-28T13:00:21.423397+0000","last_unstale":"2023-11-28T13:00:25.448989+0000","last_undegraded":"2023-11-28T13:00:25.448989+0000","last_fullsized":"2023-11-28T13:00:25.448989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.231539+0000","last_change":"2023-11-28T13:00:25.468335+0000","last_active":"2023-11-28T13:00:26.231539+0000","last_peered":"2023-11-28T13:00:26.231539+0000","last_clean":"2023-11-28T13:00:26.231539+0000","last_became_active":"2023-11-28T13:00:25.467898+0000","last_became_peered":"2023-11-28T13:00:25.467898+0000","last_unstale":"2023-11-28T13:00:26.231539+0000","last_undegraded":"2023-11-28T13:00:26.231539+0000","last_fullsized":"2023-11-28T13:00:26.231539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:20.872923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.443294+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:00:25.443294+0000","last_peered":"2023-11-28T13:00:25.443294+0000","last_clean":"2023-11-28T13:00:25.443294+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:00:25.443294+0000","last_undegraded":"2023-11-28T13:00:25.443294+0000","last_fullsized":"2023-11-28T13:00:25.443294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460654+0000","last_change":"2023-11-28T13:00:21.007444+0000","last_active":"2023-11-28T13:00:25.460654+0000","last_peered":"2023-11-28T13:00:25.460654+0000","last_clean":"2023-11-28T13:00:25.460654+0000","last_became_active":"2023-11-28T13:00:21.007159+0000","last_became_peered":"2023-11-28T13:00:21.007159+0000","last_unstale":"2023-11-28T13:00:25.460654+0000","last_undegraded":"2023-11-28T13:00:25.460654+0000","last_fullsized":"2023-11-28T13:00:25.460654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:55.415887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458654+0000","last_change":"2023-11-28T13:00:21.437327+0000","last_active":"2023-11-28T13:00:25.458654+0000","last_peered":"2023-11-28T13:00:25.458654+0000","last_clean":"2023-11-28T13:00:25.458654+0000","last_became_active":"2023-11-28T13:00:21.437058+0000","last_became_peered":"2023-11-28T13:00:21.437058+0000","last_unstale":"2023-11-28T13:00:25.458654+0000","last_undegraded":"2023-11-28T13:00:25.458654+0000","last_fullsized":"2023-11-28T13:00:25.458654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:05.426434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871438+0000","last_change":"2023-11-28T13:00:25.462821+0000","last_active":"2023-11-28T13:00:25.871438+0000","last_peered":"2023-11-28T13:00:25.871438+0000","last_clean":"2023-11-28T13:00:25.871438+0000","last_became_active":"2023-11-28T13:00:25.462378+0000","last_became_peered":"2023-11-28T13:00:25.462378+0000","last_unstale":"2023-11-28T13:00:25.871438+0000","last_undegraded":"2023-11-28T13:00:25.871438+0000","last_fullsized":"2023-11-28T13:00:25.871438+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:06:52.582450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.446128+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:00:25.446128+0000","last_peered":"2023-11-28T13:00:25.446128+0000","last_clean":"2023-11-28T13:00:25.446128+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:00:25.446128+0000","last_undegraded":"2023-11-28T13:00:25.446128+0000","last_fullsized":"2023-11-28T13:00:25.446128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441243+0000","last_change":"2023-11-28T13:00:21.005259+0000","last_active":"2023-11-28T13:00:25.441243+0000","last_peered":"2023-11-28T13:00:25.441243+0000","last_clean":"2023-11-28T13:00:25.441243+0000","last_became_active":"2023-11-28T13:00:21.004973+0000","last_became_peered":"2023-11-28T13:00:21.004973+0000","last_unstale":"2023-11-28T13:00:25.441243+0000","last_undegraded":"2023-11-28T13:00:25.441243+0000","last_fullsized":"2023-11-28T13:00:25.441243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:18.754380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442055+0000","last_change":"2023-11-28T13:00:21.428421+0000","last_active":"2023-11-28T13:00:25.442055+0000","last_peered":"2023-11-28T13:00:25.442055+0000","last_clean":"2023-11-28T13:00:25.442055+0000","last_became_active":"2023-11-28T13:00:21.427914+0000","last_became_peered":"2023-11-28T13:00:21.427914+0000","last_unstale":"2023-11-28T13:00:25.442055+0000","last_undegraded":"2023-11-28T13:00:25.442055+0000","last_fullsized":"2023-11-28T13:00:25.442055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:13.464393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073795+0000","last_change":"2023-11-28T13:00:25.468907+0000","last_active":"2023-11-28T13:00:26.073795+0000","last_peered":"2023-11-28T13:00:26.073795+0000","last_clean":"2023-11-28T13:00:26.073795+0000","last_became_active":"2023-11-28T13:00:25.468477+0000","last_became_peered":"2023-11-28T13:00:25.468477+0000","last_unstale":"2023-11-28T13:00:26.073795+0000","last_undegraded":"2023-11-28T13:00:26.073795+0000","last_fullsized":"2023-11-28T13:00:26.073795+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:36:04.014440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448786+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:00:25.448786+0000","last_peered":"2023-11-28T13:00:25.448786+0000","last_clean":"2023-11-28T13:00:25.448786+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:00:25.448786+0000","last_undegraded":"2023-11-28T13:00:25.448786+0000","last_fullsized":"2023-11-28T13:00:25.448786+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460768+0000","last_change":"2023-11-28T13:00:21.008135+0000","last_active":"2023-11-28T13:00:25.460768+0000","last_peered":"2023-11-28T13:00:25.460768+0000","last_clean":"2023-11-28T13:00:25.460768+0000","last_became_active":"2023-11-28T13:00:21.007883+0000","last_became_peered":"2023-11-28T13:00:21.007883+0000","last_unstale":"2023-11-28T13:00:25.460768+0000","last_undegraded":"2023-11-28T13:00:25.460768+0000","last_fullsized":"2023-11-28T13:00:25.460768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:42.423082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455821+0000","last_change":"2023-11-28T13:00:21.425530+0000","last_active":"2023-11-28T13:00:25.455821+0000","last_peered":"2023-11-28T13:00:25.455821+0000","last_clean":"2023-11-28T13:00:25.455821+0000","last_became_active":"2023-11-28T13:00:21.425325+0000","last_became_peered":"2023-11-28T13:00:21.425325+0000","last_unstale":"2023-11-28T13:00:25.455821+0000","last_undegraded":"2023-11-28T13:00:25.455821+0000","last_fullsized":"2023-11-28T13:00:25.455821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:43.401826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346874+0000","last_change":"2023-11-28T13:00:25.457191+0000","last_active":"2023-11-28T13:00:26.346874+0000","last_peered":"2023-11-28T13:00:26.346874+0000","last_clean":"2023-11-28T13:00:26.346874+0000","last_became_active":"2023-11-28T13:00:25.456577+0000","last_became_peered":"2023-11-28T13:00:25.456577+0000","last_unstale":"2023-11-28T13:00:26.346874+0000","last_undegraded":"2023-11-28T13:00:26.346874+0000","last_fullsized":"2023-11-28T13:00:26.346874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:42:11.602479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442371+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:00:25.442371+0000","last_peered":"2023-11-28T13:00:25.442371+0000","last_clean":"2023-11-28T13:00:25.442371+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:00:25.442371+0000","last_undegraded":"2023-11-28T13:00:25.442371+0000","last_fullsized":"2023-11-28T13:00:25.442371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455999+0000","last_change":"2023-11-28T13:00:21.006815+0000","last_active":"2023-11-28T13:00:25.455999+0000","last_peered":"2023-11-28T13:00:25.455999+0000","last_clean":"2023-11-28T13:00:25.455999+0000","last_became_active":"2023-11-28T13:00:21.006608+0000","last_became_peered":"2023-11-28T13:00:21.006608+0000","last_unstale":"2023-11-28T13:00:25.455999+0000","last_undegraded":"2023-11-28T13:00:25.455999+0000","last_fullsized":"2023-11-28T13:00:25.455999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:16:28.673649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448670+0000","last_change":"2023-11-28T13:00:21.421641+0000","last_active":"2023-11-28T13:00:25.448670+0000","last_peered":"2023-11-28T13:00:25.448670+0000","last_clean":"2023-11-28T13:00:25.448670+0000","last_became_active":"2023-11-28T13:00:21.421196+0000","last_became_peered":"2023-11-28T13:00:21.421196+0000","last_unstale":"2023-11-28T13:00:25.448670+0000","last_undegraded":"2023-11-28T13:00:25.448670+0000","last_fullsized":"2023-11-28T13:00:25.448670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:49.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.073549+0000","last_change":"2023-11-28T13:00:25.468640+0000","last_active":"2023-11-28T13:00:26.073549+0000","last_peered":"2023-11-28T13:00:26.073549+0000","last_clean":"2023-11-28T13:00:26.073549+0000","last_became_active":"2023-11-28T13:00:25.467992+0000","last_became_peered":"2023-11-28T13:00:25.467992+0000","last_unstale":"2023-11-28T13:00:26.073549+0000","last_undegraded":"2023-11-28T13:00:26.073549+0000","last_fullsized":"2023-11-28T13:00:26.073549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:45:50.526405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442668+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:00:25.442668+0000","last_peered":"2023-11-28T13:00:25.442668+0000","last_clean":"2023-11-28T13:00:25.442668+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:00:25.442668+0000","last_undegraded":"2023-11-28T13:00:25.442668+0000","last_fullsized":"2023-11-28T13:00:25.442668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450339+0000","last_change":"2023-11-28T13:00:21.010100+0000","last_active":"2023-11-28T13:00:25.450339+0000","last_peered":"2023-11-28T13:00:25.450339+0000","last_clean":"2023-11-28T13:00:25.450339+0000","last_became_active":"2023-11-28T13:00:21.009797+0000","last_became_peered":"2023-11-28T13:00:21.009797+0000","last_unstale":"2023-11-28T13:00:25.450339+0000","last_undegraded":"2023-11-28T13:00:25.450339+0000","last_fullsized":"2023-11-28T13:00:25.450339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:06:18.850685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.457393+0000","last_change":"2023-11-28T13:00:21.433346+0000","last_active":"2023-11-28T13:00:25.457393+0000","last_peered":"2023-11-28T13:00:25.457393+0000","last_clean":"2023-11-28T13:00:25.457393+0000","last_became_active":"2023-11-28T13:00:21.433022+0000","last_became_peered":"2023-11-28T13:00:21.433022+0000","last_unstale":"2023-11-28T13:00:25.457393+0000","last_undegraded":"2023-11-28T13:00:25.457393+0000","last_fullsized":"2023-11-28T13:00:25.457393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:34:14.133792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076291+0000","last_change":"2023-11-28T13:00:25.474463+0000","last_active":"2023-11-28T13:00:26.076291+0000","last_peered":"2023-11-28T13:00:26.076291+0000","last_clean":"2023-11-28T13:00:26.076291+0000","last_became_active":"2023-11-28T13:00:25.473727+0000","last_became_peered":"2023-11-28T13:00:25.473727+0000","last_unstale":"2023-11-28T13:00:26.076291+0000","last_undegraded":"2023-11-28T13:00:26.076291+0000","last_fullsized":"2023-11-28T13:00:26.076291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:35:49.356939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448491+0000","last_change":"2023-11-28T13:00:21.005791+0000","last_active":"2023-11-28T13:00:25.448491+0000","last_peered":"2023-11-28T13:00:25.448491+0000","last_clean":"2023-11-28T13:00:25.448491+0000","last_became_active":"2023-11-28T13:00:21.005474+0000","last_became_peered":"2023-11-28T13:00:21.005474+0000","last_unstale":"2023-11-28T13:00:25.448491+0000","last_undegraded":"2023-11-28T13:00:25.448491+0000","last_fullsized":"2023-11-28T13:00:25.448491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T01:00:14.890428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455579+0000","last_change":"2023-11-28T13:00:21.424133+0000","last_active":"2023-11-28T13:00:25.455579+0000","last_peered":"2023-11-28T13:00:25.455579+0000","last_clean":"2023-11-28T13:00:25.455579+0000","last_became_active":"2023-11-28T13:00:21.423849+0000","last_became_peered":"2023-11-28T13:00:21.423849+0000","last_unstale":"2023-11-28T13:00:25.455579+0000","last_undegraded":"2023-11-28T13:00:25.455579+0000","last_fullsized":"2023-11-28T13:00:25.455579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:05.801837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.871653+0000","last_change":"2023-11-28T13:00:25.468445+0000","last_active":"2023-11-28T13:00:25.871653+0000","last_peered":"2023-11-28T13:00:25.871653+0000","last_clean":"2023-11-28T13:00:25.871653+0000","last_became_active":"2023-11-28T13:00:25.468042+0000","last_became_peered":"2023-11-28T13:00:25.468042+0000","last_unstale":"2023-11-28T13:00:25.871653+0000","last_undegraded":"2023-11-28T13:00:25.871653+0000","last_fullsized":"2023-11-28T13:00:25.871653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.716007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.449852+0000","last_change":"2023-11-28T13:00:21.006819+0000","last_active":"2023-11-28T13:00:25.449852+0000","last_peered":"2023-11-28T13:00:25.449852+0000","last_clean":"2023-11-28T13:00:25.449852+0000","last_became_active":"2023-11-28T13:00:21.006504+0000","last_became_peered":"2023-11-28T13:00:21.006504+0000","last_unstale":"2023-11-28T13:00:25.449852+0000","last_undegraded":"2023-11-28T13:00:25.449852+0000","last_fullsized":"2023-11-28T13:00:25.449852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:39:37.060637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.459182+0000","last_change":"2023-11-28T13:00:21.436419+0000","last_active":"2023-11-28T13:00:25.459182+0000","last_peered":"2023-11-28T13:00:25.459182+0000","last_clean":"2023-11-28T13:00:25.459182+0000","last_became_active":"2023-11-28T13:00:21.436087+0000","last_became_peered":"2023-11-28T13:00:21.436087+0000","last_unstale":"2023-11-28T13:00:25.459182+0000","last_undegraded":"2023-11-28T13:00:25.459182+0000","last_fullsized":"2023-11-28T13:00:25.459182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:18.316015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.346972+0000","last_change":"2023-11-28T13:00:25.464176+0000","last_active":"2023-11-28T13:00:26.346972+0000","last_peered":"2023-11-28T13:00:26.346972+0000","last_clean":"2023-11-28T13:00:26.346972+0000","last_became_active":"2023-11-28T13:00:25.463779+0000","last_became_peered":"2023-11-28T13:00:25.463779+0000","last_unstale":"2023-11-28T13:00:26.346972+0000","last_undegraded":"2023-11-28T13:00:26.346972+0000","last_fullsized":"2023-11-28T13:00:26.346972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:24:15.716007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.450222+0000","last_change":"2023-11-28T13:00:21.006862+0000","last_active":"2023-11-28T13:00:25.450222+0000","last_peered":"2023-11-28T13:00:25.450222+0000","last_clean":"2023-11-28T13:00:25.450222+0000","last_became_active":"2023-11-28T13:00:21.006580+0000","last_became_peered":"2023-11-28T13:00:21.006580+0000","last_unstale":"2023-11-28T13:00:25.450222+0000","last_undegraded":"2023-11-28T13:00:25.450222+0000","last_fullsized":"2023-11-28T13:00:25.450222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:01.729666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460103+0000","last_change":"2023-11-28T13:00:21.427093+0000","last_active":"2023-11-28T13:00:25.460103+0000","last_peered":"2023-11-28T13:00:25.460103+0000","last_clean":"2023-11-28T13:00:25.460103+0000","last_became_active":"2023-11-28T13:00:21.426329+0000","last_became_peered":"2023-11-28T13:00:21.426329+0000","last_unstale":"2023-11-28T13:00:25.460103+0000","last_undegraded":"2023-11-28T13:00:25.460103+0000","last_fullsized":"2023-11-28T13:00:25.460103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:26:04.532312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.869619+0000","last_change":"2023-11-28T13:00:25.475157+0000","last_active":"2023-11-28T13:00:25.869619+0000","last_peered":"2023-11-28T13:00:25.869619+0000","last_clean":"2023-11-28T13:00:25.869619+0000","last_became_active":"2023-11-28T13:00:25.474714+0000","last_became_peered":"2023-11-28T13:00:25.474714+0000","last_unstale":"2023-11-28T13:00:25.869619+0000","last_undegraded":"2023-11-28T13:00:25.869619+0000","last_fullsized":"2023-11-28T13:00:25.869619+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:38:43.383854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441539+0000","last_change":"2023-11-28T13:00:21.004157+0000","last_active":"2023-11-28T13:00:25.441539+0000","last_peered":"2023-11-28T13:00:25.441539+0000","last_clean":"2023-11-28T13:00:25.441539+0000","last_became_active":"2023-11-28T13:00:21.003717+0000","last_became_peered":"2023-11-28T13:00:21.003717+0000","last_unstale":"2023-11-28T13:00:25.441539+0000","last_undegraded":"2023-11-28T13:00:25.441539+0000","last_fullsized":"2023-11-28T13:00:25.441539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:14.599072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441293+0000","last_change":"2023-11-28T13:00:21.427401+0000","last_active":"2023-11-28T13:00:25.441293+0000","last_peered":"2023-11-28T13:00:25.441293+0000","last_clean":"2023-11-28T13:00:25.441293+0000","last_became_active":"2023-11-28T13:00:21.425193+0000","last_became_peered":"2023-11-28T13:00:21.425193+0000","last_unstale":"2023-11-28T13:00:25.441293+0000","last_undegraded":"2023-11-28T13:00:25.441293+0000","last_fullsized":"2023-11-28T13:00:25.441293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:31:16.691765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T13:00:26.076011+0000","last_change":"2023-11-28T13:00:25.466935+0000","last_active":"2023-11-28T13:00:26.076011+0000","last_peered":"2023-11-28T13:00:26.076011+0000","last_clean":"2023-11-28T13:00:26.076011+0000","last_became_active":"2023-11-28T13:00:25.466438+0000","last_became_peered":"2023-11-28T13:00:25.466438+0000","last_unstale":"2023-11-28T13:00:26.076011+0000","last_undegraded":"2023-11-28T13:00:26.076011+0000","last_fullsized":"2023-11-28T13:00:26.076011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:24.426016+0000","last_clean_scrub_stamp":"2023-11-28T13:00:24.426016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:18:44.975678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.448944+0000","last_change":"2023-11-28T13:00:21.003458+0000","last_active":"2023-11-28T13:00:25.448944+0000","last_peered":"2023-11-28T13:00:25.448944+0000","last_clean":"2023-11-28T13:00:25.448944+0000","last_became_active":"2023-11-28T13:00:21.002912+0000","last_became_peered":"2023-11-28T13:00:21.002912+0000","last_unstale":"2023-11-28T13:00:25.448944+0000","last_undegraded":"2023-11-28T13:00:25.448944+0000","last_fullsized":"2023-11-28T13:00:25.448944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:19.964893+0000","last_clean_scrub_stamp":"2023-11-28T13:00:19.964893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:18.754380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.458864+0000","last_change":"2023-11-28T13:00:21.422316+0000","last_active":"2023-11-28T13:00:25.458864+0000","last_peered":"2023-11-28T13:00:25.458864+0000","last_clean":"2023-11-28T13:00:25.458864+0000","last_became_active":"2023-11-28T13:00:21.422012+0000","last_became_peered":"2023-11-28T13:00:21.422012+0000","last_unstale":"2023-11-28T13:00:25.458864+0000","last_undegraded":"2023-11-28T13:00:25.458864+0000","last_fullsized":"2023-11-28T13:00:25.458864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:25.654057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.442053+0000","last_change":"2023-11-28T13:00:21.423769+0000","last_active":"2023-11-28T13:00:25.442053+0000","last_peered":"2023-11-28T13:00:25.442053+0000","last_clean":"2023-11-28T13:00:25.442053+0000","last_became_active":"2023-11-28T13:00:21.423458+0000","last_became_peered":"2023-11-28T13:00:21.423458+0000","last_unstale":"2023-11-28T13:00:25.442053+0000","last_undegraded":"2023-11-28T13:00:25.442053+0000","last_fullsized":"2023-11-28T13:00:25.442053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:37:51.438322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441431+0000","last_change":"2023-11-28T13:00:21.432489+0000","last_active":"2023-11-28T13:00:25.441431+0000","last_peered":"2023-11-28T13:00:25.441431+0000","last_clean":"2023-11-28T13:00:25.441431+0000","last_became_active":"2023-11-28T13:00:21.432224+0000","last_became_peered":"2023-11-28T13:00:21.432224+0000","last_unstale":"2023-11-28T13:00:25.441431+0000","last_undegraded":"2023-11-28T13:00:25.441431+0000","last_fullsized":"2023-11-28T13:00:25.441431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:27.442626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.460221+0000","last_change":"2023-11-28T13:00:21.427335+0000","last_active":"2023-11-28T13:00:25.460221+0000","last_peered":"2023-11-28T13:00:25.460221+0000","last_clean":"2023-11-28T13:00:25.460221+0000","last_became_active":"2023-11-28T13:00:21.426777+0000","last_became_peered":"2023-11-28T13:00:21.426777+0000","last_unstale":"2023-11-28T13:00:25.460221+0000","last_undegraded":"2023-11-28T13:00:25.460221+0000","last_fullsized":"2023-11-28T13:00:25.460221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:24.706933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.441759+0000","last_change":"2023-11-28T13:00:21.436760+0000","last_active":"2023-11-28T13:00:25.441759+0000","last_peered":"2023-11-28T13:00:25.441759+0000","last_clean":"2023-11-28T13:00:25.441759+0000","last_became_active":"2023-11-28T13:00:21.436476+0000","last_became_peered":"2023-11-28T13:00:21.436476+0000","last_unstale":"2023-11-28T13:00:25.441759+0000","last_undegraded":"2023-11-28T13:00:25.441759+0000","last_fullsized":"2023-11-28T13:00:25.441759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:49:49.391583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T13:00:25.455911+0000","last_change":"2023-11-28T13:00:21.415227+0000","last_active":"2023-11-28T13:00:25.455911+0000","last_peered":"2023-11-28T13:00:25.455911+0000","last_clean":"2023-11-28T13:00:25.455911+0000","last_became_active":"2023-11-28T13:00:21.414918+0000","last_became_peered":"2023-11-28T13:00:21.414918+0000","last_unstale":"2023-11-28T13:00:25.455911+0000","last_undegraded":"2023-11-28T13:00:25.455911+0000","last_fullsized":"2023-11-28T13:00:25.455911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:20.975494+0000","last_clean_scrub_stamp":"2023-11-28T13:00:20.975494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:35:41.989664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4528,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":704092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9228,"kb_used_data":836,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362612,"statfs":{"total":96636764160,"available":96627314688,"internally_reserved":0,"allocated":856064,"data_stored":699125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8140,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363700,"statfs":{"total":96636764160,"available":96628428800,"internally_reserved":0,"allocated":266240,"data_stored":109342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0540000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6919999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8084,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363756,"statfs":{"total":96636764160,"available":96628486144,"internally_reserved":0,"allocated":274432,"data_stored":110398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:00:42.510 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T13:00:42.510 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T13:00:42.510 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T13:00:42.511 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:00:42.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.667+0000 7ff9687d7700 1 Processor -- start 2023-11-28T13:00:42.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.667+0000 7ff9687d7700 1 -- start start 2023-11-28T13:00:42.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.668+0000 7ff9687d7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 0x7ff9600ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.668+0000 7ff9687d7700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9600b2460 con 0x7ff9600acba0 2023-11-28T13:00:42.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.668+0000 7ff9687d7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9600acfe0 con 0x7ff96015c8a0 2023-11-28T13:00:42.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.668+0000 7ff9687d7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600d0280 con 0x7ff96006f120 2023-11-28T13:00:42.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.668+0000 7ff966d74700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff9600acba0 0x7ff9600cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45704/0 (socket says 172.21.15.6:45704) 2023-11-28T13:00:42.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.668+0000 7ff966d74700 1 -- 172.21.15.6:0/1562221835 learned_addr learned my addr 172.21.15.6:0/1562221835 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:42.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.668+0000 7ff965d72700 1 --2- 172.21.15.6:0/1562221835 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 0x7ff9600ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.668+0000 7ff965571700 1 -- 172.21.15.6:0/1562221835 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178483627 0 0) 0x7ff9600b2460 con 0x7ff9600acba0 2023-11-28T13:00:42.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.668+0000 7ff965571700 1 -- 172.21.15.6:0/1562221835 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff94c0035b0 con 0x7ff9600acba0 2023-11-28T13:00:42.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.668+0000 7ff965571700 1 -- 172.21.15.6:0/1562221835 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260269210 0 0) 0x7ff9600acfe0 con 0x7ff96015c8a0 2023-11-28T13:00:42.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.669+0000 7ff965571700 1 -- 172.21.15.6:0/1562221835 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9600b2460 con 0x7ff96015c8a0 2023-11-28T13:00:42.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.669+0000 7ff965d72700 1 -- 172.21.15.6:0/1562221835 >> v1:172.21.15.61:6789/0 conn(0x7ff96015c8a0 legacy=0x7ff9600b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.669+0000 7ff965d72700 1 -- 172.21.15.6:0/1562221835 >> v1:172.21.15.6:6789/0 conn(0x7ff9600acba0 legacy=0x7ff9600cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.669+0000 7ff965d72700 1 -- 172.21.15.6:0/1562221835 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9600d0560 con 0x7ff96006f120 2023-11-28T13:00:42.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.669+0000 7ff965d72700 1 --2- 172.21.15.6:0/1562221835 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 0x7ff9600ac660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009870 tx=0x7ff95c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d0f192fa06d52e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.669+0000 7ff965571700 1 -- 172.21.15.6:0/1562221835 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c011830 con 0x7ff96006f120 2023-11-28T13:00:42.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.669+0000 7ff965571700 1 -- 172.21.15.6:0/1562221835 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c011990 con 0x7ff96006f120 2023-11-28T13:00:42.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.669+0000 7ff965571700 1 -- 172.21.15.6:0/1562221835 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c011d20 con 0x7ff96006f120 2023-11-28T13:00:42.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.669+0000 7ff9687d7700 1 -- 172.21.15.6:0/1562221835 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 msgr2=0x7ff9600ac660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.669+0000 7ff9687d7700 1 --2- 172.21.15.6:0/1562221835 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 0x7ff9600ac660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009870 tx=0x7ff95c009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:42.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.670+0000 7ff9687d7700 1 -- 172.21.15.6:0/1562221835 shutdown_connections 2023-11-28T13:00:42.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.670+0000 7ff9687d7700 1 --2- 172.21.15.6:0/1562221835 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 0x7ff9600ac660 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.670+0000 7ff9687d7700 1 -- 172.21.15.6:0/1562221835 >> 172.21.15.6:0/1562221835 conn(0x7ff9600b3610 msgr2=0x7ff9600cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.670+0000 7ff9687d7700 1 -- 172.21.15.6:0/1562221835 shutdown_connections 2023-11-28T13:00:42.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.670+0000 7ff9687d7700 1 -- 172.21.15.6:0/1562221835 wait complete. 2023-11-28T13:00:42.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.671+0000 7ff9687d7700 1 Processor -- start 2023-11-28T13:00:42.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.671+0000 7ff9687d7700 1 -- start start 2023-11-28T13:00:42.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.672+0000 7ff9687d7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 0x7ff9600c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.672+0000 7ff9687d7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9600acba0 0x7ff9600c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.672+0000 7ff9687d7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff96015c8a0 0x7ff960152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.672+0000 7ff9687d7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600c3ab0 con 0x7ff96015c8a0 2023-11-28T13:00:42.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.672+0000 7ff9687d7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600c48f0 con 0x7ff9600acba0 2023-11-28T13:00:42.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.672+0000 7ff9687d7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600c9890 con 0x7ff96006f120 2023-11-28T13:00:42.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.672+0000 7ff966d74700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff96015c8a0 0x7ff960152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.672+0000 7ff966d74700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff96015c8a0 0x7ff960152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58392/0 (socket says 172.21.15.6:58392) 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.672+0000 7ff966d74700 1 -- 172.21.15.6:0/1473289288 learned_addr learned my addr 172.21.15.6:0/1473289288 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.672+0000 7ff966573700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 0x7ff9600c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff965d72700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9600acba0 0x7ff9600c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff966d74700 1 -- 172.21.15.6:0/1473289288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 msgr2=0x7ff9600c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff966d74700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 0x7ff9600c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff966d74700 1 -- 172.21.15.6:0/1473289288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9600acba0 msgr2=0x7ff9600c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff966d74700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9600acba0 0x7ff9600c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff966d74700 1 -- 172.21.15.6:0/1473289288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c0094a0 con 0x7ff96015c8a0 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff966573700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 0x7ff9600c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff966d74700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff96015c8a0 0x7ff960152b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff95800cba0 tx=0x7ff95800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff956ffd700 1 -- 172.21.15.6:0/1473289288 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95800ab20 con 0x7ff96015c8a0 2023-11-28T13:00:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff956ffd700 1 -- 172.21.15.6:0/1473289288 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95800ac80 con 0x7ff96015c8a0 2023-11-28T13:00:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff956ffd700 1 -- 172.21.15.6:0/1473289288 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958012400 con 0x7ff96015c8a0 2023-11-28T13:00:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff9687d7700 1 -- 172.21.15.6:0/1473289288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9600b1550 con 0x7ff96015c8a0 2023-11-28T13:00:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.673+0000 7ff9687d7700 1 -- 172.21.15.6:0/1473289288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9600d0840 con 0x7ff96015c8a0 2023-11-28T13:00:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.674+0000 7ff9687d7700 1 -- 172.21.15.6:0/1473289288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9600b1830 con 0x7ff96015c8a0 2023-11-28T13:00:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.674+0000 7ff956ffd700 1 -- 172.21.15.6:0/1473289288 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff958019030 con 0x7ff96015c8a0 2023-11-28T13:00:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.675+0000 7ff956ffd700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff9440682f0 0x7ff94406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.675+0000 7ff956ffd700 1 -- 172.21.15.6:0/1473289288 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff958085be0 con 0x7ff96015c8a0 2023-11-28T13:00:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.675+0000 7ff966573700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff9440682f0 0x7ff94406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:42.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.675+0000 7ff966573700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff9440682f0 0x7ff94406a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff950007fd0 tx=0x7ff950009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:42.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.679+0000 7ff956ffd700 1 -- 172.21.15.6:0/1473289288 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff958052150 con 0x7ff96015c8a0 2023-11-28T13:00:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.919+0000 7ff9687d7700 1 -- 172.21.15.6:0/1473289288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff9600c4a60 con 0x7ff96015c8a0 2023-11-28T13:00:42.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.919+0000 7ff956ffd700 1 -- 172.21.15.6:0/1473289288 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff958055770 con 0x7ff96015c8a0 2023-11-28T13:00:42.921 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:42.922 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T13:00:42.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 -- 172.21.15.6:0/1473289288 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff9440682f0 msgr2=0x7ff94406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff9440682f0 0x7ff94406a7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff950007fd0 tx=0x7ff950009040 comp rx=0 tx=0).stop 2023-11-28T13:00:42.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 -- 172.21.15.6:0/1473289288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff96015c8a0 msgr2=0x7ff960152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:42.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff96015c8a0 0x7ff960152b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff95800cba0 tx=0x7ff95800cf60 comp rx=0 tx=0).stop 2023-11-28T13:00:42.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 -- 172.21.15.6:0/1473289288 shutdown_connections 2023-11-28T13:00:42.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff96015c8a0 0x7ff960152b50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff9440682f0 0x7ff94406a7a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff96006f120 0x7ff9600c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 --2- 172.21.15.6:0/1473289288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9600acba0 0x7ff9600c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:42.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 -- 172.21.15.6:0/1473289288 >> 172.21.15.6:0/1473289288 conn(0x7ff9600b3610 msgr2=0x7ff9600ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:42.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 -- 172.21.15.6:0/1473289288 shutdown_connections 2023-11-28T13:00:42.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:42.922+0000 7ff9687d7700 1 -- 172.21.15.6:0/1473289288 wait complete. 2023-11-28T13:00:42.935 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T13:00:42.935 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:43.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.059+0000 7f413239d700 1 Processor -- start 2023-11-28T13:00:43.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.059+0000 7f413239d700 1 -- start start 2023-11-28T13:00:43.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.060+0000 7f413239d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c0cefa0 0x7f412c15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.060+0000 7f413239d700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c15e0b0 con 0x7f412c159590 2023-11-28T13:00:43.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.060+0000 7f413239d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c15d120 con 0x7f412c06e400 2023-11-28T13:00:43.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.060+0000 7f413239d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c15d6e0 con 0x7f412c0cefa0 2023-11-28T13:00:43.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.060+0000 7f412bfff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f412c159590 0x7f412c0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45716/0 (socket says 172.21.15.6:45716) 2023-11-28T13:00:43.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.060+0000 7f412bfff700 1 -- 172.21.15.6:0/1485004509 learned_addr learned my addr 172.21.15.6:0/1485004509 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:43.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.060+0000 7f413093a700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c0cefa0 0x7f412c15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.060+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2841478907 0 0) 0x7f411c001110 con 0x7f412c159590 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.060+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41180035b0 con 0x7f412c159590 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119150974 0 0) 0x7f412c15d120 con 0x7f412c06e400 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f411c001110 con 0x7f412c06e400 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2663555536 0 0) 0x7f41180035b0 con 0x7f412c159590 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c0cefa0 msgr2=0x7f412c15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c0cefa0 0x7f412c15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 >> v1:172.21.15.61:6789/0 conn(0x7f412c06e400 legacy=0x7f412c0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c0aef60 con 0x7f412c159590 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f411c001800 con 0x7f412c159590 2023-11-28T13:00:43.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 >> v1:172.21.15.6:6789/0 conn(0x7f412c159590 legacy=0x7f412c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4118003bc0 0x7f4118003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c0cefa0 0x7f4118004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4118004d90 0x7f4118005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c15d6e0 con 0x7f4118003bc0 2023-11-28T13:00:43.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411800baf0 con 0x7f4118004d90 2023-11-28T13:00:43.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411800bcb0 con 0x7f412c0cefa0 2023-11-28T13:00:43.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412bfff700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4118003bc0 0x7f4118003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f413093a700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4118004d90 0x7f4118005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412b7fe700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c0cefa0 0x7f4118004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412bfff700 1 -- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c0cefa0 msgr2=0x7f4118004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412bfff700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c0cefa0 0x7f4118004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412bfff700 1 -- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4118004d90 msgr2=0x7f4118005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412bfff700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4118004d90 0x7f4118005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412bfff700 1 -- 172.21.15.6:0/1485004509 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41180035b0 con 0x7f4118003bc0 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412bfff700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4118003bc0 0x7f4118003fb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c0e0 tx=0x7f4118005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824a64c93b164e4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c00cc80 con 0x7f4118003bc0 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c00ce00 con 0x7f4118003bc0 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.061+0000 7f412affd700 1 -- 172.21.15.6:0/1485004509 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c005ba0 con 0x7f4118003bc0 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.062+0000 7f413239d700 1 -- 172.21.15.6:0/1485004509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4118003bc0 msgr2=0x7f4118003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.062+0000 7f413239d700 1 --2- 172.21.15.6:0/1485004509 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4118003bc0 0x7f4118003fb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f411c00c0e0 tx=0x7f4118005880 comp rx=0 tx=0).stop 2023-11-28T13:00:43.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.062+0000 7f413093a700 1 -- 172.21.15.6:0/1485004509 reap_dead start 2023-11-28T13:00:43.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.062+0000 7f413239d700 1 -- 172.21.15.6:0/1485004509 shutdown_connections 2023-11-28T13:00:43.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.062+0000 7f413239d700 1 -- 172.21.15.6:0/1485004509 >> 172.21.15.6:0/1485004509 conn(0x7f412c0b1650 msgr2=0x7f412c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.062+0000 7f413239d700 1 -- 172.21.15.6:0/1485004509 shutdown_connections 2023-11-28T13:00:43.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.062+0000 7f413239d700 1 -- 172.21.15.6:0/1485004509 wait complete. 2023-11-28T13:00:43.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.064+0000 7f413239d700 1 Processor -- start 2023-11-28T13:00:43.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.064+0000 7f413239d700 1 -- start start 2023-11-28T13:00:43.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f413239d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4118003bc0 0x7f412c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f413239d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f412c0b4580 0x7f412c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f413239d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c152dc0 0x7f412c1489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f413239d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c0b3520 con 0x7f412c0b4580 2023-11-28T13:00:43.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f413239d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c0b37c0 con 0x7f4118003bc0 2023-11-28T13:00:43.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f413239d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c0b3aa0 con 0x7f412c152dc0 2023-11-28T13:00:43.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f412c0b4580 0x7f412c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f412b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f412c0b4580 0x7f412c0cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58410/0 (socket says 172.21.15.6:58410) 2023-11-28T13:00:43.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f412b7fe700 1 -- 172.21.15.6:0/2565780964 learned_addr learned my addr 172.21.15.6:0/2565780964 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:43.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f412bfff700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4118003bc0 0x7f412c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f413093a700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c152dc0 0x7f412c1489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f412b7fe700 1 -- 172.21.15.6:0/2565780964 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c152dc0 msgr2=0x7f412c1489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f412b7fe700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c152dc0 0x7f412c1489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f412b7fe700 1 -- 172.21.15.6:0/2565780964 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4118003bc0 msgr2=0x7f412c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f412b7fe700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4118003bc0 0x7f412c0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f412b7fe700 1 -- 172.21.15.6:0/2565780964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c15d370 con 0x7f412c0b4580 2023-11-28T13:00:43.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.065+0000 7f412b7fe700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f412c0b4580 0x7f412c0cc630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f411400f830 tx=0x7f411400fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.066+0000 7f4128ff9700 1 -- 172.21.15.6:0/2565780964 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114005320 con 0x7f412c0b4580 2023-11-28T13:00:43.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.066+0000 7f4128ff9700 1 -- 172.21.15.6:0/2565780964 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4114005480 con 0x7f412c0b4580 2023-11-28T13:00:43.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.066+0000 7f413239d700 1 -- 172.21.15.6:0/2565780964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412c0af240 con 0x7f412c0b4580 2023-11-28T13:00:43.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.066+0000 7f4128ff9700 1 -- 172.21.15.6:0/2565780964 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114006aa0 con 0x7f412c0b4580 2023-11-28T13:00:43.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.066+0000 7f413239d700 1 -- 172.21.15.6:0/2565780964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412c159960 con 0x7f412c0b4580 2023-11-28T13:00:43.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.067+0000 7f413239d700 1 -- 172.21.15.6:0/2565780964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412c0675a0 con 0x7f412c0b4580 2023-11-28T13:00:43.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.067+0000 7f4128ff9700 1 -- 172.21.15.6:0/2565780964 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4114006c00 con 0x7f412c0b4580 2023-11-28T13:00:43.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.067+0000 7f4128ff9700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f410c0682f0 0x7f410c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.067+0000 7f4128ff9700 1 -- 172.21.15.6:0/2565780964 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4114085780 con 0x7f412c0b4580 2023-11-28T13:00:43.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.067+0000 7f412bfff700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f410c0682f0 0x7f410c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.068+0000 7f412bfff700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f410c0682f0 0x7f410c06a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f411c000c00 tx=0x7f411c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.072+0000 7f4128ff9700 1 -- 172.21.15.6:0/2565780964 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4114051cf0 con 0x7f412c0b4580 2023-11-28T13:00:43.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.264+0000 7f413239d700 1 -- 172.21.15.6:0/2565780964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f412c0ce3e0 con 0x7f412c0b4580 2023-11-28T13:00:43.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.266+0000 7f4128ff9700 1 -- 172.21.15.6:0/2565780964 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f4114055310 con 0x7f412c0b4580 2023-11-28T13:00:43.267 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:43.268 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:43.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.267+0000 7f413239d700 1 -- 172.21.15.6:0/2565780964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f410c0682f0 msgr2=0x7f410c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.267+0000 7f413239d700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f410c0682f0 0x7f410c06a7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f411c000c00 tx=0x7f411c007000 comp rx=0 tx=0).stop 2023-11-28T13:00:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.267+0000 7f413239d700 1 -- 172.21.15.6:0/2565780964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f412c0b4580 msgr2=0x7f412c0cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.267+0000 7f413239d700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f412c0b4580 0x7f412c0cc630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f411400f830 tx=0x7f411400fbf0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.267+0000 7f413239d700 1 -- 172.21.15.6:0/2565780964 shutdown_connections 2023-11-28T13:00:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.267+0000 7f413239d700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f410c0682f0 0x7f410c06a7a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.267+0000 7f413239d700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f412c152dc0 0x7f412c1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.268+0000 7f413239d700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4118003bc0 0x7f412c0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.268+0000 7f413239d700 1 --2- 172.21.15.6:0/2565780964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f412c0b4580 0x7f412c0cc630 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.268+0000 7f413239d700 1 -- 172.21.15.6:0/2565780964 >> 172.21.15.6:0/2565780964 conn(0x7f412c0b1650 msgr2=0x7f412c15bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.268+0000 7f413239d700 1 -- 172.21.15.6:0/2565780964 shutdown_connections 2023-11-28T13:00:43.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.268+0000 7f413239d700 1 -- 172.21.15.6:0/2565780964 wait complete. 2023-11-28T13:00:43.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:00:43.282 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:43.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.407+0000 7fa55010d700 1 Processor -- start 2023-11-28T13:00:43.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.407+0000 7fa55010d700 1 -- start start 2023-11-28T13:00:43.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.408+0000 7fa55010d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa5480b1630 0x7fa5480cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.408+0000 7fa55010d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa54806afc0 0x7fa54806b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.408+0000 7fa55010d700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5480ce550 con 0x7fa5480b1a00 2023-11-28T13:00:43.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.408+0000 7fa55010d700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5480ce830 con 0x7fa5480b1630 2023-11-28T13:00:43.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.408+0000 7fa55010d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5480ceb10 con 0x7fa54806afc0 2023-11-28T13:00:43.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.408+0000 7fa54dea9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa5480b1630 0x7fa5480cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.408+0000 7fa54d6a8700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fa5480b1a00 0x7fa5480caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45738/0 (socket says 172.21.15.6:45738) 2023-11-28T13:00:43.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.408+0000 7fa54dea9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa5480b1630 0x7fa5480cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58416/0 (socket says 172.21.15.6:58416) 2023-11-28T13:00:43.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.408+0000 7fa54d6a8700 1 -- 172.21.15.6:0/750555146 learned_addr learned my addr 172.21.15.6:0/750555146 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:43.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54e6aa700 1 --2- 172.21.15.6:0/750555146 >> v2:172.21.15.61:3300/0 conn(0x7fa54806afc0 0x7fa54806b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54dea9700 1 -- 172.21.15.6:0/750555146 >> v2:172.21.15.61:3300/0 conn(0x7fa54806afc0 msgr2=0x7fa54806b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54dea9700 1 --2- 172.21.15.6:0/750555146 >> v2:172.21.15.61:3300/0 conn(0x7fa54806afc0 0x7fa54806b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54dea9700 1 -- 172.21.15.6:0/750555146 >> v1:172.21.15.6:6789/0 conn(0x7fa5480b1a00 legacy=0x7fa5480caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54cea7700 1 -- 172.21.15.6:0/750555146 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329635821 0 0) 0x7fa5480ce550 con 0x7fa5480b1a00 2023-11-28T13:00:43.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54dea9700 1 -- 172.21.15.6:0/750555146 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480cedf0 con 0x7fa5480b1630 2023-11-28T13:00:43.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54e6aa700 1 --2- 172.21.15.6:0/750555146 >> v2:172.21.15.61:3300/0 conn(0x7fa54806afc0 0x7fa54806b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:43.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54dea9700 1 --2- 172.21.15.6:0/750555146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 0x7fa5480cf5f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa538009840 tx=0x7fa538009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade3c59dd1f9b93b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54cea7700 1 -- 172.21.15.6:0/750555146 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538011830 con 0x7fa5480b1630 2023-11-28T13:00:43.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54cea7700 1 -- 172.21.15.6:0/750555146 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538011990 con 0x7fa5480b1630 2023-11-28T13:00:43.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa54cea7700 1 -- 172.21.15.6:0/750555146 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538011d20 con 0x7fa5480b1630 2023-11-28T13:00:43.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa55010d700 1 -- 172.21.15.6:0/750555146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 msgr2=0x7fa5480cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.409+0000 7fa55010d700 1 --2- 172.21.15.6:0/750555146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 0x7fa5480cf5f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa538009840 tx=0x7fa538009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:43.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.410+0000 7fa55010d700 1 -- 172.21.15.6:0/750555146 shutdown_connections 2023-11-28T13:00:43.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.410+0000 7fa55010d700 1 --2- 172.21.15.6:0/750555146 >> v2:172.21.15.61:3300/0 conn(0x7fa54806afc0 0x7fa54806b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.410+0000 7fa55010d700 1 --2- 172.21.15.6:0/750555146 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 0x7fa5480cf5f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.410+0000 7fa55010d700 1 -- 172.21.15.6:0/750555146 >> 172.21.15.6:0/750555146 conn(0x7fa5480cc630 msgr2=0x7fa5480cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.410+0000 7fa55010d700 1 -- 172.21.15.6:0/750555146 shutdown_connections 2023-11-28T13:00:43.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.410+0000 7fa55010d700 1 -- 172.21.15.6:0/750555146 wait complete. 2023-11-28T13:00:43.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.411+0000 7fa55010d700 1 Processor -- start 2023-11-28T13:00:43.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.411+0000 7fa55010d700 1 -- start start 2023-11-28T13:00:43.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa55010d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa54806afc0 0x7fa54814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa55010d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 0x7fa5481e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa55010d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5480b1a00 0x7fa5480c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa55010d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5481e7a30 con 0x7fa5480b1630 2023-11-28T13:00:43.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa55010d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5481e7190 con 0x7fa54806afc0 2023-11-28T13:00:43.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa55010d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5481e7470 con 0x7fa5480b1a00 2023-11-28T13:00:43.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa54d6a8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 0x7fa5481e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa54d6a8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 0x7fa5481e66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58424/0 (socket says 172.21.15.6:58424) 2023-11-28T13:00:43.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa54d6a8700 1 -- 172.21.15.6:0/2449083982 learned_addr learned my addr 172.21.15.6:0/2449083982 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:43.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa54e6aa700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5480b1a00 0x7fa5480c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.412+0000 7fa54dea9700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa54806afc0 0x7fa54814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa54d6a8700 1 -- 172.21.15.6:0/2449083982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5480b1a00 msgr2=0x7fa5480c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa54d6a8700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5480b1a00 0x7fa5480c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa54d6a8700 1 -- 172.21.15.6:0/2449083982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa54806afc0 msgr2=0x7fa54814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa54d6a8700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa54806afc0 0x7fa54814e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa54d6a8700 1 -- 172.21.15.6:0/2449083982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5380094a0 con 0x7fa5480b1630 2023-11-28T13:00:43.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa54dea9700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa54806afc0 0x7fa54814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:43.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa54d6a8700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 0x7fa5481e66f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c550 tx=0x7fa54400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa53e7fc700 1 -- 172.21.15.6:0/2449083982 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544004c80 con 0x7fa5480b1630 2023-11-28T13:00:43.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa53e7fc700 1 -- 172.21.15.6:0/2449083982 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa544004e00 con 0x7fa5480b1630 2023-11-28T13:00:43.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa55010d700 1 -- 172.21.15.6:0/2449083982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5480b1ea0 con 0x7fa5480b1630 2023-11-28T13:00:43.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa53e7fc700 1 -- 172.21.15.6:0/2449083982 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544007310 con 0x7fa5480b1630 2023-11-28T13:00:43.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.413+0000 7fa55010d700 1 -- 172.21.15.6:0/2449083982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5480cf0d0 con 0x7fa5480b1630 2023-11-28T13:00:43.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.414+0000 7fa55010d700 1 -- 172.21.15.6:0/2449083982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5480cb5b0 con 0x7fa5480b1630 2023-11-28T13:00:43.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.414+0000 7fa53e7fc700 1 -- 172.21.15.6:0/2449083982 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa54401a020 con 0x7fa5480b1630 2023-11-28T13:00:43.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.415+0000 7fa53e7fc700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa52c0682f0 0x7fa52c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.415+0000 7fa53e7fc700 1 -- 172.21.15.6:0/2449083982 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa544085a90 con 0x7fa5480b1630 2023-11-28T13:00:43.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.415+0000 7fa54dea9700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa52c0682f0 0x7fa52c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.415+0000 7fa54dea9700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa52c0682f0 0x7fa52c06a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa5380090b0 tx=0x7fa538008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.419+0000 7fa53e7fc700 1 -- 172.21.15.6:0/2449083982 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa544052000 con 0x7fa5480b1630 2023-11-28T13:00:43.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.610+0000 7fa55010d700 1 -- 172.21.15.6:0/2449083982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa5480739f0 con 0x7fa5480b1630 2023-11-28T13:00:43.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.610+0000 7fa53e7fc700 1 -- 172.21.15.6:0/2449083982 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa544055620 con 0x7fa5480b1630 2023-11-28T13:00:43.612 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:00:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 -- 172.21.15.6:0/2449083982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa52c0682f0 msgr2=0x7fa52c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa52c0682f0 0x7fa52c06a7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa5380090b0 tx=0x7fa538008030 comp rx=0 tx=0).stop 2023-11-28T13:00:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 -- 172.21.15.6:0/2449083982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 msgr2=0x7fa5481e66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 0x7fa5481e66f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa54400c550 tx=0x7fa54400c860 comp rx=0 tx=0).stop 2023-11-28T13:00:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 -- 172.21.15.6:0/2449083982 shutdown_connections 2023-11-28T13:00:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa52c0682f0 0x7fa52c06a7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa5480b1a00 0x7fa5480c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa54806afc0 0x7fa54814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 --2- 172.21.15.6:0/2449083982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5480b1630 0x7fa5481e66f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 -- 172.21.15.6:0/2449083982 >> 172.21.15.6:0/2449083982 conn(0x7fa5480cc630 msgr2=0x7fa54806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 -- 172.21.15.6:0/2449083982 shutdown_connections 2023-11-28T13:00:43.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.612+0000 7fa55010d700 1 -- 172.21.15.6:0/2449083982 wait complete. 2023-11-28T13:00:43.626 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:43.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.753+0000 7fa87e95b700 1 Processor -- start 2023-11-28T13:00:43.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.753+0000 7fa87e95b700 1 -- start start 2023-11-28T13:00:43.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87e95b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8780b1630 0x7fa8780cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87e95b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa87806afc0 0x7fa87806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87e95b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8780ce550 con 0x7fa8780b1a00 2023-11-28T13:00:43.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87e95b700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8780ce830 con 0x7fa87806afc0 2023-11-28T13:00:43.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87e95b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780ceb10 con 0x7fa8780b1630 2023-11-28T13:00:43.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87cef8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa87806afc0 0x7fa87806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87cef8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa87806afc0 0x7fa87806b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58434/0 (socket says 172.21.15.6:58434) 2023-11-28T13:00:43.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87cef8700 1 -- 172.21.15.6:0/3356576600 learned_addr learned my addr 172.21.15.6:0/3356576600 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:43.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa877fff700 1 --2- 172.21.15.6:0/3356576600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8780b1630 0x7fa8780cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87cef8700 1 -- 172.21.15.6:0/3356576600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8780b1630 msgr2=0x7fa8780cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87cef8700 1 --2- 172.21.15.6:0/3356576600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8780b1630 0x7fa8780cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87cef8700 1 -- 172.21.15.6:0/3356576600 >> v1:172.21.15.61:6789/0 conn(0x7fa8780b1a00 legacy=0x7fa8780caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.754+0000 7fa87cef8700 1 -- 172.21.15.6:0/3356576600 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8780cedf0 con 0x7fa87806afc0 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa87cef8700 1 --2- 172.21.15.6:0/3356576600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa87806afc0 0x7fa87806b440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa870009e50 tx=0x7fa87000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae3e0f8c2008b57 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa876ffd700 1 -- 172.21.15.6:0/3356576600 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000e070 con 0x7fa87806afc0 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa876ffd700 1 -- 172.21.15.6:0/3356576600 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87000b910 con 0x7fa87806afc0 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa876ffd700 1 -- 172.21.15.6:0/3356576600 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000bcc0 con 0x7fa87806afc0 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa87e95b700 1 -- 172.21.15.6:0/3356576600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa87806afc0 msgr2=0x7fa87806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa87e95b700 1 --2- 172.21.15.6:0/3356576600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa87806afc0 0x7fa87806b440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa870009e50 tx=0x7fa87000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:43.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa87e95b700 1 -- 172.21.15.6:0/3356576600 shutdown_connections 2023-11-28T13:00:43.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa87e95b700 1 --2- 172.21.15.6:0/3356576600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa87806afc0 0x7fa87806b440 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa87e95b700 1 --2- 172.21.15.6:0/3356576600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8780b1630 0x7fa8780cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa87e95b700 1 -- 172.21.15.6:0/3356576600 >> 172.21.15.6:0/3356576600 conn(0x7fa8780cc630 msgr2=0x7fa8780cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa87e95b700 1 -- 172.21.15.6:0/3356576600 shutdown_connections 2023-11-28T13:00:43.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.755+0000 7fa87e95b700 1 -- 172.21.15.6:0/3356576600 wait complete. 2023-11-28T13:00:43.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.757+0000 7fa87e95b700 1 Processor -- start 2023-11-28T13:00:43.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.757+0000 7fa87e95b700 1 -- start start 2023-11-28T13:00:43.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.757+0000 7fa87e95b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa87806afc0 0x7fa878157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.757+0000 7fa87e95b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8780b1630 0x7fa8780c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.757+0000 7fa87e95b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8780b1a00 0x7fa8780c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.757+0000 7fa87e95b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780c9940 con 0x7fa8780b1630 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.757+0000 7fa87e95b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780c5e90 con 0x7fa87806afc0 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.757+0000 7fa87e95b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780c5280 con 0x7fa8780b1a00 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa8777fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8780b1630 0x7fa8780c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa8777fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8780b1630 0x7fa8780c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58438/0 (socket says 172.21.15.6:58438) 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa8777fe700 1 -- 172.21.15.6:0/484523885 learned_addr learned my addr 172.21.15.6:0/484523885 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa87cef8700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8780b1a00 0x7fa8780c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa877fff700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa87806afc0 0x7fa878157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa8777fe700 1 -- 172.21.15.6:0/484523885 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8780b1a00 msgr2=0x7fa8780c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa8777fe700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8780b1a00 0x7fa8780c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa8777fe700 1 -- 172.21.15.6:0/484523885 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa87806afc0 msgr2=0x7fa878157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa8777fe700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa87806afc0 0x7fa878157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa8777fe700 1 -- 172.21.15.6:0/484523885 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870009b30 con 0x7fa8780b1630 2023-11-28T13:00:43.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa8777fe700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8780b1630 0x7fa8780c7600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00c450 tx=0x7fa86c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa874ff9700 1 -- 172.21.15.6:0/484523885 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0052d0 con 0x7fa8780b1630 2023-11-28T13:00:43.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa874ff9700 1 -- 172.21.15.6:0/484523885 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c005430 con 0x7fa8780b1630 2023-11-28T13:00:43.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa87e95b700 1 -- 172.21.15.6:0/484523885 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8780b1e70 con 0x7fa8780b1630 2023-11-28T13:00:43.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa874ff9700 1 -- 172.21.15.6:0/484523885 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0038c0 con 0x7fa8780b1630 2023-11-28T13:00:43.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.758+0000 7fa87e95b700 1 -- 172.21.15.6:0/484523885 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8780cf0d0 con 0x7fa8780b1630 2023-11-28T13:00:43.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.759+0000 7fa87e95b700 1 -- 172.21.15.6:0/484523885 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8780cb5b0 con 0x7fa8780b1630 2023-11-28T13:00:43.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.760+0000 7fa874ff9700 1 -- 172.21.15.6:0/484523885 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa86c019030 con 0x7fa8780b1630 2023-11-28T13:00:43.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.760+0000 7fa874ff9700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa8640682f0 0x7fa86406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:43.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.760+0000 7fa874ff9700 1 -- 172.21.15.6:0/484523885 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa86c085120 con 0x7fa8780b1630 2023-11-28T13:00:43.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.760+0000 7fa877fff700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa8640682f0 0x7fa86406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:43.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.761+0000 7fa877fff700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa8640682f0 0x7fa86406a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa868009be0 tx=0x7fa868009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:43.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.765+0000 7fa874ff9700 1 -- 172.21.15.6:0/484523885 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa86c051690 con 0x7fa8780b1630 2023-11-28T13:00:43.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.956+0000 7fa87e95b700 1 -- 172.21.15.6:0/484523885 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8780739f0 con 0x7fa8780b1630 2023-11-28T13:00:43.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.956+0000 7fa874ff9700 1 -- 172.21.15.6:0/484523885 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa86c00ce60 con 0x7fa8780b1630 2023-11-28T13:00:43.958 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:00:43.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.958+0000 7fa87e95b700 1 -- 172.21.15.6:0/484523885 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa8640682f0 msgr2=0x7fa86406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.958+0000 7fa87e95b700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa8640682f0 0x7fa86406a7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa868009be0 tx=0x7fa868009210 comp rx=0 tx=0).stop 2023-11-28T13:00:43.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.958+0000 7fa87e95b700 1 -- 172.21.15.6:0/484523885 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8780b1630 msgr2=0x7fa8780c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:43.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.958+0000 7fa87e95b700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8780b1630 0x7fa8780c7600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00c450 tx=0x7fa86c00c760 comp rx=0 tx=0).stop 2023-11-28T13:00:43.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.958+0000 7fa87e95b700 1 -- 172.21.15.6:0/484523885 shutdown_connections 2023-11-28T13:00:43.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.958+0000 7fa87e95b700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa8640682f0 0x7fa86406a7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.958+0000 7fa87e95b700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa8780b1a00 0x7fa8780c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.958+0000 7fa87e95b700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa87806afc0 0x7fa878157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.958+0000 7fa87e95b700 1 --2- 172.21.15.6:0/484523885 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8780b1630 0x7fa8780c7600 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:43.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.958+0000 7fa87e95b700 1 -- 172.21.15.6:0/484523885 >> 172.21.15.6:0/484523885 conn(0x7fa8780cc630 msgr2=0x7fa87815b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:43.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.959+0000 7fa87e95b700 1 -- 172.21.15.6:0/484523885 shutdown_connections 2023-11-28T13:00:43.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:43.959+0000 7fa87e95b700 1 -- 172.21.15.6:0/484523885 wait complete. 2023-11-28T13:00:43.972 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:44.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.099+0000 7fb9c3a16700 1 Processor -- start 2023-11-28T13:00:44.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.099+0000 7fb9c3a16700 1 -- start start 2023-11-28T13:00:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.100+0000 7fb9c3a16700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9bc0b1d40 0x7fb9bc0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.100+0000 7fb9c3a16700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9bc0ceff0 con 0x7fb9bc159500 2023-11-28T13:00:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.100+0000 7fb9c3a16700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9bc0cf2d0 con 0x7fb9bc15dbf0 2023-11-28T13:00:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.100+0000 7fb9c3a16700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc15e090 con 0x7fb9bc0b1d40 2023-11-28T13:00:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.100+0000 7fb9c17b2700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb9bc159500 0x7fb9bc0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45764/0 (socket says 172.21.15.6:45764) 2023-11-28T13:00:44.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.100+0000 7fb9c17b2700 1 -- 172.21.15.6:0/813385930 learned_addr learned my addr 172.21.15.6:0/813385930 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9c0fb1700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9bc0b1d40 0x7fb9bc0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027970009 0 0) 0x7fb9bc0ceff0 con 0x7fb9bc159500 2023-11-28T13:00:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9a80035b0 con 0x7fb9bc159500 2023-11-28T13:00:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999038688 0 0) 0x7fb9bc0cf2d0 con 0x7fb9bc15dbf0 2023-11-28T13:00:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9bc0ceff0 con 0x7fb9bc15dbf0 2023-11-28T13:00:44.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1816477486 0 0) 0x7fb9a80035b0 con 0x7fb9bc159500 2023-11-28T13:00:44.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9bc0b1d40 msgr2=0x7fb9bc0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:44.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9bc0b1d40 0x7fb9bc0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 >> v1:172.21.15.61:6789/0 conn(0x7fb9bc15dbf0 legacy=0x7fb9bc06bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc15d110 con 0x7fb9bc159500 2023-11-28T13:00:44.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fb9ac001670 con 0x7fb9bc159500 2023-11-28T13:00:44.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 >> v1:172.21.15.6:6789/0 conn(0x7fb9bc159500 legacy=0x7fb9bc0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9a8003bc0 0x7fb9a8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a8004850 0x7fb9a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9bc0b1d40 0x7fb9a8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc15e090 con 0x7fb9a8004850 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a800bac0 con 0x7fb9a8003bc0 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a800bc30 con 0x7fb9bc0b1d40 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9c0fb1700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a8004850 0x7fb9a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9c17b2700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9a8003bc0 0x7fb9a8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9c1fb3700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9bc0b1d40 0x7fb9a8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9c0fb1700 1 -- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9bc0b1d40 msgr2=0x7fb9a8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:44.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9c0fb1700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9bc0b1d40 0x7fb9a8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9c0fb1700 1 -- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9a8003bc0 msgr2=0x7fb9a8003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:44.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9c0fb1700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9a8003bc0 0x7fb9a8003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.101+0000 7fb9c0fb1700 1 -- 172.21.15.6:0/813385930 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a80035b0 con 0x7fb9a8004850 2023-11-28T13:00:44.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9c0fb1700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a8004850 0x7fb9a8004cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0cd680 tx=0x7fb9a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa28cc2f308414f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800e6c0 con 0x7fb9a8004850 2023-11-28T13:00:44.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b800e820 con 0x7fb9a8004850 2023-11-28T13:00:44.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9b3fff700 1 -- 172.21.15.6:0/813385930 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800eb90 con 0x7fb9a8004850 2023-11-28T13:00:44.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9c3a16700 1 -- 172.21.15.6:0/813385930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a8004850 msgr2=0x7fb9a8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9c3a16700 1 --2- 172.21.15.6:0/813385930 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9a8004850 0x7fb9a8004cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0cd680 tx=0x7fb9a8005880 comp rx=0 tx=0).stop 2023-11-28T13:00:44.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9c1fb3700 1 -- 172.21.15.6:0/813385930 reap_dead start 2023-11-28T13:00:44.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9c3a16700 1 -- 172.21.15.6:0/813385930 shutdown_connections 2023-11-28T13:00:44.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9c3a16700 1 -- 172.21.15.6:0/813385930 >> 172.21.15.6:0/813385930 conn(0x7fb9bc0cc630 msgr2=0x7fb9bc0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9c3a16700 1 -- 172.21.15.6:0/813385930 shutdown_connections 2023-11-28T13:00:44.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.102+0000 7fb9c3a16700 1 -- 172.21.15.6:0/813385930 wait complete. 2023-11-28T13:00:44.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.104+0000 7fb9c3a16700 1 Processor -- start 2023-11-28T13:00:44.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.104+0000 7fb9c3a16700 1 -- start start 2023-11-28T13:00:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.104+0000 7fb9c3a16700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9a8004850 0x7fb9bc1f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.104+0000 7fb9c3a16700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9bc143480 0x7fb9bc15b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.104+0000 7fb9c3a16700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9bc15b770 0x7fb9bc1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.104+0000 7fb9c3a16700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc154df0 con 0x7fb9bc143480 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.104+0000 7fb9c3a16700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc0c5040 con 0x7fb9bc15b770 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.104+0000 7fb9c3a16700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc1f45b0 con 0x7fb9a8004850 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.104+0000 7fb9c0fb1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9bc143480 0x7fb9bc15b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c0fb1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9bc143480 0x7fb9bc15b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58456/0 (socket says 172.21.15.6:58456) 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c0fb1700 1 -- 172.21.15.6:0/455678572 learned_addr learned my addr 172.21.15.6:0/455678572 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c1fb3700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9bc15b770 0x7fb9bc1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c17b2700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9a8004850 0x7fb9bc1f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c0fb1700 1 -- 172.21.15.6:0/455678572 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9a8004850 msgr2=0x7fb9bc1f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c0fb1700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9a8004850 0x7fb9bc1f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c0fb1700 1 -- 172.21.15.6:0/455678572 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9bc15b770 msgr2=0x7fb9bc1578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c0fb1700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9bc15b770 0x7fb9bc1578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c0fb1700 1 -- 172.21.15.6:0/455678572 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc15d6d0 con 0x7fb9bc143480 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c0fb1700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9bc143480 0x7fb9bc15b230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8000c00 tx=0x7fb9b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9b1ffb700 1 -- 172.21.15.6:0/455678572 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8021070 con 0x7fb9bc143480 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9b1ffb700 1 -- 172.21.15.6:0/455678572 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b8004550 con 0x7fb9bc143480 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9b1ffb700 1 -- 172.21.15.6:0/455678572 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800e6c0 con 0x7fb9bc143480 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c3a16700 1 -- 172.21.15.6:0/455678572 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc15d3f0 con 0x7fb9bc143480 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.105+0000 7fb9c3a16700 1 -- 172.21.15.6:0/455678572 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc1598d0 con 0x7fb9bc143480 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.106+0000 7fb9c3a16700 1 -- 172.21.15.6:0/455678572 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9bc0739f0 con 0x7fb9bc143480 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.107+0000 7fb9b1ffb700 1 -- 172.21.15.6:0/455678572 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb9b802e030 con 0x7fb9bc143480 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.107+0000 7fb9b1ffb700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a00682f0 0x7fb9a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.107+0000 7fb9b1ffb700 1 -- 172.21.15.6:0/455678572 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb9b8087500 con 0x7fb9bc143480 2023-11-28T13:00:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.107+0000 7fb9c17b2700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a00682f0 0x7fb9a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.108+0000 7fb9c17b2700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a00682f0 0x7fb9a006a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac001050 tx=0x7fb9ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.112+0000 7fb9b1ffb700 1 -- 172.21.15.6:0/455678572 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb9b8053980 con 0x7fb9bc143480 2023-11-28T13:00:44.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.307+0000 7fb9c3a16700 1 -- 172.21.15.6:0/455678572 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9bc0b2230 con 0x7fb9bc143480 2023-11-28T13:00:44.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.307+0000 7fb9b1ffb700 1 -- 172.21.15.6:0/455678572 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9b8056fa0 con 0x7fb9bc143480 2023-11-28T13:00:44.309 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:00:44.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 -- 172.21.15.6:0/455678572 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a00682f0 msgr2=0x7fb9a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a00682f0 0x7fb9a006a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac001050 tx=0x7fb9ac009040 comp rx=0 tx=0).stop 2023-11-28T13:00:44.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 -- 172.21.15.6:0/455678572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9bc143480 msgr2=0x7fb9bc15b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9bc143480 0x7fb9bc15b230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8000c00 tx=0x7fb9b80047c0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 -- 172.21.15.6:0/455678572 shutdown_connections 2023-11-28T13:00:44.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9bc15b770 0x7fb9bc1578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9a00682f0 0x7fb9a006a7a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9a8004850 0x7fb9bc1f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 --2- 172.21.15.6:0/455678572 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9bc143480 0x7fb9bc15b230 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 -- 172.21.15.6:0/455678572 >> 172.21.15.6:0/455678572 conn(0x7fb9bc0cc630 msgr2=0x7fb9bc0b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 -- 172.21.15.6:0/455678572 shutdown_connections 2023-11-28T13:00:44.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.310+0000 7fb9c3a16700 1 -- 172.21.15.6:0/455678572 wait complete. 2023-11-28T13:00:44.323 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:44.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.450+0000 7fd58aaff700 1 Processor -- start 2023-11-28T13:00:44.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.450+0000 7fd58aaff700 1 -- start start 2023-11-28T13:00:44.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.450+0000 7fd58aaff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd5840b1630 0x7fd5840cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.450+0000 7fd58aaff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840b1a00 0x7fd5840caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.450+0000 7fd58aaff700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5840ce550 con 0x7fd5840cb3e0 2023-11-28T13:00:44.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.450+0000 7fd58aaff700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5840ce830 con 0x7fd5840b1630 2023-11-28T13:00:44.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.450+0000 7fd58aaff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840ceb10 con 0x7fd5840b1a00 2023-11-28T13:00:44.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd58889b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd5840b1630 0x7fd5840cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd58909c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd5840cb3e0 0x7fd584160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45786/0 (socket says 172.21.15.6:45786) 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd58909c700 1 -- 172.21.15.6:0/1330337186 learned_addr learned my addr 172.21.15.6:0/1330337186 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd583fff700 1 --2- 172.21.15.6:0/1330337186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840b1a00 0x7fd5840caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd58889b700 1 -- 172.21.15.6:0/1330337186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840b1a00 msgr2=0x7fd5840caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd58889b700 1 --2- 172.21.15.6:0/1330337186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840b1a00 0x7fd5840caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd58889b700 1 -- 172.21.15.6:0/1330337186 >> v1:172.21.15.6:6789/0 conn(0x7fd5840cb3e0 legacy=0x7fd584160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd58889b700 1 -- 172.21.15.6:0/1330337186 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840cedf0 con 0x7fd5840b1630 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd583fff700 1 --2- 172.21.15.6:0/1330337186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840b1a00 0x7fd5840caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd58889b700 1 --2- 172.21.15.6:0/1330337186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1630 0x7fd5840cf5f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd578009840 tx=0x7fd578009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7df589c70faa47 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd5837fe700 1 -- 172.21.15.6:0/1330337186 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578011830 con 0x7fd5840b1630 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd5837fe700 1 -- 172.21.15.6:0/1330337186 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd578011990 con 0x7fd5840b1630 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.451+0000 7fd5837fe700 1 -- 172.21.15.6:0/1330337186 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578011d20 con 0x7fd5840b1630 2023-11-28T13:00:44.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.452+0000 7fd58aaff700 1 -- 172.21.15.6:0/1330337186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1630 msgr2=0x7fd5840cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.452+0000 7fd58aaff700 1 --2- 172.21.15.6:0/1330337186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1630 0x7fd5840cf5f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd578009840 tx=0x7fd578009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:44.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.452+0000 7fd58aaff700 1 -- 172.21.15.6:0/1330337186 shutdown_connections 2023-11-28T13:00:44.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.452+0000 7fd58aaff700 1 --2- 172.21.15.6:0/1330337186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1630 0x7fd5840cf5f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.452+0000 7fd58aaff700 1 --2- 172.21.15.6:0/1330337186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840b1a00 0x7fd5840caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.452+0000 7fd58aaff700 1 -- 172.21.15.6:0/1330337186 >> 172.21.15.6:0/1330337186 conn(0x7fd5840cc630 msgr2=0x7fd5840cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.452+0000 7fd58aaff700 1 -- 172.21.15.6:0/1330337186 shutdown_connections 2023-11-28T13:00:44.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.452+0000 7fd58aaff700 1 -- 172.21.15.6:0/1330337186 wait complete. 2023-11-28T13:00:44.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.453+0000 7fd58aaff700 1 Processor -- start 2023-11-28T13:00:44.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.453+0000 7fd58aaff700 1 -- start start 2023-11-28T13:00:44.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd58aaff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5840b1630 0x7fd5840b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd58aaff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1a00 0x7fd5841584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd58aaff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840cb3e0 0x7fd584158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd58aaff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd584158a30 con 0x7fd5840b1a00 2023-11-28T13:00:44.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd58aaff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840b5740 con 0x7fd5840b1630 2023-11-28T13:00:44.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd58aaff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840b3490 con 0x7fd5840cb3e0 2023-11-28T13:00:44.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd583fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1a00 0x7fd5841584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd583fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1a00 0x7fd5841584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58470/0 (socket says 172.21.15.6:58470) 2023-11-28T13:00:44.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd583fff700 1 -- 172.21.15.6:0/1645673837 learned_addr learned my addr 172.21.15.6:0/1645673837 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:44.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd58889b700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5840b1630 0x7fd5840b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd58909c700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840cb3e0 0x7fd584158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd583fff700 1 -- 172.21.15.6:0/1645673837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840cb3e0 msgr2=0x7fd584158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd583fff700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840cb3e0 0x7fd584158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd583fff700 1 -- 172.21.15.6:0/1645673837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5840b1630 msgr2=0x7fd5840b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd583fff700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5840b1630 0x7fd5840b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.454+0000 7fd583fff700 1 -- 172.21.15.6:0/1645673837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780094a0 con 0x7fd5840b1a00 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.455+0000 7fd583fff700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1a00 0x7fd5841584f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c750 tx=0x7fd57000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.455+0000 7fd5817fa700 1 -- 172.21.15.6:0/1645673837 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570011800 con 0x7fd5840b1a00 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.455+0000 7fd5817fa700 1 -- 172.21.15.6:0/1645673837 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570011960 con 0x7fd5840b1a00 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.455+0000 7fd58aaff700 1 -- 172.21.15.6:0/1645673837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5840b1e70 con 0x7fd5840b1a00 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.455+0000 7fd5817fa700 1 -- 172.21.15.6:0/1645673837 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570011cd0 con 0x7fd5840b1a00 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.455+0000 7fd58aaff700 1 -- 172.21.15.6:0/1645673837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5840cf0d0 con 0x7fd5840b1a00 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.456+0000 7fd58aaff700 1 -- 172.21.15.6:0/1645673837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5840ce140 con 0x7fd5840b1a00 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.456+0000 7fd5817fa700 1 -- 172.21.15.6:0/1645673837 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd570020070 con 0x7fd5840b1a00 2023-11-28T13:00:44.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.457+0000 7fd5817fa700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd568068320 0x7fd56806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.457+0000 7fd5817fa700 1 -- 172.21.15.6:0/1645673837 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd570085c10 con 0x7fd5840b1a00 2023-11-28T13:00:44.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.457+0000 7fd58889b700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd568068320 0x7fd56806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.457+0000 7fd58889b700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd568068320 0x7fd56806a7d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd5780090b0 tx=0x7fd578008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.461+0000 7fd5817fa700 1 -- 172.21.15.6:0/1645673837 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd570052180 con 0x7fd5840b1a00 2023-11-28T13:00:44.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.657+0000 7fd58aaff700 1 -- 172.21.15.6:0/1645673837 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5840739f0 con 0x7fd5840b1a00 2023-11-28T13:00:44.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.657+0000 7fd5817fa700 1 -- 172.21.15.6:0/1645673837 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5700557a0 con 0x7fd5840b1a00 2023-11-28T13:00:44.660 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:00:44.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.659+0000 7fd58aaff700 1 -- 172.21.15.6:0/1645673837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd568068320 msgr2=0x7fd56806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.659+0000 7fd58aaff700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd568068320 0x7fd56806a7d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd5780090b0 tx=0x7fd578008030 comp rx=0 tx=0).stop 2023-11-28T13:00:44.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.659+0000 7fd58aaff700 1 -- 172.21.15.6:0/1645673837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1a00 msgr2=0x7fd5841584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.659+0000 7fd58aaff700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1a00 0x7fd5841584f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c750 tx=0x7fd57000ca60 comp rx=0 tx=0).stop 2023-11-28T13:00:44.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.660+0000 7fd58aaff700 1 -- 172.21.15.6:0/1645673837 shutdown_connections 2023-11-28T13:00:44.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.660+0000 7fd58aaff700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd568068320 0x7fd56806a7d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.660+0000 7fd58aaff700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd5840cb3e0 0x7fd584158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.660+0000 7fd58aaff700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5840b1630 0x7fd5840b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.660+0000 7fd58aaff700 1 --2- 172.21.15.6:0/1645673837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd5840b1a00 0x7fd5841584f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.660+0000 7fd58aaff700 1 -- 172.21.15.6:0/1645673837 >> 172.21.15.6:0/1645673837 conn(0x7fd5840cc630 msgr2=0x7fd58406c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.660+0000 7fd58aaff700 1 -- 172.21.15.6:0/1645673837 shutdown_connections 2023-11-28T13:00:44.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.660+0000 7fd58aaff700 1 -- 172.21.15.6:0/1645673837 wait complete. 2023-11-28T13:00:44.673 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:00:44.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.800+0000 7fa43a8f9700 1 Processor -- start 2023-11-28T13:00:44.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.800+0000 7fa43a8f9700 1 -- start start 2023-11-28T13:00:44.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa43a8f9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa4340b1630 0x7fa4340cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa43a8f9700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4340ce550 con 0x7fa4340b1a00 2023-11-28T13:00:44.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa43a8f9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4340ce830 con 0x7fa43406afc0 2023-11-28T13:00:44.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa43a8f9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4340ceb10 con 0x7fa4340b1630 2023-11-28T13:00:44.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa4337fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fa4340b1a00 0x7fa4340caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45800/0 (socket says 172.21.15.6:45800) 2023-11-28T13:00:44.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa4337fe700 1 -- 172.21.15.6:0/4186531293 learned_addr learned my addr 172.21.15.6:0/4186531293 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:44.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa433fff700 1 --2- 172.21.15.6:0/4186531293 >> v2:172.21.15.61:3300/0 conn(0x7fa4340b1630 0x7fa4340cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa432ffd700 1 -- 172.21.15.6:0/4186531293 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172996420 0 0) 0x7fa4340ce550 con 0x7fa4340b1a00 2023-11-28T13:00:44.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa432ffd700 1 -- 172.21.15.6:0/4186531293 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4200035b0 con 0x7fa4340b1a00 2023-11-28T13:00:44.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa432ffd700 1 -- 172.21.15.6:0/4186531293 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043762948 0 0) 0x7fa4340ce830 con 0x7fa43406afc0 2023-11-28T13:00:44.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa432ffd700 1 -- 172.21.15.6:0/4186531293 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4340ce550 con 0x7fa43406afc0 2023-11-28T13:00:44.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa433fff700 1 -- 172.21.15.6:0/4186531293 >> v1:172.21.15.61:6789/0 conn(0x7fa43406afc0 legacy=0x7fa43406b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa433fff700 1 -- 172.21.15.6:0/4186531293 >> v1:172.21.15.6:6789/0 conn(0x7fa4340b1a00 legacy=0x7fa4340caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.801+0000 7fa433fff700 1 -- 172.21.15.6:0/4186531293 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340cedf0 con 0x7fa4340b1630 2023-11-28T13:00:44.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.802+0000 7fa433fff700 1 --2- 172.21.15.6:0/4186531293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4340b1630 0x7fa4340cf5f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa424009870 tx=0x7fa424009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff6e9bb0efa86c4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.802+0000 7fa432ffd700 1 -- 172.21.15.6:0/4186531293 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424011830 con 0x7fa4340b1630 2023-11-28T13:00:44.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.802+0000 7fa432ffd700 1 -- 172.21.15.6:0/4186531293 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa424011990 con 0x7fa4340b1630 2023-11-28T13:00:44.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.802+0000 7fa432ffd700 1 -- 172.21.15.6:0/4186531293 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424011d00 con 0x7fa4340b1630 2023-11-28T13:00:44.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.802+0000 7fa43a8f9700 1 -- 172.21.15.6:0/4186531293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4340b1630 msgr2=0x7fa4340cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.802+0000 7fa43a8f9700 1 --2- 172.21.15.6:0/4186531293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4340b1630 0x7fa4340cf5f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa424009870 tx=0x7fa424009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:44.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.802+0000 7fa43a8f9700 1 -- 172.21.15.6:0/4186531293 shutdown_connections 2023-11-28T13:00:44.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.802+0000 7fa43a8f9700 1 --2- 172.21.15.6:0/4186531293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4340b1630 0x7fa4340cf5f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.802+0000 7fa43a8f9700 1 -- 172.21.15.6:0/4186531293 >> 172.21.15.6:0/4186531293 conn(0x7fa4340cc630 msgr2=0x7fa4340cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:44.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.802+0000 7fa43a8f9700 1 -- 172.21.15.6:0/4186531293 shutdown_connections 2023-11-28T13:00:44.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.803+0000 7fa43a8f9700 1 -- 172.21.15.6:0/4186531293 wait complete. 2023-11-28T13:00:44.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.804+0000 7fa43a8f9700 1 Processor -- start 2023-11-28T13:00:44.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.804+0000 7fa43a8f9700 1 -- start start 2023-11-28T13:00:44.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa43a8f9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43406afc0 0x7fa43415c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa43a8f9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4340b1630 0x7fa43415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa43a8f9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4340b1a00 0x7fa43415b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa43a8f9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43415cfe0 con 0x7fa43406afc0 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa43a8f9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43415da90 con 0x7fa4340b1630 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa43a8f9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43415d1f0 con 0x7fa4340b1a00 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa433fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43406afc0 0x7fa43415c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa433fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43406afc0 0x7fa43415c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58482/0 (socket says 172.21.15.6:58482) 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa433fff700 1 -- 172.21.15.6:0/510278201 learned_addr learned my addr 172.21.15.6:0/510278201 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa4337fe700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4340b1630 0x7fa43415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa438e96700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4340b1a00 0x7fa43415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa433fff700 1 -- 172.21.15.6:0/510278201 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4340b1a00 msgr2=0x7fa43415b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa433fff700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4340b1a00 0x7fa43415b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa433fff700 1 -- 172.21.15.6:0/510278201 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4340b1630 msgr2=0x7fa43415caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:44.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa433fff700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4340b1630 0x7fa43415caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa433fff700 1 -- 172.21.15.6:0/510278201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4240094a0 con 0x7fa43406afc0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa438e96700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4340b1a00 0x7fa43415b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa433fff700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43406afc0 0x7fa43415c0f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa424000c00 tx=0x7fa424004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa430ff9700 1 -- 172.21.15.6:0/510278201 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424004550 con 0x7fa43406afc0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa430ff9700 1 -- 172.21.15.6:0/510278201 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42401f070 con 0x7fa43406afc0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa43a8f9700 1 -- 172.21.15.6:0/510278201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4340b1ea0 con 0x7fa43406afc0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.805+0000 7fa430ff9700 1 -- 172.21.15.6:0/510278201 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4240038e0 con 0x7fa43406afc0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.806+0000 7fa43a8f9700 1 -- 172.21.15.6:0/510278201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4340cf0d0 con 0x7fa43406afc0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.806+0000 7fa43a8f9700 1 -- 172.21.15.6:0/510278201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4340cb5b0 con 0x7fa43406afc0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.807+0000 7fa430ff9700 1 -- 172.21.15.6:0/510278201 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa42400f050 con 0x7fa43406afc0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.807+0000 7fa430ff9700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4140682f0 0x7fa41406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.807+0000 7fa430ff9700 1 -- 172.21.15.6:0/510278201 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa424021070 con 0x7fa43406afc0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.807+0000 7fa4337fe700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4140682f0 0x7fa41406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:44.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.808+0000 7fa4337fe700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4140682f0 0x7fa41406a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa41c007fd0 tx=0x7fa41c007e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:44.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:44.812+0000 7fa430ff9700 1 -- 172.21.15.6:0/510278201 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa424053d70 con 0x7fa43406afc0 2023-11-28T13:00:45.006 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.005+0000 7fa43a8f9700 1 -- 172.21.15.6:0/510278201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa4340739f0 con 0x7fa43406afc0 2023-11-28T13:00:45.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.006+0000 7fa430ff9700 1 -- 172.21.15.6:0/510278201 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa42402b020 con 0x7fa43406afc0 2023-11-28T13:00:45.007 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:00:45.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 -- 172.21.15.6:0/510278201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4140682f0 msgr2=0x7fa41406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4140682f0 0x7fa41406a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa41c007fd0 tx=0x7fa41c007e00 comp rx=0 tx=0).stop 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 -- 172.21.15.6:0/510278201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43406afc0 msgr2=0x7fa43415c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43406afc0 0x7fa43415c0f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa424000c00 tx=0x7fa424004750 comp rx=0 tx=0).stop 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 -- 172.21.15.6:0/510278201 shutdown_connections 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4140682f0 0x7fa41406a7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4340b1a00 0x7fa43415b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43406afc0 0x7fa43415c0f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 --2- 172.21.15.6:0/510278201 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4340b1630 0x7fa43415caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 -- 172.21.15.6:0/510278201 >> 172.21.15.6:0/510278201 conn(0x7fa4340cc630 msgr2=0x7fa43406bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 -- 172.21.15.6:0/510278201 shutdown_connections 2023-11-28T13:00:45.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.008+0000 7fa43a8f9700 1 -- 172.21.15.6:0/510278201 wait complete. 2023-11-28T13:00:45.022 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:00:45.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.147+0000 7faa823d0700 1 Processor -- start 2023-11-28T13:00:45.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.147+0000 7faa823d0700 1 -- start start 2023-11-28T13:00:45.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa823d0700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faa7c071900 0x7faa7c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa823d0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd620 0x7faa7c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa823d0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c0cc860 con 0x7faa7c158ad0 2023-11-28T13:00:45.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa823d0700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c0ccb40 con 0x7faa7c071900 2023-11-28T13:00:45.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa823d0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0cce20 con 0x7faa7c0cd620 2023-11-28T13:00:45.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa7b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faa7c071900 0x7faa7c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa7b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faa7c071900 0x7faa7c15c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58490/0 (socket says 172.21.15.6:58490) 2023-11-28T13:00:45.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa7b7fe700 1 -- 172.21.15.6:0/559696371 learned_addr learned my addr 172.21.15.6:0/559696371 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:45.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa8096d700 1 --2- 172.21.15.6:0/559696371 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd620 0x7faa7c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa7b7fe700 1 -- 172.21.15.6:0/559696371 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd620 msgr2=0x7faa7c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa7b7fe700 1 --2- 172.21.15.6:0/559696371 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd620 0x7faa7c15cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa7b7fe700 1 -- 172.21.15.6:0/559696371 >> v1:172.21.15.61:6789/0 conn(0x7faa7c158ad0 legacy=0x7faa7c0711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa7b7fe700 1 -- 172.21.15.6:0/559696371 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c0cd100 con 0x7faa7c071900 2023-11-28T13:00:45.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.148+0000 7faa8096d700 1 --2- 172.21.15.6:0/559696371 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd620 0x7faa7c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:45.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa7b7fe700 1 --2- 172.21.15.6:0/559696371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c071900 0x7faa7c15c8b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0098f0 tx=0x7faa6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61d4e3a03830259 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa7affd700 1 -- 172.21.15.6:0/559696371 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c004230 con 0x7faa7c071900 2023-11-28T13:00:45.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa7affd700 1 -- 172.21.15.6:0/559696371 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c004390 con 0x7faa7c071900 2023-11-28T13:00:45.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa7affd700 1 -- 172.21.15.6:0/559696371 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0036a0 con 0x7faa7c071900 2023-11-28T13:00:45.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa823d0700 1 -- 172.21.15.6:0/559696371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c071900 msgr2=0x7faa7c15c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa823d0700 1 --2- 172.21.15.6:0/559696371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c071900 0x7faa7c15c8b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0098f0 tx=0x7faa6c009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:45.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa823d0700 1 -- 172.21.15.6:0/559696371 shutdown_connections 2023-11-28T13:00:45.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa823d0700 1 --2- 172.21.15.6:0/559696371 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd620 0x7faa7c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa823d0700 1 --2- 172.21.15.6:0/559696371 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c071900 0x7faa7c15c8b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa823d0700 1 -- 172.21.15.6:0/559696371 >> 172.21.15.6:0/559696371 conn(0x7faa7c0ce9e0 msgr2=0x7faa7c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa823d0700 1 -- 172.21.15.6:0/559696371 shutdown_connections 2023-11-28T13:00:45.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.149+0000 7faa823d0700 1 -- 172.21.15.6:0/559696371 wait complete. 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.151+0000 7faa823d0700 1 Processor -- start 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.151+0000 7faa823d0700 1 -- start start 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa823d0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c071900 0x7faa7c0c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa823d0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c0cd620 0x7faa7c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa823d0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c158ad0 0x7faa7c1491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa823d0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c15b760 con 0x7faa7c158ad0 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa823d0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c163260 con 0x7faa7c0cd620 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa823d0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c1486f0 con 0x7faa7c071900 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa8096d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c158ad0 0x7faa7c1491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa8096d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c158ad0 0x7faa7c1491d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58498/0 (socket says 172.21.15.6:58498) 2023-11-28T13:00:45.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa8096d700 1 -- 172.21.15.6:0/348689802 learned_addr learned my addr 172.21.15.6:0/348689802 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:45.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa7bfff700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c071900 0x7faa7c0c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa7b7fe700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c0cd620 0x7faa7c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa8096d700 1 -- 172.21.15.6:0/348689802 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c071900 msgr2=0x7faa7c0c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa8096d700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c071900 0x7faa7c0c43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa8096d700 1 -- 172.21.15.6:0/348689802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c0cd620 msgr2=0x7faa7c15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa8096d700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c0cd620 0x7faa7c15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa8096d700 1 -- 172.21.15.6:0/348689802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c009550 con 0x7faa7c158ad0 2023-11-28T13:00:45.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.152+0000 7faa8096d700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c158ad0 0x7faa7c1491d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faa7400cef0 tx=0x7faa7400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.153+0000 7faa78ff9700 1 -- 172.21.15.6:0/348689802 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7400ae50 con 0x7faa7c158ad0 2023-11-28T13:00:45.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.153+0000 7faa78ff9700 1 -- 172.21.15.6:0/348689802 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa74015440 con 0x7faa7c158ad0 2023-11-28T13:00:45.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.153+0000 7faa823d0700 1 -- 172.21.15.6:0/348689802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c15d610 con 0x7faa7c158ad0 2023-11-28T13:00:45.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.153+0000 7faa78ff9700 1 -- 172.21.15.6:0/348689802 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa740157f0 con 0x7faa7c158ad0 2023-11-28T13:00:45.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.153+0000 7faa823d0700 1 -- 172.21.15.6:0/348689802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c15d330 con 0x7faa7c158ad0 2023-11-28T13:00:45.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.154+0000 7faa823d0700 1 -- 172.21.15.6:0/348689802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c158ea0 con 0x7faa7c158ad0 2023-11-28T13:00:45.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.154+0000 7faa78ff9700 1 -- 172.21.15.6:0/348689802 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faa7401c030 con 0x7faa7c158ad0 2023-11-28T13:00:45.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.154+0000 7faa78ff9700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 0x7faa5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.155+0000 7faa78ff9700 1 -- 172.21.15.6:0/348689802 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7faa740865d0 con 0x7faa7c158ad0 2023-11-28T13:00:45.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.155+0000 7faa7bfff700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 0x7faa5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.155+0000 7faa7bfff700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 0x7faa5c06a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faa64009870 tx=0x7faa64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.159+0000 7faa78ff9700 1 -- 172.21.15.6:0/348689802 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa740529d0 con 0x7faa7c158ad0 2023-11-28T13:00:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.399+0000 7faa823d0700 1 -- 172.21.15.6:0/348689802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa7c068570 con 0x7faa7c158ad0 2023-11-28T13:00:45.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.400+0000 7faa78ff9700 1 -- 172.21.15.6:0/348689802 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4638 (secure 0 0 0) 0x7faa74055ff0 con 0x7faa7c158ad0 2023-11-28T13:00:45.402 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:45.402 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6834/1568686201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":1568686201},{"type":"v1","addr":"172.21.15.61:6834","nonce":1568686201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/3333487635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6833","nonce":3333487635},{"type":"v1","addr":"172.21.15.61:6835","nonce":3333487635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3917075730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3917075730},{"type":"v1","addr":"172.21.15.6:6835","nonce":3917075730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:21.404170+0000","modified":"2023-11-28T13:00:37.945716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/1668558371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1668558371},{"type":"v1","addr":"172.21.15.6:6837","nonce":1668558371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:00:45.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 -- 172.21.15.6:0/348689802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 msgr2=0x7faa5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 0x7faa5c06a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faa64009870 tx=0x7faa64008040 comp rx=0 tx=0).stop 2023-11-28T13:00:45.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 -- 172.21.15.6:0/348689802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c158ad0 msgr2=0x7faa7c1491d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c158ad0 0x7faa7c1491d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faa7400cef0 tx=0x7faa7400a340 comp rx=0 tx=0).stop 2023-11-28T13:00:45.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 -- 172.21.15.6:0/348689802 shutdown_connections 2023-11-28T13:00:45.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c158ad0 0x7faa7c1491d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 0x7faa5c06a7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c071900 0x7faa7c0c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 --2- 172.21.15.6:0/348689802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c0cd620 0x7faa7c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 -- 172.21.15.6:0/348689802 >> 172.21.15.6:0/348689802 conn(0x7faa7c0ce9e0 msgr2=0x7faa7c06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 -- 172.21.15.6:0/348689802 shutdown_connections 2023-11-28T13:00:45.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.402+0000 7faa823d0700 1 -- 172.21.15.6:0/348689802 wait complete. 2023-11-28T13:00:45.407 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 9 2023-11-28T13:00:45.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:00:21.404170+0000', 'modified': '2023-11-28T13:00:37.945716+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6837/1668558371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6836', 'nonce': 1668558371}, {'type': 'v1', 'addr': '172.21.15.6:6837', 'nonce': 1668558371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:00:45.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:00:45.818 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T13:00:45.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.951+0000 7f100be1e700 1 Processor -- start 2023-11-28T13:00:45.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.951+0000 7f100be1e700 1 -- start start 2023-11-28T13:00:45.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f100be1e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f100415c990 0x7f10040cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f100be1e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1004161090 0x7f10040ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f100be1e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10040ce340 con 0x7f10040caeb0 2023-11-28T13:00:45.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f100be1e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10040cebe0 con 0x7f100415c990 2023-11-28T13:00:45.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f100be1e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040cb3f0 con 0x7f1004161090 2023-11-28T13:00:45.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f1009bba700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f100415c990 0x7f10040cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f1009bba700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f100415c990 0x7f10040cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58502/0 (socket says 172.21.15.6:58502) 2023-11-28T13:00:45.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f1009bba700 1 -- 172.21.15.6:0/92455071 learned_addr learned my addr 172.21.15.6:0/92455071 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:45.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f10093b9700 1 --2- 172.21.15.6:0/92455071 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1004161090 0x7f10040ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f1009bba700 1 -- 172.21.15.6:0/92455071 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1004161090 msgr2=0x7f10040ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f1009bba700 1 --2- 172.21.15.6:0/92455071 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1004161090 0x7f10040ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f1009bba700 1 -- 172.21.15.6:0/92455071 >> v1:172.21.15.61:6789/0 conn(0x7f10040caeb0 legacy=0x7f10040cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.952+0000 7f1009bba700 1 -- 172.21.15.6:0/92455071 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040ce590 con 0x7f100415c990 2023-11-28T13:00:45.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f1009bba700 1 --2- 172.21.15.6:0/92455071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f100415c990 0x7f10040cf070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40098f0 tx=0x7f0ff4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc10b8b36b67d85 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f1008bb8700 1 -- 172.21.15.6:0/92455071 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4011830 con 0x7f100415c990 2023-11-28T13:00:45.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f1008bb8700 1 -- 172.21.15.6:0/92455071 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff4011990 con 0x7f100415c990 2023-11-28T13:00:45.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f1008bb8700 1 -- 172.21.15.6:0/92455071 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4011d20 con 0x7f100415c990 2023-11-28T13:00:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f100be1e700 1 -- 172.21.15.6:0/92455071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f100415c990 msgr2=0x7f10040cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f100be1e700 1 --2- 172.21.15.6:0/92455071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f100415c990 0x7f10040cf070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40098f0 tx=0x7f0ff4009c00 comp rx=0 tx=0).stop 2023-11-28T13:00:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f100be1e700 1 -- 172.21.15.6:0/92455071 shutdown_connections 2023-11-28T13:00:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f100be1e700 1 --2- 172.21.15.6:0/92455071 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f100415c990 0x7f10040cf070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f100be1e700 1 --2- 172.21.15.6:0/92455071 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1004161090 0x7f10040ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f100be1e700 1 -- 172.21.15.6:0/92455071 >> 172.21.15.6:0/92455071 conn(0x7f10040b1650 msgr2=0x7f10040b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f100be1e700 1 -- 172.21.15.6:0/92455071 shutdown_connections 2023-11-28T13:00:45.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.953+0000 7f100be1e700 1 -- 172.21.15.6:0/92455071 wait complete. 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.955+0000 7f100be1e700 1 Processor -- start 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.955+0000 7f100be1e700 1 -- start start 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.955+0000 7f100be1e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10040caeb0 0x7f100415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.955+0000 7f100be1e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f100415c990 0x7f100415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.955+0000 7f100be1e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1004161090 0x7f100415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.955+0000 7f100be1e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040ab8c0 con 0x7f1004161090 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.955+0000 7f100be1e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100415a940 con 0x7f100415c990 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.955+0000 7f100be1e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100406b820 con 0x7f10040caeb0 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100a3bb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1004161090 0x7f100415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100a3bb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1004161090 0x7f100415efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58512/0 (socket says 172.21.15.6:58512) 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100a3bb700 1 -- 172.21.15.6:0/3437725159 learned_addr learned my addr 172.21.15.6:0/3437725159 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:45.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f1009bba700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10040caeb0 0x7f100415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f10093b9700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f100415c990 0x7f100415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100a3bb700 1 -- 172.21.15.6:0/3437725159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10040caeb0 msgr2=0x7f100415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100a3bb700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10040caeb0 0x7f100415ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100a3bb700 1 -- 172.21.15.6:0/3437725159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f100415c990 msgr2=0x7f100415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100a3bb700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f100415c990 0x7f100415a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100a3bb700 1 -- 172.21.15.6:0/3437725159 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff4009550 con 0x7f1004161090 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100a3bb700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1004161090 0x7f100415efc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00cc50 tx=0x7f0ffc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f0ffa7fc700 1 -- 172.21.15.6:0/3437725159 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc003c00 con 0x7f1004161090 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f0ffa7fc700 1 -- 172.21.15.6:0/3437725159 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc003d60 con 0x7f1004161090 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100be1e700 1 -- 172.21.15.6:0/3437725159 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10040cf5b0 con 0x7f1004161090 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f0ffa7fc700 1 -- 172.21.15.6:0/3437725159 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc00aab0 con 0x7f1004161090 2023-11-28T13:00:45.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.956+0000 7f100be1e700 1 -- 172.21.15.6:0/3437725159 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10040ce870 con 0x7f1004161090 2023-11-28T13:00:45.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.958+0000 7f0ffa7fc700 1 -- 172.21.15.6:0/3437725159 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0ffc010050 con 0x7f1004161090 2023-11-28T13:00:45.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.958+0000 7f0ffa7fc700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0fe80682f0 0x7f0fe806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.958+0000 7f1009bba700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0fe80682f0 0x7f0fe806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.958+0000 7f0ffa7fc700 1 -- 172.21.15.6:0/3437725159 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ffc085db0 con 0x7f1004161090 2023-11-28T13:00:45.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.959+0000 7f1009bba700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0fe80682f0 0x7f0fe806a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4000c00 tx=0x7f0ff4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.960+0000 7f1017fa7b80 1 Processor -- start 2023-11-28T13:00:45.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.960+0000 7f1017fa7b80 1 -- start start 2023-11-28T13:00:45.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.961+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fb5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.961+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fdd130 0x560906fdfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.961+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fe0120 0x560906ffb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.961+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906e9fbc0 con 0x560906fdf810 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.961+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906f0e350 con 0x560906fe0120 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.961+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x560906f07880 con 0x560906fdd130 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.961+0000 7f1009bba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fb5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.961+0000 7f1009bba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fb5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58516/0 (socket says 172.21.15.6:58516) 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.961+0000 7f1009bba700 1 -- 172.21.15.6:0/1911674471 learned_addr learned my addr 172.21.15.6:0/1911674471 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f100a3bb700 1 --2- 172.21.15.6:0/1911674471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fe0120 0x560906ffb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f10093b9700 1 --2- 172.21.15.6:0/1911674471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fdd130 0x560906fdfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f1009bba700 1 -- 172.21.15.6:0/1911674471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fdd130 msgr2=0x560906fdfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f1009bba700 1 --2- 172.21.15.6:0/1911674471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fdd130 0x560906fdfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f1009bba700 1 -- 172.21.15.6:0/1911674471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fe0120 msgr2=0x560906ffb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f1009bba700 1 --2- 172.21.15.6:0/1911674471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fe0120 0x560906ffb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f1009bba700 1 -- 172.21.15.6:0/1911674471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560906d895e0 con 0x560906fdf810 2023-11-28T13:00:45.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f1009bba700 1 --2- 172.21.15.6:0/1911674471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fb5a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f100406eac0 tx=0x7f0ff4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4806f0165f0a146 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f0fe77fe700 1 -- 172.21.15.6:0/1911674471 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff402c070 con 0x560906fdf810 2023-11-28T13:00:45.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f0fe77fe700 1 -- 172.21.15.6:0/1911674471 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff40036a0 con 0x560906fdf810 2023-11-28T13:00:45.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f0fe77fe700 1 -- 172.21.15.6:0/1911674471 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4020070 con 0x560906fdf810 2023-11-28T13:00:45.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f1017fa7b80 1 -- 172.21.15.6:0/1911674471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 msgr2=0x560906fb5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.962+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/1911674471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fb5a20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f100406eac0 tx=0x7f0ff4010000 comp rx=0 tx=0).stop 2023-11-28T13:00:45.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.963+0000 7f1017fa7b80 1 -- 172.21.15.6:0/1911674471 shutdown_connections 2023-11-28T13:00:45.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.963+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/1911674471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fe0120 0x560906ffb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.963+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/1911674471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fb5a20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.963+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/1911674471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fdd130 0x560906fdfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.963+0000 7f1017fa7b80 1 -- 172.21.15.6:0/1911674471 >> 172.21.15.6:0/1911674471 conn(0x560906fb2d60 msgr2=0x560906fb36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.963+0000 7f1017fa7b80 1 -- 172.21.15.6:0/1911674471 shutdown_connections 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.963+0000 7f1017fa7b80 1 -- 172.21.15.6:0/1911674471 wait complete. 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.963+0000 7f1017fa7b80 1 -- start start 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.963+0000 7f1017fa7b80 1 Processor -- start 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.964+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.964+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fdf810 0x560906fd1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.964+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 0x560906fd22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.964+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906d8b8d0 con 0x560906fdd130 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.964+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906fd5590 con 0x560906fdf810 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.964+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x560906ff28c0 con 0x560906fe0120 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.964+0000 7f1017fa7b80 10 client.0 _ll_register_callbacks cb 0x560906dad240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:00:45.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.964+0000 7f1009bba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.964+0000 7f1009bba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58522/0 (socket says 172.21.15.6:58522) 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.964+0000 7f1009bba700 1 -- 172.21.15.6:0/650123724 learned_addr learned my addr 172.21.15.6:0/650123724 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f10093b9700 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fdf810 0x560906fd1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f100a3bb700 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 0x560906fd22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1009bba700 1 -- 172.21.15.6:0/650123724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 msgr2=0x560906fd22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1009bba700 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 0x560906fd22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1009bba700 1 -- 172.21.15.6:0/650123724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fdf810 msgr2=0x560906fd1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1009bba700 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fdf810 0x560906fd1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1009bba700 1 -- 172.21.15.6:0/650123724 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10040ce590 con 0x560906fdd130 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1009bba700 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd1210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009f50 tx=0x7f0ff40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e882e1bc2f77221 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f0fe4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/650123724 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4031460 con 0x560906fdd130 2023-11-28T13:00:45.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1017fa7b80 1 -- 172.21.15.6:0/650123724 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560906ed5140 con 0x560906fdd130 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/650123724 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ff40315c0 con 0x560906fdd130 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/650123724 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ff4031780 con 0x560906fdd130 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1017fa7b80 10 client.4773 fetch_fsmap learned FSMap version 9 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1017fa7b80 1 -- 172.21.15.6:0/650123724 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x560906d35d90 con 0x560906fdd130 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/650123724 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560906ed5140 con 0x560906fdd130 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/650123724 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 9) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0ff40318e0 con 0x560906fdd130 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1017fa7b80 10 client.4773 fetch_fsmap finished waiting for FSMap version 9 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1017fa7b80 10 client.4773 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1017fa7b80 20 client.4773 populate_metadata read hostname 'smithi006' 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x560906ff1c10 0x560906ff1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1017fa7b80 4 client.4773 mds_command: new command op to 4287 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:00:45.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.965+0000 7f1017fa7b80 1 -- 172.21.15.6:0/650123724 --> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560906d8bf10 con 0x560906ff1c10 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.966+0000 7f10093b9700 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x560906ff1c10 0x560906ff1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.966+0000 7f0fe4ff9700 10 client.4773 ms_handle_connect on v2:172.21.15.6:6836/1668558371 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.966+0000 7f10093b9700 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x560906ff1c10 0x560906ff1fe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4486c4ff45c4bb99 server_cookie=86534e6d7539097b in_seq=0 out_seq=0 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.966+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/650123724 <== mds.0 v2:172.21.15.6:6836/1668558371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x560906d8bf10 con 0x560906ff1c10 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.966+0000 7f0fe4ff9700 10 client.4773 handle_command_reply: tid=0 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.966+0000 7f1017fa7b80 1 client.4773 shutdown 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 1 -- 172.21.15.6:0/650123724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 msgr2=0x560906fd1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd1210 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009f50 tx=0x7f0ff40047c0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 1 -- 172.21.15.6:0/650123724 shutdown_connections 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 0x560906fd22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd1210 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x560906ff1c10 0x560906ff1fe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/650123724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fdf810 0x560906fd1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f0fe4ff9700 0 client.4773 ms_handle_reset on v2:172.21.15.6:6836/1668558371 2023-11-28T13:00:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 1 -- 172.21.15.6:0/650123724 >> 172.21.15.6:0/650123724 conn(0x560906fb2d60 msgr2=0x560906fb5010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 1 -- 172.21.15.6:0/650123724 shutdown_connections 2023-11-28T13:00:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 1 -- 172.21.15.6:0/650123724 wait complete. 2023-11-28T13:00:45.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.967+0000 7f1017fa7b80 20 client.4773 trim_cache size 0 max 16384 2023-11-28T13:00:45.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.985+0000 7f1017fa7b80 1 Processor -- start 2023-11-28T13:00:45.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.985+0000 7f1017fa7b80 1 -- start start 2023-11-28T13:00:45.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.985+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fdd130 0x560906fae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fd3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 0x560906fd42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906fdcf80 con 0x560906fdf810 2023-11-28T13:00:45.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906fd27d0 con 0x560906fdd130 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x560906fdfec0 con 0x560906fe0120 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f10093b9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fd3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f10093b9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fd3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58532/0 (socket says 172.21.15.6:58532) 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f10093b9700 1 -- 172.21.15.6:0/2494577667 learned_addr learned my addr 172.21.15.6:0/2494577667 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f1009bba700 1 --2- 172.21.15.6:0/2494577667 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fdd130 0x560906fae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f100a3bb700 1 --2- 172.21.15.6:0/2494577667 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 0x560906fd42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f10093b9700 1 -- 172.21.15.6:0/2494577667 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 msgr2=0x560906fd42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f10093b9700 1 --2- 172.21.15.6:0/2494577667 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 0x560906fd42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f10093b9700 1 -- 172.21.15.6:0/2494577667 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fdd130 msgr2=0x560906fae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f10093b9700 1 --2- 172.21.15.6:0/2494577667 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fdd130 0x560906fae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f10093b9700 1 -- 172.21.15.6:0/2494577667 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560906ece140 con 0x560906fdf810 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f100a3bb700 1 --2- 172.21.15.6:0/2494577667 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 0x560906fd42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f10093b9700 1 --2- 172.21.15.6:0/2494577667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fd3d60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f100406d0f0 tx=0x7f1000008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1613e95192267e5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f0fe7fff700 1 -- 172.21.15.6:0/2494577667 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100001d1b0 con 0x560906fdf810 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f0fe7fff700 1 -- 172.21.15.6:0/2494577667 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100001d040 con 0x560906fdf810 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.986+0000 7f0fe7fff700 1 -- 172.21.15.6:0/2494577667 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000017a60 con 0x560906fdf810 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.987+0000 7f1017fa7b80 1 -- 172.21.15.6:0/2494577667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 msgr2=0x560906fd3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.987+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/2494577667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fd3d60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f100406d0f0 tx=0x7f1000008040 comp rx=0 tx=0).stop 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.987+0000 7f1017fa7b80 1 -- 172.21.15.6:0/2494577667 shutdown_connections 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.987+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/2494577667 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fe0120 0x560906fd42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.987+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/2494577667 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fdd130 0x560906fae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.987+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/2494577667 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdf810 0x560906fd3d60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.987+0000 7f1017fa7b80 1 -- 172.21.15.6:0/2494577667 >> 172.21.15.6:0/2494577667 conn(0x560906fb2d60 msgr2=0x560906fd04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:45.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.987+0000 7f1017fa7b80 1 -- 172.21.15.6:0/2494577667 shutdown_connections 2023-11-28T13:00:45.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.987+0000 7f1017fa7b80 1 -- 172.21.15.6:0/2494577667 wait complete. 2023-11-28T13:00:45.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.988+0000 7f1017fa7b80 1 -- start start 2023-11-28T13:00:45.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.988+0000 7f1017fa7b80 1 Processor -- start 2023-11-28T13:00:45.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fdf810 0x56090705e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1017fa7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fe0120 0x56090705ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56090705f400 con 0x560906fdd130 2023-11-28T13:00:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560906ffd610 con 0x560906fe0120 2023-11-28T13:00:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1017fa7b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x56090705ef60 con 0x560906fdf810 2023-11-28T13:00:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1017fa7b80 10 client.0 _ll_register_callbacks cb 0x560906dad240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:00:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1009bba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1009bba700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58536/0 (socket says 172.21.15.6:58536) 2023-11-28T13:00:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1009bba700 1 -- 172.21.15.6:0/3274551910 learned_addr learned my addr 172.21.15.6:0/3274551910 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:45.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f10093b9700 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fdf810 0x56090705e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f100a3bb700 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fe0120 0x56090705ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1009bba700 1 -- 172.21.15.6:0/3274551910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fdf810 msgr2=0x56090705e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1009bba700 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fdf810 0x56090705e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1009bba700 1 -- 172.21.15.6:0/3274551910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fe0120 msgr2=0x56090705ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1009bba700 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fe0120 0x56090705ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1009bba700 1 -- 172.21.15.6:0/3274551910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1000009c70 con 0x560906fdd130 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1009bba700 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd4110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009130 tx=0x7f0ff4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e2f5fe5f7f22ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f0fe4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/3274551910 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4031df0 con 0x560906fdd130 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/3274551910 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0ff403b990 con 0x560906fdd130 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.989+0000 7f1017fa7b80 1 -- 172.21.15.6:0/3274551910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560906ec5160 con 0x560906fdd130 2023-11-28T13:00:45.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/3274551910 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ff403bb50 con 0x560906fdd130 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f1017fa7b80 10 client.4779 fetch_fsmap learned FSMap version 9 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f1017fa7b80 1 -- 172.21.15.6:0/3274551910 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x560906d5c200 con 0x560906fdd130 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/3274551910 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560906ec5160 con 0x560906fdd130 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/3274551910 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 9) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0ff403bcb0 con 0x560906fdd130 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f1017fa7b80 10 client.4779 fetch_fsmap finished waiting for FSMap version 9 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f1017fa7b80 10 client.4779 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f1017fa7b80 20 client.4779 populate_metadata read hostname 'smithi006' 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x560907062600 0x5609070629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f1017fa7b80 4 client.4779 mds_command: new command op to 4287 tid=0[{"prefix": "status"}] 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f1017fa7b80 1 -- 172.21.15.6:0/3274551910 --> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560906eaf170 con 0x560907062600 2023-11-28T13:00:45.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f10093b9700 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x560907062600 0x5609070629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f0fe4ff9700 10 client.4779 ms_handle_connect on v2:172.21.15.6:6836/1668558371 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.990+0000 7f10093b9700 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x560907062600 0x5609070629d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4c43705491d48a server_cookie=2d97a51b47533cde in_seq=0 out_seq=0 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f0fe4ff9700 1 -- 172.21.15.6:0/3274551910 <== mds.0 v2:172.21.15.6:6836/1668558371 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x560906eaf170 con 0x560907062600 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f0fe4ff9700 10 client.4779 handle_command_reply: tid=0 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 client.4779 shutdown 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 -- 172.21.15.6:0/3274551910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 msgr2=0x560906fd4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd4110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009130 tx=0x7f0ff4011000 comp rx=0 tx=0).stop 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 -- 172.21.15.6:0/3274551910 shutdown_connections 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.6:6836/1668558371,v1:172.21.15.6:6837/1668558371] conn(0x560907062600 0x5609070629d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560906fe0120 0x56090705ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560906fdd130 0x560906fd4110 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:45.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f0fe4ff9700 0 client.4779 ms_handle_reset on v2:172.21.15.6:6836/1668558371 2023-11-28T13:00:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 --2- 172.21.15.6:0/3274551910 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560906fdf810 0x56090705e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 -- 172.21.15.6:0/3274551910 >> 172.21.15.6:0/3274551910 conn(0x560906fb2d60 msgr2=0x560906fd04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 -- 172.21.15.6:0/3274551910 shutdown_connections 2023-11-28T13:00:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 1 -- 172.21.15.6:0/3274551910 wait complete. 2023-11-28T13:00:46.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f1017fa7b80 20 client.4779 trim_cache size 0 max 16384 2023-11-28T13:00:46.000 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "dc643718-e974-4d8a-b829-ff6ced554687", 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4287, 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 24.586161175000001, 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 9, 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 25, 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 25, 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 29.922848067 2023-11-28T13:00:46.001 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:00:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f100be1e700 1 -- 172.21.15.6:0/3437725159 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0fe80682f0 msgr2=0x7f0fe806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f100be1e700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0fe80682f0 0x7f0fe806a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4000c00 tx=0x7f0ff4008030 comp rx=0 tx=0).stop 2023-11-28T13:00:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f100be1e700 1 -- 172.21.15.6:0/3437725159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1004161090 msgr2=0x7f100415efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.991+0000 7f100be1e700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1004161090 0x7f100415efc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00cc50 tx=0x7f0ffc00cf60 comp rx=0 tx=0).stop 2023-11-28T13:00:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.992+0000 7f100be1e700 1 -- 172.21.15.6:0/3437725159 shutdown_connections 2023-11-28T13:00:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.992+0000 7f100be1e700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1004161090 0x7f100415efc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.992+0000 7f100be1e700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0fe80682f0 0x7f0fe806a7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.992+0000 7f100be1e700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f10040caeb0 0x7f100415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.992+0000 7f100be1e700 1 --2- 172.21.15.6:0/3437725159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f100415c990 0x7f100415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.992+0000 7f100be1e700 1 -- 172.21.15.6:0/3437725159 >> 172.21.15.6:0/3437725159 conn(0x7f10040b1650 msgr2=0x7f10040af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.992+0000 7f100be1e700 1 -- 172.21.15.6:0/3437725159 shutdown_connections 2023-11-28T13:00:46.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:45.992+0000 7f100be1e700 1 -- 172.21.15.6:0/3437725159 wait complete. 2023-11-28T13:00:46.007 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-28T13:00:46.018 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-28T13:00:46.018 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T13:00:46.018 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-28T13:00:46.018 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:00:46.018 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:46.018 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-28T13:00:46.018 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:00:46.018 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:46.019 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-28T13:00:46.019 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:00:46.019 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:46.019 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-28T13:00:46.019 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T13:00:46.019 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:46.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:46.019 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list 2023-11-28T13:00:46.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:46.041 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link delete ceph-brx 2023-11-28T13:00:46.117 INFO:teuthology.orchestra.run.smithi061.stderr:Cannot find device "ceph-brx" 2023-11-28T13:00:46.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:00:46.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:46.119 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-28T13:00:46.136 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-28T13:00:46.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:46.137 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T13:00:46.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:46.218 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2023-11-28T13:00:46.327 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-28T13:00:46.327 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:00:46.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.453+0000 7f1b6a890700 1 Processor -- start 2023-11-28T13:00:46.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.453+0000 7f1b6a890700 1 -- start start 2023-11-28T13:00:46.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.454+0000 7f1b6a890700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b6415a410 0x7f1b640adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.454+0000 7f1b6a890700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1b64071070 0x7f1b640cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.454+0000 7f1b6a890700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6415a7e0 con 0x7f1b6415df40 2023-11-28T13:00:46.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.454+0000 7f1b6a890700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6415e3c0 con 0x7f1b64071070 2023-11-28T13:00:46.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.454+0000 7f1b6a890700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6415e6a0 con 0x7f1b6415a410 2023-11-28T13:00:46.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.454+0000 7f1b637fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1b6415df40 0x7f1b64070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45864/0 (socket says 172.21.15.6:45864) 2023-11-28T13:00:46.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.454+0000 7f1b637fe700 1 -- 172.21.15.6:0/538109196 learned_addr learned my addr 172.21.15.6:0/538109196 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:46.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.454+0000 7f1b68e2d700 1 --2- 172.21.15.6:0/538109196 >> v2:172.21.15.61:3300/0 conn(0x7f1b64071070 0x7f1b640cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.454+0000 7f1b63fff700 1 --2- 172.21.15.6:0/538109196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b6415a410 0x7f1b640adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b62ffd700 1 -- 172.21.15.6:0/538109196 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186996371 0 0) 0x7f1b6415a7e0 con 0x7f1b6415df40 2023-11-28T13:00:46.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b62ffd700 1 -- 172.21.15.6:0/538109196 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b500035b0 con 0x7f1b6415df40 2023-11-28T13:00:46.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b63fff700 1 -- 172.21.15.6:0/538109196 >> v2:172.21.15.61:3300/0 conn(0x7f1b64071070 msgr2=0x7f1b640cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:46.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b63fff700 1 --2- 172.21.15.6:0/538109196 >> v2:172.21.15.61:3300/0 conn(0x7f1b64071070 0x7f1b640cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b63fff700 1 -- 172.21.15.6:0/538109196 >> v1:172.21.15.6:6789/0 conn(0x7f1b6415df40 legacy=0x7f1b64070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b63fff700 1 -- 172.21.15.6:0/538109196 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6415e980 con 0x7f1b6415a410 2023-11-28T13:00:46.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b68e2d700 1 --2- 172.21.15.6:0/538109196 >> v2:172.21.15.61:3300/0 conn(0x7f1b64071070 0x7f1b640cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:46.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b63fff700 1 --2- 172.21.15.6:0/538109196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b6415a410 0x7f1b640adf90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009840 tx=0x7f1b54009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20befff33432076 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b62ffd700 1 -- 172.21.15.6:0/538109196 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54011830 con 0x7f1b6415a410 2023-11-28T13:00:46.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b62ffd700 1 -- 172.21.15.6:0/538109196 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b54011990 con 0x7f1b6415a410 2023-11-28T13:00:46.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.455+0000 7f1b62ffd700 1 -- 172.21.15.6:0/538109196 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54011d20 con 0x7f1b6415a410 2023-11-28T13:00:46.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.456+0000 7f1b6a890700 1 -- 172.21.15.6:0/538109196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b6415a410 msgr2=0x7f1b640adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.456+0000 7f1b6a890700 1 --2- 172.21.15.6:0/538109196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b6415a410 0x7f1b640adf90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009840 tx=0x7f1b54009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:46.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.456+0000 7f1b6a890700 1 -- 172.21.15.6:0/538109196 shutdown_connections 2023-11-28T13:00:46.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.456+0000 7f1b6a890700 1 --2- 172.21.15.6:0/538109196 >> v2:172.21.15.61:3300/0 conn(0x7f1b64071070 0x7f1b640cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.456+0000 7f1b6a890700 1 --2- 172.21.15.6:0/538109196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b6415a410 0x7f1b640adf90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.456+0000 7f1b6a890700 1 -- 172.21.15.6:0/538109196 >> 172.21.15.6:0/538109196 conn(0x7f1b640ce9e0 msgr2=0x7f1b640cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:46.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.456+0000 7f1b6a890700 1 -- 172.21.15.6:0/538109196 shutdown_connections 2023-11-28T13:00:46.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.456+0000 7f1b6a890700 1 -- 172.21.15.6:0/538109196 wait complete. 2023-11-28T13:00:46.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.457+0000 7f1b6a890700 1 Processor -- start 2023-11-28T13:00:46.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.457+0000 7f1b6a890700 1 -- start start 2023-11-28T13:00:46.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b6a890700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b64071070 0x7f1b641f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b6a890700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b6415a410 0x7f1b641566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b6a890700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b64157ec0 0x7f1b64155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b6a890700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b64156c10 con 0x7f1b64157ec0 2023-11-28T13:00:46.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b6a890700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b64156e20 con 0x7f1b6415a410 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b6a890700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b64157100 con 0x7f1b64071070 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b68e2d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b64157ec0 0x7f1b64155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b68e2d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b64157ec0 0x7f1b64155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58554/0 (socket says 172.21.15.6:58554) 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b68e2d700 1 -- 172.21.15.6:0/3429147199 learned_addr learned my addr 172.21.15.6:0/3429147199 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b637fe700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b6415a410 0x7f1b641566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b63fff700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b64071070 0x7f1b641f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b68e2d700 1 -- 172.21.15.6:0/3429147199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b64071070 msgr2=0x7f1b641f0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b68e2d700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b64071070 0x7f1b641f0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b68e2d700 1 -- 172.21.15.6:0/3429147199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b6415a410 msgr2=0x7f1b641566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b68e2d700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b6415a410 0x7f1b641566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.458+0000 7f1b68e2d700 1 -- 172.21.15.6:0/3429147199 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c009a70 con 0x7f1b64157ec0 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.459+0000 7f1b68e2d700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b64157ec0 0x7f1b64155d30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00efc0 tx=0x7f1b5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.459+0000 7f1b60ff9700 1 -- 172.21.15.6:0/3429147199 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c00ca90 con 0x7f1b64157ec0 2023-11-28T13:00:46.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.459+0000 7f1b60ff9700 1 -- 172.21.15.6:0/3429147199 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c00f040 con 0x7f1b64157ec0 2023-11-28T13:00:46.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.459+0000 7f1b6a890700 1 -- 172.21.15.6:0/3429147199 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b540094a0 con 0x7f1b64157ec0 2023-11-28T13:00:46.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.459+0000 7f1b60ff9700 1 -- 172.21.15.6:0/3429147199 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c00ce50 con 0x7f1b64157ec0 2023-11-28T13:00:46.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.459+0000 7f1b6a890700 1 -- 172.21.15.6:0/3429147199 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b640adbd0 con 0x7f1b64157ec0 2023-11-28T13:00:46.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.460+0000 7f1b6a890700 1 -- 172.21.15.6:0/3429147199 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6415ec60 con 0x7f1b64157ec0 2023-11-28T13:00:46.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.460+0000 7f1b60ff9700 1 -- 172.21.15.6:0/3429147199 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1b5c014480 con 0x7f1b64157ec0 2023-11-28T13:00:46.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.461+0000 7f1b60ff9700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b440682f0 0x7f1b4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:46.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.461+0000 7f1b60ff9700 1 -- 172.21.15.6:0/3429147199 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1b5c0866a0 con 0x7f1b64157ec0 2023-11-28T13:00:46.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.461+0000 7f1b63fff700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b440682f0 0x7f1b4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:46.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.461+0000 7f1b63fff700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b440682f0 0x7f1b4406a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1b54000c00 tx=0x7f1b54008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:46.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.465+0000 7f1b60ff9700 1 -- 172.21.15.6:0/3429147199 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1b5c052bb0 con 0x7f1b64157ec0 2023-11-28T13:00:46.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.705+0000 7f1b6a890700 1 -- 172.21.15.6:0/3429147199 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1b640add60 con 0x7f1b64157ec0 2023-11-28T13:00:46.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.705+0000 7f1b60ff9700 1 -- 172.21.15.6:0/3429147199 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1b5c0561d0 con 0x7f1b64157ec0 2023-11-28T13:00:46.709 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T13:00:46.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 -- 172.21.15.6:0/3429147199 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b440682f0 msgr2=0x7f1b4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b440682f0 0x7f1b4406a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1b54000c00 tx=0x7f1b54008030 comp rx=0 tx=0).stop 2023-11-28T13:00:46.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 -- 172.21.15.6:0/3429147199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b64157ec0 msgr2=0x7f1b64155d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:46.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b64157ec0 0x7f1b64155d30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00efc0 tx=0x7f1b5c00c3a0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 -- 172.21.15.6:0/3429147199 shutdown_connections 2023-11-28T13:00:46.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1b64157ec0 0x7f1b64155d30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b440682f0 0x7f1b4406a7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1b64071070 0x7f1b641f0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 --2- 172.21.15.6:0/3429147199 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b6415a410 0x7f1b641566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:46.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 -- 172.21.15.6:0/3429147199 >> 172.21.15.6:0/3429147199 conn(0x7f1b640ce9e0 msgr2=0x7f1b6415b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:46.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 -- 172.21.15.6:0/3429147199 shutdown_connections 2023-11-28T13:00:46.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:46.707+0000 7f1b6a890700 1 -- 172.21.15.6:0/3429147199 wait complete. 2023-11-28T13:00:46.720 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:00:46.720 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:00:46.720 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-28T13:00:46.720 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:00:46.720 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:46.721 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:00:46.721 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:00:46.721 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:00:46.721 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:00:46.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:46.721 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-28T13:00:46.738 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T13:00:46.739 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42316sec preferred_lft 42316sec 2023-11-28T13:00:46.740 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2023-11-28T13:00:46.740 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:46.740 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:46.740 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:46.740 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T13:00:46.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:46.740 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:46.740 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:46.740 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2023-11-28T13:00:46.741 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2023-11-28T13:00:46.741 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2023-11-28T13:00:46.741 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T13:00:46.741 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:46.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:46.857 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:00:46.885 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-28T13:00:46.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:46.886 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-28T13:00:46.945 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-28T13:00:46.945 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:00:46.945 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T13:00:46.945 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T13:00:46.945 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:00:46.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:46.946 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:46.946 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:46.946 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T13:00:46.946 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T13:00:46.946 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T13:00:46.946 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:47.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:47.053 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-28T13:00:47.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:47.070 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2023-11-28T13:00:47.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:47.127 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:47.127 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:47.127 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:00:47.128 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T13:00:47.128 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:47.222 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T13:00:47.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:47.222 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:47.222 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:47.222 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T13:00:47.222 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T13:00:47.222 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T13:00:47.223 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T13:00:47.223 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T13:00:47.223 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:47.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:47.426 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:47.426 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:47.426 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2023-11-28T13:00:47.426 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T13:00:47.426 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:47.542 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T13:00:47.542 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:47.542 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:47.573 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:00:47.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:47.574 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:47.631 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:00:47.700 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:00:47.700 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:00:47.766 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:00:47.767 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:00:47.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:47.768 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:00:47.825 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T13:00:47.825 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T13:00:47.867 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:00:47.896 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:00:47.896 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:00:47.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:2023-11-28T13:00:47.957+0000 7fc363ce15c0 -1 init, newargv = 0x55a357dc3720 newargc=15 2023-11-28T13:00:47.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[114701]: starting ceph client 2023-11-28T13:00:47.964 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[114701]: starting fuse 2023-11-28T13:00:47.966 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:00:47.967 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:00:47.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:47.968 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:00:48.027 INFO:teuthology.orchestra.run.smithi006.stdout:47 2023-11-28T13:00:48.027 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> import re 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2023-11-28T13:00:48.028 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> return f 2023-11-28T13:00:48.029 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T13:00:48.030 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:48.030 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.0")) 2023-11-28T13:00:48.030 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:00:48.151 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.0.114701.asok 2023-11-28T13:00:48.332 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.114701.asok 2023-11-28T13:00:48.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:48.333 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.114701.asok status 2023-11-28T13:00:48.495 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:00:48.495 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2023-11-28T13:00:48.495 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T13:00:48.495 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T13:00:48.495 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "0", 2023-11-28T13:00:48.495 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "114701", 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4791, 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 4791 2023-11-28T13:00:48.496 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3972775629 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3972775629 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.4791 192.168.0.1:0/3972775629", 2023-11-28T13:00:48.497 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.1:0/3972775629", 2023-11-28T13:00:48.498 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2023-11-28T13:00:48.498 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 9, 2023-11-28T13:00:48.498 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 25, 2023-11-28T13:00:48.498 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2023-11-28T13:00:48.498 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2023-11-28T13:00:48.498 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2023-11-28T13:00:48.498 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:00:48.505 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:00:48.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.628+0000 7f26fee44700 1 Processor -- start 2023-11-28T13:00:48.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.628+0000 7f26fee44700 1 -- start start 2023-11-28T13:00:48.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26fee44700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f26f815a6c0 0x7f26f80cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26fee44700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f26f80d0020 0x7f26f80ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26fee44700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f80cfb90 con 0x7f26f80b2660 2023-11-28T13:00:48.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26fee44700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f80cb930 con 0x7f26f80d0020 2023-11-28T13:00:48.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26fee44700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f80cf260 con 0x7f26f815a6c0 2023-11-28T13:00:48.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26f7fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f26f80d0020 0x7f26f80ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26fd3e1700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f26f80b2660 0x7f26f80b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45896/0 (socket says 172.21.15.6:45896) 2023-11-28T13:00:48.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26fd3e1700 1 -- 172.21.15.6:0/648598264 learned_addr learned my addr 172.21.15.6:0/648598264 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26fcbe0700 1 --2- 172.21.15.6:0/648598264 >> v2:172.21.15.61:3300/0 conn(0x7f26f815a6c0 0x7f26f80cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26f7fff700 1 -- 172.21.15.6:0/648598264 >> v2:172.21.15.61:3300/0 conn(0x7f26f815a6c0 msgr2=0x7f26f80cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26f7fff700 1 --2- 172.21.15.6:0/648598264 >> v2:172.21.15.61:3300/0 conn(0x7f26f815a6c0 0x7f26f80cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26f7fff700 1 -- 172.21.15.6:0/648598264 >> v1:172.21.15.6:6789/0 conn(0x7f26f80b2660 legacy=0x7f26f80b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.629+0000 7f26f7fff700 1 -- 172.21.15.6:0/648598264 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f80cf540 con 0x7f26f80d0020 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26f7fff700 1 --2- 172.21.15.6:0/648598264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80d0020 0x7f26f80ccea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f26f806ffd0 tx=0x7f26e4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4563f2ae36a915 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26f77fe700 1 -- 172.21.15.6:0/648598264 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4011830 con 0x7f26f80d0020 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26f77fe700 1 -- 172.21.15.6:0/648598264 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e4011990 con 0x7f26f80d0020 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26f77fe700 1 -- 172.21.15.6:0/648598264 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4011d20 con 0x7f26f80d0020 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26fee44700 1 -- 172.21.15.6:0/648598264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80d0020 msgr2=0x7f26f80ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26fee44700 1 --2- 172.21.15.6:0/648598264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80d0020 0x7f26f80ccea0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f26f806ffd0 tx=0x7f26e4009b20 comp rx=0 tx=0).stop 2023-11-28T13:00:48.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26fee44700 1 -- 172.21.15.6:0/648598264 shutdown_connections 2023-11-28T13:00:48.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26fee44700 1 --2- 172.21.15.6:0/648598264 >> v2:172.21.15.61:3300/0 conn(0x7f26f815a6c0 0x7f26f80cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26fee44700 1 --2- 172.21.15.6:0/648598264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80d0020 0x7f26f80ccea0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26fee44700 1 -- 172.21.15.6:0/648598264 >> 172.21.15.6:0/648598264 conn(0x7f26f80cd660 msgr2=0x7f26f80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:48.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26fee44700 1 -- 172.21.15.6:0/648598264 shutdown_connections 2023-11-28T13:00:48.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.630+0000 7f26fee44700 1 -- 172.21.15.6:0/648598264 wait complete. 2023-11-28T13:00:48.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.632+0000 7f26fee44700 1 Processor -- start 2023-11-28T13:00:48.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.632+0000 7f26fee44700 1 -- start start 2023-11-28T13:00:48.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.632+0000 7f26fee44700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80b2660 0x7f26f806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.632+0000 7f26fee44700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26f80d0020 0x7f26f806c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.632+0000 7f26fee44700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f815a6c0 0x7f26f80b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fee44700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81547d0 con 0x7f26f80b2660 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fee44700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f806bc20 con 0x7f26f815a6c0 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fee44700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f806d340 con 0x7f26f80d0020 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fcbe0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80b2660 0x7f26f806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fcbe0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80b2660 0x7f26f806cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58578/0 (socket says 172.21.15.6:58578) 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fcbe0700 1 -- 172.21.15.6:0/3674832627 learned_addr learned my addr 172.21.15.6:0/3674832627 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fd3e1700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f815a6c0 0x7f26f80b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26f7fff700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26f80d0020 0x7f26f806c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fcbe0700 1 -- 172.21.15.6:0/3674832627 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26f80d0020 msgr2=0x7f26f806c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fcbe0700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26f80d0020 0x7f26f806c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fcbe0700 1 -- 172.21.15.6:0/3674832627 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f815a6c0 msgr2=0x7f26f80b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fcbe0700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f815a6c0 0x7f26f80b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fcbe0700 1 -- 172.21.15.6:0/3674832627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40094a0 con 0x7f26f80b2660 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26f7fff700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26f80d0020 0x7f26f806c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fd3e1700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f815a6c0 0x7f26f80b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fcbe0700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80b2660 0x7f26f806cc40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00d830 tx=0x7f26ec00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26f57fa700 1 -- 172.21.15.6:0/3674832627 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec0125f0 con 0x7f26f80b2660 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26f57fa700 1 -- 172.21.15.6:0/3674832627 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec012750 con 0x7f26f80b2660 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26f57fa700 1 -- 172.21.15.6:0/3674832627 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec012b00 con 0x7f26f80b2660 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fee44700 1 -- 172.21.15.6:0/3674832627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f80cd3e0 con 0x7f26f80b2660 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.633+0000 7f26fee44700 1 -- 172.21.15.6:0/3674832627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f80cf820 con 0x7f26f80b2660 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.634+0000 7f26fee44700 1 -- 172.21.15.6:0/3674832627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f80d0470 con 0x7f26f80b2660 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.635+0000 7f26f57fa700 1 -- 172.21.15.6:0/3674832627 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f26ec012c60 con 0x7f26f80b2660 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.635+0000 7f26f57fa700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26dc0682f0 0x7f26dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:48.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.635+0000 7f26f57fa700 1 -- 172.21.15.6:0/3674832627 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f26ec086480 con 0x7f26f80b2660 2023-11-28T13:00:48.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.635+0000 7f26f7fff700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26dc0682f0 0x7f26dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:48.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.636+0000 7f26f7fff700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26dc0682f0 0x7f26dc06a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000c00 tx=0x7f26e4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:48.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.640+0000 7f26f57fa700 1 -- 172.21.15.6:0/3674832627 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f26ec0529f0 con 0x7f26f80b2660 2023-11-28T13:00:48.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.878+0000 7f26fee44700 1 -- 172.21.15.6:0/3674832627 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f26f806c200 con 0x7f26f80b2660 2023-11-28T13:00:48.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.878+0000 7f26f57fa700 1 -- 172.21.15.6:0/3674832627 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f26ec056010 con 0x7f26f80b2660 2023-11-28T13:00:48.880 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T13:00:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 -- 172.21.15.6:0/3674832627 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26dc0682f0 msgr2=0x7f26dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26dc0682f0 0x7f26dc06a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000c00 tx=0x7f26e4008030 comp rx=0 tx=0).stop 2023-11-28T13:00:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 -- 172.21.15.6:0/3674832627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80b2660 msgr2=0x7f26f806cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80b2660 0x7f26f806cc40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00d830 tx=0x7f26ec00dbf0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 -- 172.21.15.6:0/3674832627 shutdown_connections 2023-11-28T13:00:48.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f815a6c0 0x7f26f80b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26f80b2660 0x7f26f806cc40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26dc0682f0 0x7f26dc06a7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 --2- 172.21.15.6:0/3674832627 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26f80d0020 0x7f26f806c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:48.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 -- 172.21.15.6:0/3674832627 >> 172.21.15.6:0/3674832627 conn(0x7f26f80cd660 msgr2=0x7f26f80b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:48.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 -- 172.21.15.6:0/3674832627 shutdown_connections 2023-11-28T13:00:48.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:48.880+0000 7f26fee44700 1 -- 172.21.15.6:0/3674832627 wait complete. 2023-11-28T13:00:48.894 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:00:48.894 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:00:48.894 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-28T13:00:48.894 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T13:00:48.894 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:48.894 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:00:48.894 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:00:48.894 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:00:48.894 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T13:00:48.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:48.895 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-28T13:00:48.912 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:48.912 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:48.912 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T13:00:48.913 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42314sec preferred_lft 42314sec 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether ba:07:72:c8:31:18 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::68d7:25ff:fe50:263/64 scope link 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:00:48.914 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether ba:07:72:c8:31:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:00:48.915 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::b807:72ff:fec8:3118/64 scope link tentative 2023-11-28T13:00:48.915 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:48.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:48.915 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:00:48.979 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-28T13:00:48.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:48.981 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-28T13:00:49.040 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-28T13:00:49.040 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:00:49.041 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T13:00:49.041 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T13:00:49.041 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:00:49.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:49.041 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:49.041 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:49.041 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T13:00:49.041 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T13:00:49.041 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T13:00:49.042 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:49.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:49.149 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-28T13:00:49.166 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:00:49.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:49.167 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2023-11-28T13:00:49.224 INFO:teuthology.orchestra.run.smithi006.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T13:00:49.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:49.225 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:49.225 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:49.225 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:00:49.225 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T13:00:49.225 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:49.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:49.321 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T13:00:49.359 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:49.359 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:49.359 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:49.359 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:49.359 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:49.360 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:49.360 INFO:teuthology.orchestra.run.smithi006.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:49.360 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 4a:6c:3f:45:8e:5e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:00:49.360 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:00:49.360 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:49.360 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::486c:3fff:fe45:8e5e/64 scope link 2023-11-28T13:00:49.360 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:49.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:49.370 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T13:00:49.444 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:49.444 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:49.444 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:49.444 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:49.444 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:49.444 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:49.445 INFO:teuthology.orchestra.run.smithi006.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:49.445 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 4a:6c:3f:45:8e:5e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:00:49.445 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:00:49.445 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:49.445 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::486c:3fff:fe45:8e5e/64 scope link 2023-11-28T13:00:49.445 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:49.460 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T13:00:49.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:49.461 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:49.461 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:49.461 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T13:00:49.461 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T13:00:49.461 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T13:00:49.461 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T13:00:49.461 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T13:00:49.461 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:49.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:49.725 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:00:49.725 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:00:49.725 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.1 up 2023-11-28T13:00:49.725 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T13:00:49.725 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:00:49.839 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-28T13:00:49.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:49.839 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:49.863 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T13:00:49.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:49.863 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:49.920 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:00:49.988 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:00:49.988 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:00:50.056 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:00:50.057 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:00:50.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:50.057 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:00:50.115 INFO:teuthology.orchestra.run.smithi006.stdout:47 2023-11-28T13:00:50.115 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2023-11-28T13:00:50.116 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2 --client_mountpoint=/) 2023-11-28T13:00:50.157 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:00:50.185 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:00:50.186 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:00:50.248 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi006.stderr:2023-11-28T13:00:50.247+0000 7f4c35eb15c0 -1 init, newargv = 0x55b01852cc90 newargc=15 2023-11-28T13:00:50.248 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi006.stderr:ceph-fuse[115355]: starting ceph client 2023-11-28T13:00:50.253 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi006.stderr:ceph-fuse[115355]: starting fuse 2023-11-28T13:00:50.257 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:00:50.258 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:00:50.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:50.259 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:00:50.316 INFO:teuthology.orchestra.run.smithi006.stdout:47 2023-11-28T13:00:50.316 INFO:teuthology.orchestra.run.smithi006.stdout:48 2023-11-28T13:00:50.317 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2023-11-28T13:00:50.317 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:00:50.317 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2023-11-28T13:00:50.317 DEBUG:teuthology.orchestra.run.smithi006:> import re 2023-11-28T13:00:50.317 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:00:50.317 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2023-11-28T13:00:50.317 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:50.317 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2023-11-28T13:00:50.317 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-28T13:00:50.317 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2023-11-28T13:00:50.317 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2023-11-28T13:00:50.318 DEBUG:teuthology.orchestra.run.smithi006:> return f 2023-11-28T13:00:50.319 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T13:00:50.319 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:00:50.319 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.2")) 2023-11-28T13:00:50.319 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:00:50.441 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.2.115355.asok 2023-11-28T13:00:50.624 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.115355.asok 2023-11-28T13:00:50.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:50.624 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.115355.asok status 2023-11-28T13:00:50.786 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:00:50.786 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2023-11-28T13:00:50.786 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T13:00:50.786 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T13:00:50.786 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "2", 2023-11-28T13:00:50.787 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2023-11-28T13:00:50.787 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-28T13:00:50.787 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "115355", 2023-11-28T13:00:50.787 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2023-11-28T13:00:50.787 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:50.787 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2023-11-28T13:00:50.787 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4806, 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 4806 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.2:0", 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2643380977 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:00:50.788 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.2:0", 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 2643380977 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.4806 192.168.0.2:0/2643380977", 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.2:0/2643380977", 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 9, 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 25, 2023-11-28T13:00:50.789 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2023-11-28T13:00:50.790 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2023-11-28T13:00:50.790 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2023-11-28T13:00:50.790 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:00:50.796 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:00:50.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.920+0000 7f849aba7700 1 Processor -- start 2023-11-28T13:00:50.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.920+0000 7f849aba7700 1 -- start start 2023-11-28T13:00:50.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.920+0000 7f849aba7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f849406e400 0x7f849415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.920+0000 7f849aba7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f84940ce890 0x7f84940ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.920+0000 7f849aba7700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f849415e0b0 con 0x7f8494159590 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f849aba7700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84940cf250 con 0x7f849406e400 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f849aba7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849415d120 con 0x7f84940ce890 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f8493fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f849406e400 0x7f849415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f8498943700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f8494159590 0x7f84940ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45920/0 (socket says 172.21.15.6:45920) 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f8498943700 1 -- 172.21.15.6:0/3318661711 learned_addr learned my addr 172.21.15.6:0/3318661711 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f8499144700 1 --2- 172.21.15.6:0/3318661711 >> v2:172.21.15.61:3300/0 conn(0x7f84940ce890 0x7f84940ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f8493fff700 1 -- 172.21.15.6:0/3318661711 >> v2:172.21.15.61:3300/0 conn(0x7f84940ce890 msgr2=0x7f84940ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f8493fff700 1 --2- 172.21.15.6:0/3318661711 >> v2:172.21.15.61:3300/0 conn(0x7f84940ce890 0x7f84940ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f8493fff700 1 -- 172.21.15.6:0/3318661711 >> v1:172.21.15.6:6789/0 conn(0x7f8494159590 legacy=0x7f84940ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f84937fe700 1 -- 172.21.15.6:0/3318661711 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053944903 0 0) 0x7f8488001110 con 0x7f8494159590 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f8493fff700 1 -- 172.21.15.6:0/3318661711 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849415d6e0 con 0x7f849406e400 2023-11-28T13:00:50.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f8493fff700 1 --2- 172.21.15.6:0/3318661711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f849406e400 0x7f849415db70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f84940cd500 tx=0x7f8480009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b167562828f822b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f84937fe700 1 -- 172.21.15.6:0/3318661711 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480011830 con 0x7f849406e400 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f84937fe700 1 -- 172.21.15.6:0/3318661711 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480011990 con 0x7f849406e400 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.921+0000 7f84937fe700 1 -- 172.21.15.6:0/3318661711 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480011d20 con 0x7f849406e400 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.922+0000 7f849aba7700 1 -- 172.21.15.6:0/3318661711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f849406e400 msgr2=0x7f849415db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.922+0000 7f849aba7700 1 --2- 172.21.15.6:0/3318661711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f849406e400 0x7f849415db70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f84940cd500 tx=0x7f8480009b20 comp rx=0 tx=0).stop 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.922+0000 7f849aba7700 1 -- 172.21.15.6:0/3318661711 shutdown_connections 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.922+0000 7f849aba7700 1 --2- 172.21.15.6:0/3318661711 >> v2:172.21.15.61:3300/0 conn(0x7f84940ce890 0x7f84940ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.922+0000 7f849aba7700 1 --2- 172.21.15.6:0/3318661711 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f849406e400 0x7f849415db70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.922+0000 7f849aba7700 1 -- 172.21.15.6:0/3318661711 >> 172.21.15.6:0/3318661711 conn(0x7f84940b1650 msgr2=0x7f84940b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.922+0000 7f849aba7700 1 -- 172.21.15.6:0/3318661711 shutdown_connections 2023-11-28T13:00:50.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.922+0000 7f849aba7700 1 -- 172.21.15.6:0/3318661711 wait complete. 2023-11-28T13:00:50.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.923+0000 7f849aba7700 1 Processor -- start 2023-11-28T13:00:50.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.923+0000 7f849aba7700 1 -- start start 2023-11-28T13:00:50.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f849aba7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f849406e400 0x7f8494152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f849aba7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84940ce890 0x7f8494157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f849aba7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8494159590 0x7f8494158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f849aba7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84940c8560 con 0x7f8494159590 2023-11-28T13:00:50.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f849aba7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8494140b40 con 0x7f84940ce890 2023-11-28T13:00:50.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f849aba7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8494143390 con 0x7f849406e400 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8499144700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8494159590 0x7f8494158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8499144700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8494159590 0x7f8494158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58610/0 (socket says 172.21.15.6:58610) 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8499144700 1 -- 172.21.15.6:0/3938561527 learned_addr learned my addr 172.21.15.6:0/3938561527 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8498943700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f849406e400 0x7f8494152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8493fff700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84940ce890 0x7f8494157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8499144700 1 -- 172.21.15.6:0/3938561527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f849406e400 msgr2=0x7f8494152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8499144700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f849406e400 0x7f8494152170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8499144700 1 -- 172.21.15.6:0/3938561527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84940ce890 msgr2=0x7f8494157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8499144700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84940ce890 0x7f8494157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8499144700 1 -- 172.21.15.6:0/3938561527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800094a0 con 0x7f8494159590 2023-11-28T13:00:50.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8498943700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f849406e400 0x7f8494152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:50.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.924+0000 7f8493fff700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84940ce890 0x7f8494157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:50.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.925+0000 7f8499144700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8494159590 0x7f8494158510 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f848c00cd70 tx=0x7f848c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.925+0000 7f84917fa700 1 -- 172.21.15.6:0/3938561527 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c003a20 con 0x7f8494159590 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.925+0000 7f84917fa700 1 -- 172.21.15.6:0/3938561527 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c003b80 con 0x7f8494159590 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.925+0000 7f84917fa700 1 -- 172.21.15.6:0/3938561527 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c004020 con 0x7f8494159590 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.925+0000 7f849aba7700 1 -- 172.21.15.6:0/3938561527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84940af240 con 0x7f8494159590 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.925+0000 7f849aba7700 1 -- 172.21.15.6:0/3938561527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84940aef60 con 0x7f8494159590 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.926+0000 7f849aba7700 1 -- 172.21.15.6:0/3938561527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849415d370 con 0x7f8494159590 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.926+0000 7f84917fa700 1 -- 172.21.15.6:0/3938561527 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f848c01c020 con 0x7f8494159590 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.927+0000 7f84917fa700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f84780682f0 0x7f847806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.927+0000 7f84917fa700 1 -- 172.21.15.6:0/3938561527 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f848c012070 con 0x7f8494159590 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.927+0000 7f8498943700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f84780682f0 0x7f847806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:50.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.927+0000 7f8498943700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f84780682f0 0x7f847806a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8488007fd0 tx=0x7f8488009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:50.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:50.931+0000 7f84917fa700 1 -- 172.21.15.6:0/3938561527 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f848c052d20 con 0x7f8494159590 2023-11-28T13:00:51.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.173+0000 7f849aba7700 1 -- 172.21.15.6:0/3938561527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f84940c86a0 con 0x7f8494159590 2023-11-28T13:00:51.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.173+0000 7f84917fa700 1 -- 172.21.15.6:0/3938561527 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f84940c86a0 con 0x7f8494159590 2023-11-28T13:00:51.175 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T13:00:51.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 -- 172.21.15.6:0/3938561527 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f84780682f0 msgr2=0x7f847806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f84780682f0 0x7f847806a7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8488007fd0 tx=0x7f8488009040 comp rx=0 tx=0).stop 2023-11-28T13:00:51.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 -- 172.21.15.6:0/3938561527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8494159590 msgr2=0x7f8494158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:51.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8494159590 0x7f8494158510 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f848c00cd70 tx=0x7f848c00a570 comp rx=0 tx=0).stop 2023-11-28T13:00:51.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 -- 172.21.15.6:0/3938561527 shutdown_connections 2023-11-28T13:00:51.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8494159590 0x7f8494158510 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f84780682f0 0x7f847806a7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f849406e400 0x7f8494152170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 --2- 172.21.15.6:0/3938561527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f84940ce890 0x7f8494157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:51.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 -- 172.21.15.6:0/3938561527 >> 172.21.15.6:0/3938561527 conn(0x7f84940b1650 msgr2=0x7f8494153920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:51.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 -- 172.21.15.6:0/3938561527 shutdown_connections 2023-11-28T13:00:51.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:51.175+0000 7f849aba7700 1 -- 172.21.15.6:0/3938561527 wait complete. 2023-11-28T13:00:51.188 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:00:51.189 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:00:51.189 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi061.front.sepia.ceph.com 2023-11-28T13:00:51.189 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T13:00:51.189 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:51.189 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:00:51.189 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:00:51.189 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:00:51.189 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T13:00:51.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.189 DEBUG:teuthology.orchestra.run.smithi061:> ip addr 2023-11-28T13:00:51.211 INFO:teuthology.orchestra.run.smithi061.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:51.211 INFO:teuthology.orchestra.run.smithi061.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:51.211 INFO:teuthology.orchestra.run.smithi061.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:a8:bb:0c brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:a8:bb:0d brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:b6:f8:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:b6:f8:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:51.212 INFO:teuthology.orchestra.run.smithi061.stdout: inet 172.21.15.61/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:00:51.213 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft 42307sec preferred_lft 42307sec 2023-11-28T13:00:51.213 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 fe80::f650:6185:a053:ed96/64 scope link noprefixroute 2023-11-28T13:00:51.213 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:51.213 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T13:00:51.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.213 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-11-28T13:00:51.213 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-11-28T13:00:51.213 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link add name ceph-brx type bridge 2023-11-28T13:00:51.213 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip addr flush dev ceph-brx 2023-11-28T13:00:51.214 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set ceph-brx up 2023-11-28T13:00:51.214 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T13:00:51.214 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-11-28T13:00:51.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.392 DEBUG:teuthology.orchestra.run.smithi061:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:00:51.422 INFO:teuthology.orchestra.run.smithi061.stdout:1 2023-11-28T13:00:51.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.423 DEBUG:teuthology.orchestra.run.smithi061:> route 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi061.stdout:Kernel IP routing table 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi061.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi061.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run.smithi061.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:00:51.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.500 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-11-28T13:00:51.500 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-11-28T13:00:51.500 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:00:51.500 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:00:51.500 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:00:51.500 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-11-28T13:00:51.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.792 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list 2023-11-28T13:00:51.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.809 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list-id 2023-11-28T13:00:51.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.865 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-11-28T13:00:51.866 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-11-28T13:00:51.866 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T13:00:51.866 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T13:00:51.866 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-11-28T13:00:51.963 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T13:00:51.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:51.963 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-11-28T13:00:51.963 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-11-28T13:00:51.963 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T13:00:51.963 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T13:00:51.964 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T13:00:51.964 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T13:00:51.964 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T13:00:51.964 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-11-28T13:00:52.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:52.191 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-11-28T13:00:52.191 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-11-28T13:00:52.191 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set brx.0 up 2023-11-28T13:00:52.191 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T13:00:52.191 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-11-28T13:00:52.299 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-28T13:00:52.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:52.299 DEBUG:teuthology.orchestra.run.smithi061:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:52.330 INFO:teuthology.orchestra.run.smithi061.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T13:00:52.330 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:52.330 DEBUG:teuthology.orchestra.run.smithi061:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:52.387 DEBUG:teuthology.orchestra.run.smithi061:> sudo modprobe fuse 2023-11-28T13:00:52.454 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:00:52.454 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:00:52.521 INFO:teuthology.orchestra.run.smithi061.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:00:52.521 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:00:52.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:52.522 DEBUG:teuthology.orchestra.run.smithi061:> ls /sys/fs/fuse/connections 2023-11-28T13:00:52.579 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T13:00:52.579 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1 --client_mountpoint=/) 2023-11-28T13:00:52.621 DEBUG:teuthology.orchestra.run.smithi061:> sudo modprobe fuse 2023-11-28T13:00:52.648 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:00:52.648 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:00:52.716 INFO:teuthology.orchestra.run.smithi061.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:00:52.717 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:00:52.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:52.718 DEBUG:teuthology.orchestra.run.smithi061:> ls /sys/fs/fuse/connections 2023-11-28T13:00:52.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi061.stderr:2023-11-28T13:00:52.724+0000 7f26df4f45c0 -1 init, newargv = 0x55e7d9b61970 newargc=15 2023-11-28T13:00:52.726 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi061.stderr:ceph-fuse[110198]: starting ceph client 2023-11-28T13:00:52.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi061.stderr:ceph-fuse[110198]: starting fuse 2023-11-28T13:00:52.743 INFO:teuthology.orchestra.run.smithi061.stdout:46 2023-11-28T13:00:52.743 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-11-28T13:00:52.744 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:00:52.744 DEBUG:teuthology.orchestra.run.smithi061:> import glob 2023-11-28T13:00:52.744 DEBUG:teuthology.orchestra.run.smithi061:> import re 2023-11-28T13:00:52.744 DEBUG:teuthology.orchestra.run.smithi061:> import os 2023-11-28T13:00:52.744 DEBUG:teuthology.orchestra.run.smithi061:> import subprocess 2023-11-28T13:00:52.744 DEBUG:teuthology.orchestra.run.smithi061:> 2023-11-28T13:00:52.744 DEBUG:teuthology.orchestra.run.smithi061:> def _find_admin_socket(client_name): 2023-11-28T13:00:52.744 DEBUG:teuthology.orchestra.run.smithi061:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-28T13:00:52.744 DEBUG:teuthology.orchestra.run.smithi061:> files = glob.glob(asok_path) 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> # Given a non-glob path, it better be there 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> if "*" not in asok_path: 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> assert(len(files) == 1) 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> return files[0] 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> for f in files: 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> contents = proc_f.read() 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> if mountpoint in contents: 2023-11-28T13:00:52.745 DEBUG:teuthology.orchestra.run.smithi061:> return f 2023-11-28T13:00:52.746 DEBUG:teuthology.orchestra.run.smithi061:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T13:00:52.746 DEBUG:teuthology.orchestra.run.smithi061:> 2023-11-28T13:00:52.746 DEBUG:teuthology.orchestra.run.smithi061:> print(_find_admin_socket("client.1")) 2023-11-28T13:00:52.746 DEBUG:teuthology.orchestra.run.smithi061:> ' 2023-11-28T13:00:52.868 INFO:teuthology.orchestra.run.smithi061.stdout:/var/run/ceph/ceph-client.1.110198.asok 2023-11-28T13:00:53.050 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.110198.asok 2023-11-28T13:00:53.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:53.050 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.110198.asok status 2023-11-28T13:00:53.211 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-11-28T13:00:53.211 INFO:teuthology.orchestra.run.smithi061.stdout: "metadata": { 2023-11-28T13:00:53.211 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T13:00:53.211 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "entity_id": "1", 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "hostname": "smithi061", 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "pid": "110198", 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "root": "/" 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "dentry_count": 0, 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "dentry_pinned_count": 0, 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 4589, 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "inst": { 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "name": { 2023-11-28T13:00:53.212 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "client", 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 4589 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": { 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 538668218 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": { 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:00:53.213 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 538668218 2023-11-28T13:00:53.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-11-28T13:00:53.214 INFO:teuthology.orchestra.run.smithi061.stdout: "inst_str": "client.4589 192.168.0.1:0/538668218", 2023-11-28T13:00:53.214 INFO:teuthology.orchestra.run.smithi061.stdout: "addr_str": "192.168.0.1:0/538668218", 2023-11-28T13:00:53.214 INFO:teuthology.orchestra.run.smithi061.stdout: "inode_count": 1, 2023-11-28T13:00:53.214 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_epoch": 9, 2023-11-28T13:00:53.214 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_epoch": 25, 2023-11-28T13:00:53.214 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_epoch_barrier": 0, 2023-11-28T13:00:53.214 INFO:teuthology.orchestra.run.smithi061.stdout: "blocklisted": false, 2023-11-28T13:00:53.214 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs" 2023-11-28T13:00:53.214 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-11-28T13:00:53.221 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:00:53.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.359+0000 7f3648cc9700 1 Processor -- start 2023-11-28T13:00:53.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.359+0000 7f3648cc9700 1 -- start start 2023-11-28T13:00:53.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.360+0000 7f3648cc9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3644159590 0x7f364406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.360+0000 7f3648cc9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f364415db70 0x7f36440ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.360+0000 7f3648cc9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f36440ce890 0x7f36440ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.360+0000 7f3648cc9700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36440cf250 con 0x7f36440ce890 2023-11-28T13:00:53.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.360+0000 7f3648cc9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f364415d120 con 0x7f3644159590 2023-11-28T13:00:53.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.360+0000 7f3648cc9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364415d6e0 con 0x7f364415db70 2023-11-28T13:00:53.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.360+0000 7f3642d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f36440ce890 0x7f36440ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f3642d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f36440ce890 0x7f36440ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58618/0 (socket says 172.21.15.6:58618) 2023-11-28T13:00:53.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f3642d9d700 1 -- 172.21.15.6:0/3276660301 learned_addr learned my addr 172.21.15.6:0/3276660301 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:53.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f3641d9b700 1 --2- 172.21.15.6:0/3276660301 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f364415db70 0x7f36440ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f364259c700 1 --2- 172.21.15.6:0/3276660301 >> v2:172.21.15.61:3300/0 conn(0x7f3644159590 0x7f364406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f3642d9d700 1 -- 172.21.15.6:0/3276660301 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f364415db70 msgr2=0x7f36440ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:53.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f3642d9d700 1 --2- 172.21.15.6:0/3276660301 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f364415db70 0x7f36440ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f3642d9d700 1 -- 172.21.15.6:0/3276660301 >> v2:172.21.15.61:3300/0 conn(0x7f3644159590 msgr2=0x7f364406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:53.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f3642d9d700 1 --2- 172.21.15.6:0/3276660301 >> v2:172.21.15.61:3300/0 conn(0x7f3644159590 0x7f364406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f3642d9d700 1 -- 172.21.15.6:0/3276660301 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36440aef60 con 0x7f36440ce890 2023-11-28T13:00:53.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f3642d9d700 1 --2- 172.21.15.6:0/3276660301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36440ce890 0x7f36440ced10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f364406bd10 tx=0x7f363c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36895cb80e27b06d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:53.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f364159a700 1 -- 172.21.15.6:0/3276660301 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c00e070 con 0x7f36440ce890 2023-11-28T13:00:53.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f364159a700 1 -- 172.21.15.6:0/3276660301 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c00b910 con 0x7f36440ce890 2023-11-28T13:00:53.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.361+0000 7f364159a700 1 -- 172.21.15.6:0/3276660301 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c00bcc0 con 0x7f36440ce890 2023-11-28T13:00:53.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.362+0000 7f3648cc9700 1 -- 172.21.15.6:0/3276660301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36440ce890 msgr2=0x7f36440ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.362+0000 7f3648cc9700 1 --2- 172.21.15.6:0/3276660301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36440ce890 0x7f36440ced10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f364406bd10 tx=0x7f363c00b330 comp rx=0 tx=0).stop 2023-11-28T13:00:53.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.362+0000 7f3648cc9700 1 -- 172.21.15.6:0/3276660301 shutdown_connections 2023-11-28T13:00:53.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.362+0000 7f3648cc9700 1 --2- 172.21.15.6:0/3276660301 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36440ce890 0x7f36440ced10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.362+0000 7f3648cc9700 1 --2- 172.21.15.6:0/3276660301 >> v2:172.21.15.61:3300/0 conn(0x7f3644159590 0x7f364406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.362+0000 7f3648cc9700 1 --2- 172.21.15.6:0/3276660301 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f364415db70 0x7f36440ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.362+0000 7f3648cc9700 1 -- 172.21.15.6:0/3276660301 >> 172.21.15.6:0/3276660301 conn(0x7f36440b1650 msgr2=0x7f36440b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:53.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.362+0000 7f3648cc9700 1 -- 172.21.15.6:0/3276660301 shutdown_connections 2023-11-28T13:00:53.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.362+0000 7f3648cc9700 1 -- 172.21.15.6:0/3276660301 wait complete. 2023-11-28T13:00:53.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.363+0000 7f3648cc9700 1 Processor -- start 2023-11-28T13:00:53.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.363+0000 7f3648cc9700 1 -- start start 2023-11-28T13:00:53.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3648cc9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36440ce890 0x7f3644154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3648cc9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3644159590 0x7f36440c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3648cc9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f364415db70 0x7f36440c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3648cc9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36440c9ae0 con 0x7f3644159590 2023-11-28T13:00:53.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3648cc9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3644148570 con 0x7f36440ce890 2023-11-28T13:00:53.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3648cc9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3644148850 con 0x7f364415db70 2023-11-28T13:00:53.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3641d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3644159590 0x7f36440c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3641d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3644159590 0x7f36440c56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58622/0 (socket says 172.21.15.6:58622) 2023-11-28T13:00:53.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3641d9b700 1 -- 172.21.15.6:0/4186667370 learned_addr learned my addr 172.21.15.6:0/4186667370 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:53.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f364259c700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36440ce890 0x7f3644154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3642d9d700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f364415db70 0x7f36440c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3641d9b700 1 -- 172.21.15.6:0/4186667370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f364415db70 msgr2=0x7f36440c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3641d9b700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f364415db70 0x7f36440c32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3641d9b700 1 -- 172.21.15.6:0/4186667370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36440ce890 msgr2=0x7f3644154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.364+0000 7f3641d9b700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36440ce890 0x7f3644154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.365+0000 7f3641d9b700 1 -- 172.21.15.6:0/4186667370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c009b30 con 0x7f3644159590 2023-11-28T13:00:53.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.365+0000 7f3641d9b700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3644159590 0x7f36440c56e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f363800e870 tx=0x7f363800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:53.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.365+0000 7f3632ffd700 1 -- 172.21.15.6:0/4186667370 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638009590 con 0x7f3644159590 2023-11-28T13:00:53.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.365+0000 7f3632ffd700 1 -- 172.21.15.6:0/4186667370 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638013680 con 0x7f3644159590 2023-11-28T13:00:53.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.365+0000 7f3648cc9700 1 -- 172.21.15.6:0/4186667370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364415d370 con 0x7f3644159590 2023-11-28T13:00:53.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.365+0000 7f3632ffd700 1 -- 172.21.15.6:0/4186667370 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380139f0 con 0x7f3644159590 2023-11-28T13:00:53.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.365+0000 7f3648cc9700 1 -- 172.21.15.6:0/4186667370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36440af240 con 0x7f3644159590 2023-11-28T13:00:53.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.366+0000 7f3648cc9700 1 -- 172.21.15.6:0/4186667370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36440aec70 con 0x7f3644159590 2023-11-28T13:00:53.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.366+0000 7f3632ffd700 1 -- 172.21.15.6:0/4186667370 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3638019030 con 0x7f3644159590 2023-11-28T13:00:53.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.367+0000 7f3632ffd700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36240682f0 0x7f362406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:53.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.367+0000 7f364259c700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36240682f0 0x7f362406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:53.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.367+0000 7f3632ffd700 1 -- 172.21.15.6:0/4186667370 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3638085bc0 con 0x7f3644159590 2023-11-28T13:00:53.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.367+0000 7f364259c700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36240682f0 0x7f362406a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3634009be0 tx=0x7f3634009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:53.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.371+0000 7f3632ffd700 1 -- 172.21.15.6:0/4186667370 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3638052130 con 0x7f3644159590 2023-11-28T13:00:53.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.613+0000 7f3648cc9700 1 -- 172.21.15.6:0/4186667370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f36440675a0 con 0x7f3644159590 2023-11-28T13:00:53.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.614+0000 7f3632ffd700 1 -- 172.21.15.6:0/4186667370 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3638017070 con 0x7f3644159590 2023-11-28T13:00:53.614 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T13:00:53.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 -- 172.21.15.6:0/4186667370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36240682f0 msgr2=0x7f362406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36240682f0 0x7f362406a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3634009be0 tx=0x7f3634009210 comp rx=0 tx=0).stop 2023-11-28T13:00:53.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 -- 172.21.15.6:0/4186667370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3644159590 msgr2=0x7f36440c56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:53.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3644159590 0x7f36440c56e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f363800e870 tx=0x7f363800eb80 comp rx=0 tx=0).stop 2023-11-28T13:00:53.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 -- 172.21.15.6:0/4186667370 shutdown_connections 2023-11-28T13:00:53.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36240682f0 0x7f362406a7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f364415db70 0x7f36440c32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f36440ce890 0x7f3644154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 --2- 172.21.15.6:0/4186667370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3644159590 0x7f36440c56e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:53.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 -- 172.21.15.6:0/4186667370 >> 172.21.15.6:0/4186667370 conn(0x7f36440b1650 msgr2=0x7f364406b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:53.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 -- 172.21.15.6:0/4186667370 shutdown_connections 2023-11-28T13:00:53.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:53.616+0000 7f3648cc9700 1 -- 172.21.15.6:0/4186667370 wait complete. 2023-11-28T13:00:53.628 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:00:53.628 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:00:53.628 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi061.front.sepia.ceph.com 2023-11-28T13:00:53.628 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T13:00:53.628 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:53.628 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:00:53.628 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T13:00:53.628 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:00:53.628 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T13:00:53.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:53.629 DEBUG:teuthology.orchestra.run.smithi061:> ip addr 2023-11-28T13:00:53.648 INFO:teuthology.orchestra.run.smithi061.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:53.648 INFO:teuthology.orchestra.run.smithi061.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:53.648 INFO:teuthology.orchestra.run.smithi061.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:53.648 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:53.648 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:53.648 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:53.648 INFO:teuthology.orchestra.run.smithi061.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:a8:bb:0c brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:a8:bb:0d brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:b6:f8:b8 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:b6:f8:b9 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout: inet 172.21.15.61/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft 42305sec preferred_lft 42305sec 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 fe80::f650:6185:a053:ed96/64 scope link noprefixroute 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:53.649 INFO:teuthology.orchestra.run.smithi061.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:53.650 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0e:54:83:a1:88:a6 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:00:53.650 INFO:teuthology.orchestra.run.smithi061.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:00:53.650 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:53.650 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 fe80::30fc:5cff:fe8a:85ed/64 scope link 2023-11-28T13:00:53.650 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:53.650 INFO:teuthology.orchestra.run.smithi061.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:00:53.650 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0e:54:83:a1:88:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T13:00:53.650 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 fe80::c54:83ff:fea1:88a6/64 scope link 2023-11-28T13:00:53.650 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:53.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:53.651 DEBUG:teuthology.orchestra.run.smithi061:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:00:53.716 INFO:teuthology.orchestra.run.smithi061.stdout:1 2023-11-28T13:00:53.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:53.718 DEBUG:teuthology.orchestra.run.smithi061:> route 2023-11-28T13:00:53.777 INFO:teuthology.orchestra.run.smithi061.stdout:Kernel IP routing table 2023-11-28T13:00:53.777 INFO:teuthology.orchestra.run.smithi061.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:00:53.778 INFO:teuthology.orchestra.run.smithi061.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T13:00:53.778 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T13:00:53.778 INFO:teuthology.orchestra.run.smithi061.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:00:53.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:53.779 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-11-28T13:00:53.779 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-11-28T13:00:53.779 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T13:00:53.779 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T13:00:53.779 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T13:00:53.780 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-11-28T13:00:53.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:53.886 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list 2023-11-28T13:00:53.902 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T13:00:53.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:53.903 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list-id 2023-11-28T13:00:53.959 INFO:teuthology.orchestra.run.smithi061.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T13:00:53.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:53.960 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-11-28T13:00:53.960 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-11-28T13:00:53.960 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T13:00:53.960 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T13:00:53.960 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-11-28T13:00:54.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:54.059 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T13:00:54.091 INFO:teuthology.orchestra.run.smithi061.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:54.091 INFO:teuthology.orchestra.run.smithi061.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:54.092 INFO:teuthology.orchestra.run.smithi061.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:54.092 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:54.092 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:54.092 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:54.092 INFO:teuthology.orchestra.run.smithi061.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:54.092 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 92:50:75:56:e9:97 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:00:54.092 INFO:teuthology.orchestra.run.smithi061.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:00:54.092 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:54.092 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 fe80::9050:75ff:fe56:e997/64 scope link 2023-11-28T13:00:54.092 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:54.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:54.100 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T13:00:54.190 INFO:teuthology.orchestra.run.smithi061.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:00:54.190 INFO:teuthology.orchestra.run.smithi061.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:00:54.190 INFO:teuthology.orchestra.run.smithi061.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:00:54.190 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 ::1/128 scope host 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi061.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 92:50:75:56:e9:97 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi061.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 fe80::9050:75ff:fe56:e997/64 scope link 2023-11-28T13:00:54.191 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:00:54.202 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T13:00:54.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:54.203 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-11-28T13:00:54.203 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-11-28T13:00:54.203 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T13:00:54.203 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T13:00:54.203 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T13:00:54.203 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T13:00:54.203 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T13:00:54.203 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-11-28T13:00:54.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:54.412 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-11-28T13:00:54.412 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-11-28T13:00:54.412 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set brx.1 up 2023-11-28T13:00:54.412 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T13:00:54.412 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-11-28T13:00:54.519 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-28T13:00:54.519 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:54.519 DEBUG:teuthology.orchestra.run.smithi061:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:54.543 INFO:teuthology.orchestra.run.smithi061.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T13:00:54.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:00:54.544 DEBUG:teuthology.orchestra.run.smithi061:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:54.602 DEBUG:teuthology.orchestra.run.smithi061:> sudo modprobe fuse 2023-11-28T13:00:54.669 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:00:54.670 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:00:54.737 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:00:54.738 INFO:teuthology.orchestra.run.smithi061.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:00:54.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:54.738 DEBUG:teuthology.orchestra.run.smithi061:> ls /sys/fs/fuse/connections 2023-11-28T13:00:54.796 INFO:teuthology.orchestra.run.smithi061.stdout:46 2023-11-28T13:00:54.797 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [46] 2023-11-28T13:00:54.797 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3 --client_mountpoint=/) 2023-11-28T13:00:54.839 DEBUG:teuthology.orchestra.run.smithi061:> sudo modprobe fuse 2023-11-28T13:00:54.865 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:00:54.866 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:00:54.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi061.stderr:2023-11-28T13:00:54.926+0000 7f97056d35c0 -1 init, newargv = 0x55903094b6e0 newargc=15 2023-11-28T13:00:54.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi061.stderr:ceph-fuse[110787]: starting ceph client 2023-11-28T13:00:54.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi061.stderr:ceph-fuse[110787]: starting fuse 2023-11-28T13:00:54.936 INFO:teuthology.orchestra.run.smithi061.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:00:54.937 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:00:54.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:54.938 DEBUG:teuthology.orchestra.run.smithi061:> ls /sys/fs/fuse/connections 2023-11-28T13:00:54.995 INFO:teuthology.orchestra.run.smithi061.stdout:46 2023-11-28T13:00:54.995 INFO:teuthology.orchestra.run.smithi061.stdout:48 2023-11-28T13:00:54.995 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46, 48] 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> import glob 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> import re 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> import os 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> import subprocess 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> def _find_admin_socket(client_name): 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> files = glob.glob(asok_path) 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-28T13:00:54.996 DEBUG:teuthology.orchestra.run.smithi061:> 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> # Given a non-glob path, it better be there 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> if "*" not in asok_path: 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> assert(len(files) == 1) 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> return files[0] 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> for f in files: 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> contents = proc_f.read() 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> if mountpoint in contents: 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> return f 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> 2023-11-28T13:00:54.997 DEBUG:teuthology.orchestra.run.smithi061:> print(_find_admin_socket("client.3")) 2023-11-28T13:00:54.998 DEBUG:teuthology.orchestra.run.smithi061:> ' 2023-11-28T13:00:55.119 INFO:teuthology.orchestra.run.smithi061.stdout:/var/run/ceph/ceph-client.3.110787.asok 2023-11-28T13:00:55.302 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.110787.asok 2023-11-28T13:00:55.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.302 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.110787.asok status 2023-11-28T13:00:55.464 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-11-28T13:00:55.464 INFO:teuthology.orchestra.run.smithi061.stdout: "metadata": { 2023-11-28T13:00:55.465 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T13:00:55.465 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T13:00:55.465 INFO:teuthology.orchestra.run.smithi061.stdout: "entity_id": "3", 2023-11-28T13:00:55.465 INFO:teuthology.orchestra.run.smithi061.stdout: "hostname": "smithi061", 2023-11-28T13:00:55.465 INFO:teuthology.orchestra.run.smithi061.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-28T13:00:55.465 INFO:teuthology.orchestra.run.smithi061.stdout: "pid": "110787", 2023-11-28T13:00:55.466 INFO:teuthology.orchestra.run.smithi061.stdout: "root": "/" 2023-11-28T13:00:55.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-11-28T13:00:55.466 INFO:teuthology.orchestra.run.smithi061.stdout: "dentry_count": 0, 2023-11-28T13:00:55.466 INFO:teuthology.orchestra.run.smithi061.stdout: "dentry_pinned_count": 0, 2023-11-28T13:00:55.466 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 4714, 2023-11-28T13:00:55.466 INFO:teuthology.orchestra.run.smithi061.stdout: "inst": { 2023-11-28T13:00:55.466 INFO:teuthology.orchestra.run.smithi061.stdout: "name": { 2023-11-28T13:00:55.466 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "client", 2023-11-28T13:00:55.466 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 4714 2023-11-28T13:00:55.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": { 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "192.168.0.2:0", 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3131555553 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": { 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "192.168.0.2:0", 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3131555553 2023-11-28T13:00:55.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-11-28T13:00:55.468 INFO:teuthology.orchestra.run.smithi061.stdout: "inst_str": "client.4714 192.168.0.2:0/3131555553", 2023-11-28T13:00:55.468 INFO:teuthology.orchestra.run.smithi061.stdout: "addr_str": "192.168.0.2:0/3131555553", 2023-11-28T13:00:55.468 INFO:teuthology.orchestra.run.smithi061.stdout: "inode_count": 1, 2023-11-28T13:00:55.468 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_epoch": 9, 2023-11-28T13:00:55.468 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_epoch": 25, 2023-11-28T13:00:55.468 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_epoch_barrier": 0, 2023-11-28T13:00:55.468 INFO:teuthology.orchestra.run.smithi061.stdout: "blocklisted": false, 2023-11-28T13:00:55.468 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs" 2023-11-28T13:00:55.468 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-11-28T13:00:55.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.475 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2023-11-28T13:00:55.475 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:55.498 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2023-11-28T13:00:55.499 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:55.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.499 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:00:55.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.573 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2023-11-28T13:00:55.573 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:55.634 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2023-11-28T13:00:55.634 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:55.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.634 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T13:00:55.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.705 DEBUG:teuthology.orchestra.run.smithi061:> stat --file-system '--printf=%T 2023-11-28T13:00:55.705 DEBUG:teuthology.orchestra.run.smithi061:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:55.723 INFO:teuthology.orchestra.run.smithi061.stdout:fuseblk 2023-11-28T13:00:55.723 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:55.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.723 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T13:00:55.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.792 DEBUG:teuthology.orchestra.run.smithi061:> stat --file-system '--printf=%T 2023-11-28T13:00:55.792 DEBUG:teuthology.orchestra.run.smithi061:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:55.848 INFO:teuthology.orchestra.run.smithi061.stdout:fuseblk 2023-11-28T13:00:55.849 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:55.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:00:55.849 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T13:00:55.916 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T13:00:55.935 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:56.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.074+0000 7f3cfde17700 1 Processor -- start 2023-11-28T13:00:56.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.074+0000 7f3cfde17700 1 -- start start 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cfde17700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3cf80cd620 0x7f3cf815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cfde17700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf80cc860 con 0x7f3cf8158ad0 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cfde17700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf80ccb40 con 0x7f3cf8071900 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cfde17700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf80cce20 con 0x7f3cf80cd620 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf77fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f3cf8158ad0 0x7f3cf80711f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:45948/0 (socket says 172.21.15.6:45948) 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf77fe700 1 -- 172.21.15.6:0/1259897461 learned_addr learned my addr 172.21.15.6:0/1259897461 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf7fff700 1 --2- 172.21.15.6:0/1259897461 >> v2:172.21.15.61:3300/0 conn(0x7f3cf80cd620 0x7f3cf815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484700932 0 0) 0x7f3cf80cc860 con 0x7f3cf8158ad0 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce00035b0 con 0x7f3cf8158ad0 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1984602469 0 0) 0x7f3ce00035b0 con 0x7f3cf8158ad0 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 >> v2:172.21.15.61:3300/0 conn(0x7f3cf80cd620 msgr2=0x7f3cf815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf67fc700 1 --2- 172.21.15.6:0/1259897461 >> v2:172.21.15.61:3300/0 conn(0x7f3cf80cd620 0x7f3cf815cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 >> v1:172.21.15.61:6789/0 conn(0x7f3cf8071900 legacy=0x7f3cf815c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80cd100 con 0x7f3cf8158ad0 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.075+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f3ce8001670 con 0x7f3cf8158ad0 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 >> v1:172.21.15.6:6789/0 conn(0x7f3cf8158ad0 legacy=0x7f3cf80711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf67fc700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ce0003a00 0x7f3ce0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf67fc700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ce0004850 0x7f3ce0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf67fc700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ce0005200 0x7f3ce000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80cce20 con 0x7f3ce0005200 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce000c080 con 0x7f3ce0003a00 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce000d350 con 0x7f3ce0004850 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf7fff700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ce0005200 0x7f3ce000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf77fe700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ce0003a00 0x7f3ce0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf6ffd700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ce0004850 0x7f3ce0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf7fff700 1 -- 172.21.15.6:0/1259897461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ce0004850 msgr2=0x7f3ce0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf7fff700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ce0004850 0x7f3ce0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf7fff700 1 -- 172.21.15.6:0/1259897461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ce0003a00 msgr2=0x7f3ce0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf7fff700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ce0003a00 0x7f3ce0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf7fff700 1 -- 172.21.15.6:0/1259897461 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce00035b0 con 0x7f3ce0005200 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf77fe700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ce0003a00 0x7f3ce0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf7fff700 1 -- 172.21.15.6:0/1259897461 reap_dead start 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf7fff700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ce0005200 0x7f3ce000ba10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009fc0 tx=0x7f3ce0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af99e8bc84b3edb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0003a20 con 0x7f3ce0005200 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf000f040 con 0x7f3ce0005200 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.076+0000 7f3cf67fc700 1 -- 172.21.15.6:0/1259897461 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0003de0 con 0x7f3ce0005200 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.077+0000 7f3cfde17700 1 -- 172.21.15.6:0/1259897461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ce0005200 msgr2=0x7f3ce000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.077+0000 7f3cfde17700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ce0005200 0x7f3ce000ba10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009fc0 tx=0x7f3ce0005880 comp rx=0 tx=0).stop 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.077+0000 7f3cfde17700 1 -- 172.21.15.6:0/1259897461 shutdown_connections 2023-11-28T13:00:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.077+0000 7f3cfde17700 1 --2- 172.21.15.6:0/1259897461 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ce0005200 0x7f3ce000ba10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.077+0000 7f3cfde17700 1 -- 172.21.15.6:0/1259897461 >> 172.21.15.6:0/1259897461 conn(0x7f3cf80ce9e0 msgr2=0x7f3cf80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.077+0000 7f3cfde17700 1 -- 172.21.15.6:0/1259897461 shutdown_connections 2023-11-28T13:00:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.077+0000 7f3cfde17700 1 -- 172.21.15.6:0/1259897461 wait complete. 2023-11-28T13:00:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.078+0000 7f3cfde17700 1 Processor -- start 2023-11-28T13:00:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.079+0000 7f3cfde17700 1 -- start start 2023-11-28T13:00:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.079+0000 7f3cfde17700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ce0005200 0x7f3cf80c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.079+0000 7f3cfde17700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cf80afd70 0x7f3cf806ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.079+0000 7f3cfde17700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cf806b090 0x7f3cf80c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.079+0000 7f3cfde17700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80b2940 con 0x7f3cf806b090 2023-11-28T13:00:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.079+0000 7f3cfde17700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80c53f0 con 0x7f3cf80afd70 2023-11-28T13:00:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.079+0000 7f3cfde17700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80c8ce0 con 0x7f3ce0005200 2023-11-28T13:00:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.079+0000 7f3cf77fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ce0005200 0x7f3cf80c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.079+0000 7f3cf7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cf806b090 0x7f3cf80c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.079+0000 7f3cf6ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cf80afd70 0x7f3cf806ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cf77fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ce0005200 0x7f3cf80c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:48344/0 (socket says 172.21.15.6:48344) 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cf77fe700 1 -- 172.21.15.6:0/945748938 learned_addr learned my addr 172.21.15.6:0/945748938 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cf6ffd700 1 -- 172.21.15.6:0/945748938 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ce0005200 msgr2=0x7f3cf80c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cf6ffd700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ce0005200 0x7f3cf80c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cf6ffd700 1 -- 172.21.15.6:0/945748938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cf806b090 msgr2=0x7f3cf80c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cf6ffd700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cf806b090 0x7f3cf80c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cf6ffd700 1 -- 172.21.15.6:0/945748938 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf8158ea0 con 0x7f3cf80afd70 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cf7fff700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cf806b090 0x7f3cf80c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cf6ffd700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cf80afd70 0x7f3cf806ab50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3cec007fd0 tx=0x7f3cec00f8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cdffff700 1 -- 172.21.15.6:0/945748938 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec005320 con 0x7f3cf80afd70 2023-11-28T13:00:56.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cdffff700 1 -- 172.21.15.6:0/945748938 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec005480 con 0x7f3cf80afd70 2023-11-28T13:00:56.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cdffff700 1 -- 172.21.15.6:0/945748938 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec006aa0 con 0x7f3cf80afd70 2023-11-28T13:00:56.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cfde17700 1 -- 172.21.15.6:0/945748938 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf815d610 con 0x7f3cf80afd70 2023-11-28T13:00:56.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.080+0000 7f3cfde17700 1 -- 172.21.15.6:0/945748938 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf8159030 con 0x7f3cf80afd70 2023-11-28T13:00:56.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.081+0000 7f3cfde17700 1 -- 172.21.15.6:0/945748938 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf8068570 con 0x7f3cf80afd70 2023-11-28T13:00:56.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.081+0000 7f3cdffff700 1 -- 172.21.15.6:0/945748938 <== mon.1 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3cec006c00 con 0x7f3cf80afd70 2023-11-28T13:00:56.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.082+0000 7f3cdffff700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ce40682f0 0x7f3ce406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.082+0000 7f3cdffff700 1 -- 172.21.15.6:0/945748938 <== mon.1 v2:172.21.15.61:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3cec085390 con 0x7f3cf80afd70 2023-11-28T13:00:56.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.082+0000 7f3cf77fe700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ce40682f0 0x7f3ce406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.082+0000 7f3cf77fe700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ce40682f0 0x7f3ce406a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8000fb0 tx=0x7f3ce80011c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.087+0000 7f3cdffff700 1 -- 172.21.15.6:0/945748938 <== mon.1 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3cec051900 con 0x7f3cf80afd70 2023-11-28T13:00:56.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.282+0000 7f3cfde17700 1 -- 172.21.15.6:0/945748938 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cf806cef0 con 0x7f3cf80afd70 2023-11-28T13:00:56.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.284+0000 7f3cdffff700 1 -- 172.21.15.6:0/945748938 <== mon.1 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f3cec054f20 con 0x7f3cf80afd70 2023-11-28T13:00:56.284 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:56.285 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:56.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 -- 172.21.15.6:0/945748938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ce40682f0 msgr2=0x7f3ce406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ce40682f0 0x7f3ce406a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8000fb0 tx=0x7f3ce80011c0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 -- 172.21.15.6:0/945748938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cf80afd70 msgr2=0x7f3cf806ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cf80afd70 0x7f3cf806ab50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3cec007fd0 tx=0x7f3cec00f8a0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 -- 172.21.15.6:0/945748938 shutdown_connections 2023-11-28T13:00:56.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3cf806b090 0x7f3cf80c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ce40682f0 0x7f3ce406a7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ce0005200 0x7f3cf80c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 --2- 172.21.15.6:0/945748938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3cf80afd70 0x7f3cf806ab50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 -- 172.21.15.6:0/945748938 >> 172.21.15.6:0/945748938 conn(0x7f3cf80ce9e0 msgr2=0x7f3cf80c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 -- 172.21.15.6:0/945748938 shutdown_connections 2023-11-28T13:00:56.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.286+0000 7f3cfde17700 1 -- 172.21.15.6:0/945748938 wait complete. 2023-11-28T13:00:56.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:00:56.299 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:56.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.426+0000 7fc2be165700 1 Processor -- start 2023-11-28T13:00:56.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.426+0000 7fc2be165700 1 -- start start 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.427+0000 7fc2be165700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 0x7fc2b80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.427+0000 7fc2be165700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc2b80b1d40 0x7fc2b80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.427+0000 7fc2be165700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b815e040 con 0x7fc2b815dbf0 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.427+0000 7fc2be165700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b80cede0 con 0x7fc2b80b1d40 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.427+0000 7fc2be165700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b80ceff0 con 0x7fc2b8159500 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.427+0000 7fc2b7fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc2b80b1d40 0x7fc2b80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.427+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 0x7fc2b80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.428+0000 7fc2aeffd700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fc2b815dbf0 0x7fc2b80b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:35998/0 (socket says 172.21.15.6:35998) 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.428+0000 7fc2aeffd700 1 -- 172.21.15.6:0/1971399727 learned_addr learned my addr 172.21.15.6:0/1971399727 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.428+0000 7fc2b77fe700 1 -- 172.21.15.6:0/1971399727 >> v2:172.21.15.61:3300/0 conn(0x7fc2b80b1d40 msgr2=0x7fc2b80ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.428+0000 7fc2b77fe700 1 --2- 172.21.15.6:0/1971399727 >> v2:172.21.15.61:3300/0 conn(0x7fc2b80b1d40 0x7fc2b80ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.428+0000 7fc2b77fe700 1 -- 172.21.15.6:0/1971399727 >> v1:172.21.15.61:6789/0 conn(0x7fc2b815dbf0 legacy=0x7fc2b80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.428+0000 7fc2b77fe700 1 -- 172.21.15.6:0/1971399727 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80cf2d0 con 0x7fc2b8159500 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.428+0000 7fc2b77fe700 1 --2- 172.21.15.6:0/1971399727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 0x7fc2b80cd140 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009840 tx=0x7fc2a8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d4c04b8de88d25 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.428+0000 7fc2b6ffd700 1 -- 172.21.15.6:0/1971399727 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8011830 con 0x7fc2b8159500 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.428+0000 7fc2b6ffd700 1 -- 172.21.15.6:0/1971399727 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a8011990 con 0x7fc2b8159500 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.428+0000 7fc2b6ffd700 1 -- 172.21.15.6:0/1971399727 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8011d20 con 0x7fc2b8159500 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.429+0000 7fc2be165700 1 -- 172.21.15.6:0/1971399727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 msgr2=0x7fc2b80cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.429+0000 7fc2be165700 1 --2- 172.21.15.6:0/1971399727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 0x7fc2b80cd140 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009840 tx=0x7fc2a8009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.429+0000 7fc2be165700 1 -- 172.21.15.6:0/1971399727 shutdown_connections 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.429+0000 7fc2be165700 1 --2- 172.21.15.6:0/1971399727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 0x7fc2b80cd140 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.429+0000 7fc2be165700 1 --2- 172.21.15.6:0/1971399727 >> v2:172.21.15.61:3300/0 conn(0x7fc2b80b1d40 0x7fc2b80ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.429+0000 7fc2be165700 1 -- 172.21.15.6:0/1971399727 >> 172.21.15.6:0/1971399727 conn(0x7fc2b80cc630 msgr2=0x7fc2b80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.429+0000 7fc2be165700 1 -- 172.21.15.6:0/1971399727 shutdown_connections 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.429+0000 7fc2be165700 1 -- 172.21.15.6:0/1971399727 wait complete. 2023-11-28T13:00:56.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.430+0000 7fc2be165700 1 Processor -- start 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.430+0000 7fc2be165700 1 -- start start 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2be165700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2b80b1d40 0x7fc2b81549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2be165700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 0x7fc2b80b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2be165700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2b815dbf0 0x7fc2b80ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2be165700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8154f20 con 0x7fc2b80b1d40 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2be165700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b80ae780 con 0x7fc2b815dbf0 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2be165700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b80aba80 con 0x7fc2b8159500 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2b80b1d40 0x7fc2b81549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2b80b1d40 0x7fc2b81549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58652/0 (socket says 172.21.15.6:58652) 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2b77fe700 1 -- 172.21.15.6:0/1252792578 learned_addr learned my addr 172.21.15.6:0/1252792578 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2aeffd700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 0x7fc2b80b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2b7fff700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2b815dbf0 0x7fc2b80ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2b77fe700 1 -- 172.21.15.6:0/1252792578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 msgr2=0x7fc2b80b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2b77fe700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 0x7fc2b80b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2b77fe700 1 -- 172.21.15.6:0/1252792578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2b815dbf0 msgr2=0x7fc2b80ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2b77fe700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2b815dbf0 0x7fc2b80ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.431+0000 7fc2b77fe700 1 -- 172.21.15.6:0/1252792578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a80094a0 con 0x7fc2b80b1d40 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.432+0000 7fc2b77fe700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2b80b1d40 0x7fc2b81549e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80090b0 tx=0x7fc2a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.432+0000 7fc2b4ff9700 1 -- 172.21.15.6:0/1252792578 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80036a0 con 0x7fc2b80b1d40 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.432+0000 7fc2b4ff9700 1 -- 172.21.15.6:0/1252792578 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a801f070 con 0x7fc2b80b1d40 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.432+0000 7fc2be165700 1 -- 172.21.15.6:0/1252792578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b815d3f0 con 0x7fc2b80b1d40 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.432+0000 7fc2b4ff9700 1 -- 172.21.15.6:0/1252792578 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8003a60 con 0x7fc2b80b1d40 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.432+0000 7fc2be165700 1 -- 172.21.15.6:0/1252792578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b815d110 con 0x7fc2b80b1d40 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.433+0000 7fc2be165700 1 -- 172.21.15.6:0/1252792578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b815d6d0 con 0x7fc2b80b1d40 2023-11-28T13:00:56.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.433+0000 7fc2b4ff9700 1 -- 172.21.15.6:0/1252792578 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc2a800f050 con 0x7fc2b80b1d40 2023-11-28T13:00:56.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.434+0000 7fc2b4ff9700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2980682f0 0x7fc29806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.434+0000 7fc2b4ff9700 1 -- 172.21.15.6:0/1252792578 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc2a8021070 con 0x7fc2b80b1d40 2023-11-28T13:00:56.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.434+0000 7fc2aeffd700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2980682f0 0x7fc29806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.434+0000 7fc2aeffd700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2980682f0 0x7fc29806a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80ca2a0 tx=0x7fc2a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.438+0000 7fc2b4ff9700 1 -- 172.21.15.6:0/1252792578 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc2a8053c30 con 0x7fc2b80b1d40 2023-11-28T13:00:56.573 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:00:56.576 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:00:56.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.634+0000 7fc2be165700 1 -- 172.21.15.6:0/1252792578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc2b80739f0 con 0x7fc2b80b1d40 2023-11-28T13:00:56.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.635+0000 7fc2b4ff9700 1 -- 172.21.15.6:0/1252792578 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc2a8015020 con 0x7fc2b80b1d40 2023-11-28T13:00:56.635 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:00:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.636+0000 7fc2be165700 1 -- 172.21.15.6:0/1252792578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2980682f0 msgr2=0x7fc29806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.636+0000 7fc2be165700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2980682f0 0x7fc29806a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80ca2a0 tx=0x7fc2a0009040 comp rx=0 tx=0).stop 2023-11-28T13:00:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.636+0000 7fc2be165700 1 -- 172.21.15.6:0/1252792578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2b80b1d40 msgr2=0x7fc2b81549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.636+0000 7fc2be165700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2b80b1d40 0x7fc2b81549e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80090b0 tx=0x7fc2a8004750 comp rx=0 tx=0).stop 2023-11-28T13:00:56.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.636+0000 7fc2be165700 1 -- 172.21.15.6:0/1252792578 shutdown_connections 2023-11-28T13:00:56.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.636+0000 7fc2be165700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2b815dbf0 0x7fc2b80ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.636+0000 7fc2be165700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2b80b1d40 0x7fc2b81549e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.636+0000 7fc2be165700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2980682f0 0x7fc29806a7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.636+0000 7fc2be165700 1 --2- 172.21.15.6:0/1252792578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2b8159500 0x7fc2b80b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.636+0000 7fc2be165700 1 -- 172.21.15.6:0/1252792578 >> 172.21.15.6:0/1252792578 conn(0x7fc2b80cc630 msgr2=0x7fc2b80cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.637+0000 7fc2be165700 1 -- 172.21.15.6:0/1252792578 shutdown_connections 2023-11-28T13:00:56.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.637+0000 7fc2be165700 1 -- 172.21.15.6:0/1252792578 wait complete. 2023-11-28T13:00:56.649 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:56.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.816+0000 7fda9f356700 1 Processor -- start 2023-11-28T13:00:56.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.816+0000 7fda9f356700 1 -- start start 2023-11-28T13:00:56.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.817+0000 7fda9f356700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fda9815a490 0x7fda9806f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.817+0000 7fda9f356700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda980b2150 0x7fda9806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.817+0000 7fda9f356700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fda9806bcd0 0x7fda9806db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.817+0000 7fda9f356700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9815a860 con 0x7fda9806bcd0 2023-11-28T13:00:56.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.817+0000 7fda9f356700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9806b510 con 0x7fda9815a490 2023-11-28T13:00:56.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.817+0000 7fda9f356700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda980c9a80 con 0x7fda980b2150 2023-11-28T13:00:56.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.817+0000 7fda9d8f3700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fda9806bcd0 0x7fda9806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.817+0000 7fda9d8f3700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fda9806bcd0 0x7fda9806db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58664/0 (socket says 172.21.15.6:58664) 2023-11-28T13:00:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.817+0000 7fda9d8f3700 1 -- 172.21.15.6:0/3942439536 learned_addr learned my addr 172.21.15.6:0/3942439536 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9d0f2700 1 --2- 172.21.15.6:0/3942439536 >> v2:172.21.15.61:3300/0 conn(0x7fda9815a490 0x7fda9806f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9d8f3700 1 -- 172.21.15.6:0/3942439536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda980b2150 msgr2=0x7fda9806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9c8f1700 1 --2- 172.21.15.6:0/3942439536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda980b2150 0x7fda9806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9d8f3700 1 --2- 172.21.15.6:0/3942439536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda980b2150 0x7fda9806d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9d8f3700 1 -- 172.21.15.6:0/3942439536 >> v2:172.21.15.61:3300/0 conn(0x7fda9815a490 msgr2=0x7fda9806f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9d8f3700 1 --2- 172.21.15.6:0/3942439536 >> v2:172.21.15.61:3300/0 conn(0x7fda9815a490 0x7fda9806f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9d8f3700 1 -- 172.21.15.6:0/3942439536 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9806b230 con 0x7fda9806bcd0 2023-11-28T13:00:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9d0f2700 1 --2- 172.21.15.6:0/3942439536 >> v2:172.21.15.61:3300/0 conn(0x7fda9815a490 0x7fda9806f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9d8f3700 1 --2- 172.21.15.6:0/3942439536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9806bcd0 0x7fda9806db70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda980cf1b0 tx=0x7fda9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f3f4608492ae99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda8ffff700 1 -- 172.21.15.6:0/3942439536 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9000e070 con 0x7fda9806bcd0 2023-11-28T13:00:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda8ffff700 1 -- 172.21.15.6:0/3942439536 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9000b9c0 con 0x7fda9806bcd0 2023-11-28T13:00:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda8ffff700 1 -- 172.21.15.6:0/3942439536 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9000bd70 con 0x7fda9806bcd0 2023-11-28T13:00:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9f356700 1 -- 172.21.15.6:0/3942439536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9806bcd0 msgr2=0x7fda9806db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.818+0000 7fda9f356700 1 --2- 172.21.15.6:0/3942439536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9806bcd0 0x7fda9806db70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda980cf1b0 tx=0x7fda9000b330 comp rx=0 tx=0).stop 2023-11-28T13:00:56.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.819+0000 7fda9f356700 1 -- 172.21.15.6:0/3942439536 shutdown_connections 2023-11-28T13:00:56.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.819+0000 7fda9f356700 1 --2- 172.21.15.6:0/3942439536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9806bcd0 0x7fda9806db70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.819+0000 7fda9f356700 1 --2- 172.21.15.6:0/3942439536 >> v2:172.21.15.61:3300/0 conn(0x7fda9815a490 0x7fda9806f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.819+0000 7fda9f356700 1 --2- 172.21.15.6:0/3942439536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda980b2150 0x7fda9806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.819+0000 7fda9f356700 1 -- 172.21.15.6:0/3942439536 >> 172.21.15.6:0/3942439536 conn(0x7fda980b0df0 msgr2=0x7fda980b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:56.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.819+0000 7fda9f356700 1 -- 172.21.15.6:0/3942439536 shutdown_connections 2023-11-28T13:00:56.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.819+0000 7fda9f356700 1 -- 172.21.15.6:0/3942439536 wait complete. 2023-11-28T13:00:56.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.820+0000 7fda9f356700 1 Processor -- start 2023-11-28T13:00:56.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.820+0000 7fda9f356700 1 -- start start 2023-11-28T13:00:56.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9f356700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda9806bcd0 0x7fda981f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9f356700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda980b2150 0x7fda9814d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9f356700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9815a490 0x7fda980c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9f356700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda980cced0 con 0x7fda9815a490 2023-11-28T13:00:56.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9f356700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda981f28b0 con 0x7fda980b2150 2023-11-28T13:00:56.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9f356700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda980ac9a0 con 0x7fda9806bcd0 2023-11-28T13:00:56.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9d8f3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9815a490 0x7fda980c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9d8f3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9815a490 0x7fda980c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58670/0 (socket says 172.21.15.6:58670) 2023-11-28T13:00:56.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9d8f3700 1 -- 172.21.15.6:0/3889728571 learned_addr learned my addr 172.21.15.6:0/3889728571 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:56.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9c8f1700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda980b2150 0x7fda9814d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9d0f2700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda9806bcd0 0x7fda981f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9d8f3700 1 -- 172.21.15.6:0/3889728571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda9806bcd0 msgr2=0x7fda981f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9d8f3700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda9806bcd0 0x7fda981f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9d8f3700 1 -- 172.21.15.6:0/3889728571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda980b2150 msgr2=0x7fda9814d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:56.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9d8f3700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda980b2150 0x7fda9814d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:56.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9d8f3700 1 -- 172.21.15.6:0/3889728571 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90009b30 con 0x7fda9815a490 2023-11-28T13:00:56.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9d8f3700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9815a490 0x7fda980c8cb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fda9000b370 tx=0x7fda90004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda8dffb700 1 -- 172.21.15.6:0/3889728571 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90004310 con 0x7fda9815a490 2023-11-28T13:00:56.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda8dffb700 1 -- 172.21.15.6:0/3889728571 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90004470 con 0x7fda9815a490 2023-11-28T13:00:56.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda9f356700 1 -- 172.21.15.6:0/3889728571 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda980ca000 con 0x7fda9815a490 2023-11-28T13:00:56.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.821+0000 7fda8dffb700 1 -- 172.21.15.6:0/3889728571 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90004820 con 0x7fda9815a490 2023-11-28T13:00:56.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.822+0000 7fda9f356700 1 -- 172.21.15.6:0/3889728571 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda980c9d20 con 0x7fda9815a490 2023-11-28T13:00:56.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.823+0000 7fda9f356700 1 -- 172.21.15.6:0/3889728571 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9806ae90 con 0x7fda9815a490 2023-11-28T13:00:56.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.823+0000 7fda8dffb700 1 -- 172.21.15.6:0/3889728571 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fda9002b070 con 0x7fda9815a490 2023-11-28T13:00:56.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.824+0000 7fda8dffb700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda7c0682f0 0x7fda7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:56.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.824+0000 7fda8dffb700 1 -- 172.21.15.6:0/3889728571 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda900874c0 con 0x7fda9815a490 2023-11-28T13:00:56.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.824+0000 7fda9d0f2700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda7c0682f0 0x7fda7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:56.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.824+0000 7fda9d0f2700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda7c0682f0 0x7fda7c06a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda88006fd0 tx=0x7fda88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:56.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:56.828+0000 7fda8dffb700 1 -- 172.21.15.6:0/3889728571 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fda90053ae0 con 0x7fda9815a490 2023-11-28T13:00:57.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.021+0000 7fda9f356700 1 -- 172.21.15.6:0/3889728571 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda980739f0 con 0x7fda9815a490 2023-11-28T13:00:57.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.022+0000 7fda8dffb700 1 -- 172.21.15.6:0/3889728571 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda90057100 con 0x7fda9815a490 2023-11-28T13:00:57.023 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:00:57.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 -- 172.21.15.6:0/3889728571 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda7c0682f0 msgr2=0x7fda7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda7c0682f0 0x7fda7c06a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda88006fd0 tx=0x7fda88008040 comp rx=0 tx=0).stop 2023-11-28T13:00:57.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 -- 172.21.15.6:0/3889728571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9815a490 msgr2=0x7fda980c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9815a490 0x7fda980c8cb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fda9000b370 tx=0x7fda90004ad0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 -- 172.21.15.6:0/3889728571 shutdown_connections 2023-11-28T13:00:57.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda9815a490 0x7fda980c8cb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda7c0682f0 0x7fda7c06a7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda9806bcd0 0x7fda981f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 --2- 172.21.15.6:0/3889728571 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda980b2150 0x7fda9814d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 -- 172.21.15.6:0/3889728571 >> 172.21.15.6:0/3889728571 conn(0x7fda980b0df0 msgr2=0x7fda980b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 -- 172.21.15.6:0/3889728571 shutdown_connections 2023-11-28T13:00:57.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.024+0000 7fda9f356700 1 -- 172.21.15.6:0/3889728571 wait complete. 2023-11-28T13:00:57.037 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.165+0000 7fef47fb7700 1 Processor -- start 2023-11-28T13:00:57.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.165+0000 7fef47fb7700 1 -- start start 2023-11-28T13:00:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef47fb7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fef400cf5f0 0x7fef400caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef47fb7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef400cb3e0 0x7fef4006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef47fb7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef400ce550 con 0x7fef400b1630 2023-11-28T13:00:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef47fb7700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef400ce830 con 0x7fef400cf5f0 2023-11-28T13:00:57.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef47fb7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef400ceb10 con 0x7fef400cb3e0 2023-11-28T13:00:57.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef45552700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fef400cf5f0 0x7fef400caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef45552700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fef400cf5f0 0x7fef400caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58676/0 (socket says 172.21.15.6:58676) 2023-11-28T13:00:57.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef45552700 1 -- 172.21.15.6:0/2436230814 learned_addr learned my addr 172.21.15.6:0/2436230814 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:57.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef46554700 1 --2- 172.21.15.6:0/2436230814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef400cb3e0 0x7fef4006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef45552700 1 -- 172.21.15.6:0/2436230814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef400cb3e0 msgr2=0x7fef4006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef45552700 1 --2- 172.21.15.6:0/2436230814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef400cb3e0 0x7fef4006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef45552700 1 -- 172.21.15.6:0/2436230814 >> v1:172.21.15.61:6789/0 conn(0x7fef400b1630 legacy=0x7fef400b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef45552700 1 -- 172.21.15.6:0/2436230814 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400cedf0 con 0x7fef400cf5f0 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.166+0000 7fef45552700 1 --2- 172.21.15.6:0/2436230814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cf5f0 0x7fef400caea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef30009920 tx=0x7fef30009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a134e4d6ac35ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef44d51700 1 -- 172.21.15.6:0/2436230814 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30004230 con 0x7fef400cf5f0 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef44d51700 1 -- 172.21.15.6:0/2436230814 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef30004390 con 0x7fef400cf5f0 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef44d51700 1 -- 172.21.15.6:0/2436230814 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30009e70 con 0x7fef400cf5f0 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef47fb7700 1 -- 172.21.15.6:0/2436230814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cf5f0 msgr2=0x7fef400caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef47fb7700 1 --2- 172.21.15.6:0/2436230814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cf5f0 0x7fef400caea0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef30009920 tx=0x7fef30009c30 comp rx=0 tx=0).stop 2023-11-28T13:00:57.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef47fb7700 1 -- 172.21.15.6:0/2436230814 shutdown_connections 2023-11-28T13:00:57.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef47fb7700 1 --2- 172.21.15.6:0/2436230814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef400cb3e0 0x7fef4006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef47fb7700 1 --2- 172.21.15.6:0/2436230814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cf5f0 0x7fef400caea0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef47fb7700 1 -- 172.21.15.6:0/2436230814 >> 172.21.15.6:0/2436230814 conn(0x7fef400cc630 msgr2=0x7fef400cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef47fb7700 1 -- 172.21.15.6:0/2436230814 shutdown_connections 2023-11-28T13:00:57.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.167+0000 7fef47fb7700 1 -- 172.21.15.6:0/2436230814 wait complete. 2023-11-28T13:00:57.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.169+0000 7fef47fb7700 1 Processor -- start 2023-11-28T13:00:57.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.169+0000 7fef47fb7700 1 -- start start 2023-11-28T13:00:57.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef47fb7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef400b1630 0x7fef400c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef47fb7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cb3e0 0x7fef4015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef47fb7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef400cf5f0 0x7fef40156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef47fb7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40157a00 con 0x7fef400cb3e0 2023-11-28T13:00:57.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef47fb7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40158de0 con 0x7fef400cf5f0 2023-11-28T13:00:57.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef47fb7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4015b030 con 0x7fef400b1630 2023-11-28T13:00:57.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef45552700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cb3e0 0x7fef4015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef45552700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cb3e0 0x7fef4015a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58682/0 (socket says 172.21.15.6:58682) 2023-11-28T13:00:57.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef45552700 1 -- 172.21.15.6:0/3527730285 learned_addr learned my addr 172.21.15.6:0/3527730285 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:57.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef45d53700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef400b1630 0x7fef400c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef46554700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef400cf5f0 0x7fef40156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef45552700 1 -- 172.21.15.6:0/3527730285 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef400b1630 msgr2=0x7fef400c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef45552700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef400b1630 0x7fef400c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef45552700 1 -- 172.21.15.6:0/3527730285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef400cf5f0 msgr2=0x7fef40156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef45552700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef400cf5f0 0x7fef40156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef45552700 1 -- 172.21.15.6:0/3527730285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30009550 con 0x7fef400cb3e0 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.170+0000 7fef45552700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cb3e0 0x7fef4015a9a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef300098f0 tx=0x7fef30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.171+0000 7fef367fc700 1 -- 172.21.15.6:0/3527730285 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3000be30 con 0x7fef400cb3e0 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.171+0000 7fef367fc700 1 -- 172.21.15.6:0/3527730285 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef300036a0 con 0x7fef400cb3e0 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.171+0000 7fef47fb7700 1 -- 172.21.15.6:0/3527730285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef400ce140 con 0x7fef400cb3e0 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.171+0000 7fef367fc700 1 -- 172.21.15.6:0/3527730285 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30003a50 con 0x7fef400cb3e0 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.171+0000 7fef47fb7700 1 -- 172.21.15.6:0/3527730285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef400cf0d0 con 0x7fef400cb3e0 2023-11-28T13:00:57.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.172+0000 7fef47fb7700 1 -- 172.21.15.6:0/3527730285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef400ce2d0 con 0x7fef400cb3e0 2023-11-28T13:00:57.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.173+0000 7fef367fc700 1 -- 172.21.15.6:0/3527730285 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fef30017020 con 0x7fef400cb3e0 2023-11-28T13:00:57.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.173+0000 7fef367fc700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef240682f0 0x7fef2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.173+0000 7fef367fc700 1 -- 172.21.15.6:0/3527730285 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fef30087a00 con 0x7fef400cb3e0 2023-11-28T13:00:57.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.173+0000 7fef45d53700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef240682f0 0x7fef2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.173+0000 7fef45d53700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef240682f0 0x7fef2406a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fef3c007fd0 tx=0x7fef3c007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.177+0000 7fef367fc700 1 -- 172.21.15.6:0/3527730285 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fef30054010 con 0x7fef400cb3e0 2023-11-28T13:00:57.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.373+0000 7fef47fb7700 1 -- 172.21.15.6:0/3527730285 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef400739f0 con 0x7fef400cb3e0 2023-11-28T13:00:57.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.373+0000 7fef367fc700 1 -- 172.21.15.6:0/3527730285 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef30057630 con 0x7fef400cb3e0 2023-11-28T13:00:57.374 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:00:57.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 -- 172.21.15.6:0/3527730285 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef240682f0 msgr2=0x7fef2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef240682f0 0x7fef2406a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fef3c007fd0 tx=0x7fef3c007ec0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 -- 172.21.15.6:0/3527730285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cb3e0 msgr2=0x7fef4015a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cb3e0 0x7fef4015a9a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef300098f0 tx=0x7fef30004750 comp rx=0 tx=0).stop 2023-11-28T13:00:57.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 -- 172.21.15.6:0/3527730285 shutdown_connections 2023-11-28T13:00:57.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef400cf5f0 0x7fef40156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef240682f0 0x7fef2406a7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fef400b1630 0x7fef400c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 --2- 172.21.15.6:0/3527730285 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef400cb3e0 0x7fef4015a9a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 -- 172.21.15.6:0/3527730285 >> 172.21.15.6:0/3527730285 conn(0x7fef400cc630 msgr2=0x7fef400b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.375+0000 7fef47fb7700 1 -- 172.21.15.6:0/3527730285 shutdown_connections 2023-11-28T13:00:57.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.376+0000 7fef47fb7700 1 -- 172.21.15.6:0/3527730285 wait complete. 2023-11-28T13:00:57.388 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:57.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.517+0000 7fc2ddbb2700 1 Processor -- start 2023-11-28T13:00:57.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.517+0000 7fc2ddbb2700 1 -- start start 2023-11-28T13:00:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.517+0000 7fc2ddbb2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80cf5f0 0x7fc2d80caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.517+0000 7fc2ddbb2700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d80ce550 con 0x7fc2d80cb3e0 2023-11-28T13:00:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2ddbb2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d80ce830 con 0x7fc2d80b1630 2023-11-28T13:00:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2ddbb2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80cfa60 con 0x7fc2d80cf5f0 2023-11-28T13:00:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d7fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fc2d80cb3e0 0x7fc2d806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46012/0 (socket says 172.21.15.6:46012) 2023-11-28T13:00:57.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d7fff700 1 -- 172.21.15.6:0/2014886403 learned_addr learned my addr 172.21.15.6:0/2014886403 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:57.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d6ffd700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80cf5f0 0x7fc2d80caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151693085 0 0) 0x7fc2d80ce550 con 0x7fc2d80cb3e0 2023-11-28T13:00:57.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2c00035b0 con 0x7fc2d80cb3e0 2023-11-28T13:00:57.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2438874141 0 0) 0x7fc2c00035b0 con 0x7fc2d80cb3e0 2023-11-28T13:00:57.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80cf5f0 msgr2=0x7fc2d80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80cf5f0 0x7fc2d80caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 >> v1:172.21.15.61:6789/0 conn(0x7fc2d80b1630 legacy=0x7fc2d80b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d80ceb10 con 0x7fc2d80cb3e0 2023-11-28T13:00:57.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fc2d00019f0 con 0x7fc2d80cb3e0 2023-11-28T13:00:57.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 >> v1:172.21.15.6:6789/0 conn(0x7fc2d80cb3e0 legacy=0x7fc2d806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80cf5f0 0x7fc2c0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2c0003f70 0x7fc2c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2c0004d90 0x7fc2c0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c000bac0 con 0x7fc2c0004d90 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c000bc00 con 0x7fc2c0003f70 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c000bdc0 con 0x7fc2d80cf5f0 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.518+0000 7fc2d7fff700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2c0004d90 0x7fc2c0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d6ffd700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2c0003f70 0x7fc2c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d77fe700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80cf5f0 0x7fc2c0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d7fff700 1 -- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80cf5f0 msgr2=0x7fc2c0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d7fff700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80cf5f0 0x7fc2c0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d7fff700 1 -- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2c0003f70 msgr2=0x7fc2c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d7fff700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2c0003f70 0x7fc2c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d7fff700 1 -- 172.21.15.6:0/2014886403 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c00035b0 con 0x7fc2c0004d90 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d7fff700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2c0004d90 0x7fc2c0005200 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000a7e0 tx=0x7fc2c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dcf8b78f292804 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000a250 con 0x7fc2c0004d90 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d0012620 con 0x7fc2c0004d90 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.519+0000 7fc2d67fc700 1 -- 172.21.15.6:0/2014886403 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00129d0 con 0x7fc2c0004d90 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.520+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/2014886403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2c0004d90 msgr2=0x7fc2c0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.520+0000 7fc2ddbb2700 1 --2- 172.21.15.6:0/2014886403 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2c0004d90 0x7fc2c0005200 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000a7e0 tx=0x7fc2c0005880 comp rx=0 tx=0).stop 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.520+0000 7fc2d7fff700 1 -- 172.21.15.6:0/2014886403 reap_dead start 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.520+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/2014886403 shutdown_connections 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.520+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/2014886403 >> 172.21.15.6:0/2014886403 conn(0x7fc2d80cc630 msgr2=0x7fc2d80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.520+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/2014886403 shutdown_connections 2023-11-28T13:00:57.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.520+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/2014886403 wait complete. 2023-11-28T13:00:57.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.521+0000 7fc2ddbb2700 1 Processor -- start 2023-11-28T13:00:57.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.521+0000 7fc2ddbb2700 1 -- start start 2023-11-28T13:00:57.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2ddbb2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2d80ad7f0 0x7fc2d80c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2ddbb2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80c90b0 0x7fc2d80adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2ddbb2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2d80ae100 0x7fc2d80ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2ddbb2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d8153c40 con 0x7fc2d80ae100 2023-11-28T13:00:57.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2ddbb2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80b46e0 con 0x7fc2d80ad7f0 2023-11-28T13:00:57.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2ddbb2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80c9f30 con 0x7fc2d80c90b0 2023-11-28T13:00:57.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2d80ae100 0x7fc2d80ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2d80ae100 0x7fc2d80ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58704/0 (socket says 172.21.15.6:58704) 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d7fff700 1 -- 172.21.15.6:0/3083720619 learned_addr learned my addr 172.21.15.6:0/3083720619 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d77fe700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2d80ad7f0 0x7fc2d80c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d6ffd700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80c90b0 0x7fc2d80adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d7fff700 1 -- 172.21.15.6:0/3083720619 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80c90b0 msgr2=0x7fc2d80adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d7fff700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80c90b0 0x7fc2d80adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d7fff700 1 -- 172.21.15.6:0/3083720619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2d80ad7f0 msgr2=0x7fc2d80c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d7fff700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2d80ad7f0 0x7fc2d80c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d7fff700 1 -- 172.21.15.6:0/3083720619 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d80cf0d0 con 0x7fc2d80ae100 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.522+0000 7fc2d7fff700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2d80ae100 0x7fc2d80ac2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000add0 tx=0x7fc2d0001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.523+0000 7fc2bffff700 1 -- 172.21.15.6:0/3083720619 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0012480 con 0x7fc2d80ae100 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.523+0000 7fc2bffff700 1 -- 172.21.15.6:0/3083720619 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d00017f0 con 0x7fc2d80ae100 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.523+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/3083720619 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d80cedf0 con 0x7fc2d80ae100 2023-11-28T13:00:57.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.523+0000 7fc2bffff700 1 -- 172.21.15.6:0/3083720619 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d001e650 con 0x7fc2d80ae100 2023-11-28T13:00:57.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.523+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/3083720619 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d80ce140 con 0x7fc2d80ae100 2023-11-28T13:00:57.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.523+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/3083720619 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d8067150 con 0x7fc2d80ae100 2023-11-28T13:00:57.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.524+0000 7fc2bffff700 1 -- 172.21.15.6:0/3083720619 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc2d0010070 con 0x7fc2d80ae100 2023-11-28T13:00:57.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.524+0000 7fc2bffff700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c40682f0 0x7fc2c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.524+0000 7fc2bffff700 1 -- 172.21.15.6:0/3083720619 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc2d0087bd0 con 0x7fc2d80ae100 2023-11-28T13:00:57.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.524+0000 7fc2d77fe700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c40682f0 0x7fc2c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.525+0000 7fc2d77fe700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c40682f0 0x7fc2c406a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80b20e0 tx=0x7fc2c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.529+0000 7fc2bffff700 1 -- 172.21.15.6:0/3083720619 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc2d0054200 con 0x7fc2d80ae100 2023-11-28T13:00:57.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.720+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/3083720619 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2d80739f0 con 0x7fc2d80ae100 2023-11-28T13:00:57.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.720+0000 7fc2bffff700 1 -- 172.21.15.6:0/3083720619 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc2d0007b60 con 0x7fc2d80ae100 2023-11-28T13:00:57.722 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:00:57.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/3083720619 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c40682f0 msgr2=0x7fc2c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c40682f0 0x7fc2c406a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80b20e0 tx=0x7fc2c8009040 comp rx=0 tx=0).stop 2023-11-28T13:00:57.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/3083720619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2d80ae100 msgr2=0x7fc2d80ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2d80ae100 0x7fc2d80ac2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000add0 tx=0x7fc2d0001cb0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/3083720619 shutdown_connections 2023-11-28T13:00:57.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2d80ae100 0x7fc2d80ac2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2d80ad7f0 0x7fc2d80c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c40682f0 0x7fc2c406a7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 --2- 172.21.15.6:0/3083720619 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2d80c90b0 0x7fc2d80adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/3083720619 >> 172.21.15.6:0/3083720619 conn(0x7fc2d80cc630 msgr2=0x7fc2d80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/3083720619 shutdown_connections 2023-11-28T13:00:57.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.722+0000 7fc2ddbb2700 1 -- 172.21.15.6:0/3083720619 wait complete. 2023-11-28T13:00:57.738 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:00:57.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.866+0000 7f57e640e700 1 Processor -- start 2023-11-28T13:00:57.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.866+0000 7f57e640e700 1 -- start start 2023-11-28T13:00:57.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.866+0000 7f57e640e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f57e00b1630 0x7f57e00cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.866+0000 7f57e640e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 0x7f57e00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.866+0000 7f57e640e700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e00ce550 con 0x7f57e00cb3e0 2023-11-28T13:00:57.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57e640e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e00ce830 con 0x7f57e00b1630 2023-11-28T13:00:57.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57e640e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00ceb10 con 0x7f57e00b1a00 2023-11-28T13:00:57.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57dffff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f57e00b1630 0x7f57e00cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57e49ab700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f57e00cb3e0 0x7f57e0160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46032/0 (socket says 172.21.15.6:46032) 2023-11-28T13:00:57.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57e49ab700 1 -- 172.21.15.6:0/2050161662 learned_addr learned my addr 172.21.15.6:0/2050161662 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:57.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57df7fe700 1 --2- 172.21.15.6:0/2050161662 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 0x7f57e00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57dffff700 1 -- 172.21.15.6:0/2050161662 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 msgr2=0x7f57e00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:57.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57dffff700 1 --2- 172.21.15.6:0/2050161662 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 0x7f57e00caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57dffff700 1 -- 172.21.15.6:0/2050161662 >> v1:172.21.15.6:6789/0 conn(0x7f57e00cb3e0 legacy=0x7f57e0160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57dffff700 1 -- 172.21.15.6:0/2050161662 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00cedf0 con 0x7f57e00b1630 2023-11-28T13:00:57.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57df7fe700 1 --2- 172.21.15.6:0/2050161662 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 0x7f57e00caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:57.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57dffff700 1 --2- 172.21.15.6:0/2050161662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00b1630 0x7f57e00cf5f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009840 tx=0x7f57d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f35b6a1820f4e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57deffd700 1 -- 172.21.15.6:0/2050161662 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d0011830 con 0x7f57e00b1630 2023-11-28T13:00:57.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57deffd700 1 -- 172.21.15.6:0/2050161662 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d0011990 con 0x7f57e00b1630 2023-11-28T13:00:57.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.867+0000 7f57deffd700 1 -- 172.21.15.6:0/2050161662 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d0011d20 con 0x7f57e00b1630 2023-11-28T13:00:57.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.868+0000 7f57e640e700 1 -- 172.21.15.6:0/2050161662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00b1630 msgr2=0x7f57e00cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.868+0000 7f57e640e700 1 --2- 172.21.15.6:0/2050161662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00b1630 0x7f57e00cf5f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009840 tx=0x7f57d0009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:57.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.868+0000 7f57e640e700 1 -- 172.21.15.6:0/2050161662 shutdown_connections 2023-11-28T13:00:57.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.868+0000 7f57e640e700 1 --2- 172.21.15.6:0/2050161662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00b1630 0x7f57e00cf5f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.868+0000 7f57e640e700 1 --2- 172.21.15.6:0/2050161662 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 0x7f57e00caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.868+0000 7f57e640e700 1 -- 172.21.15.6:0/2050161662 >> 172.21.15.6:0/2050161662 conn(0x7f57e00cc630 msgr2=0x7f57e00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:57.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.868+0000 7f57e640e700 1 -- 172.21.15.6:0/2050161662 shutdown_connections 2023-11-28T13:00:57.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.868+0000 7f57e640e700 1 -- 172.21.15.6:0/2050161662 wait complete. 2023-11-28T13:00:57.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.870+0000 7f57e640e700 1 Processor -- start 2023-11-28T13:00:57.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.870+0000 7f57e640e700 1 -- start start 2023-11-28T13:00:57.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e640e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57e00b1630 0x7f57e00b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e640e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 0x7f57e01584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e640e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00cb3e0 0x7f57e0158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e640e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0158a30 con 0x7f57e00cb3e0 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e640e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00b5740 con 0x7f57e00b1630 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e640e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00b3490 con 0x7f57e00b1a00 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e49ab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00cb3e0 0x7f57e0158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e49ab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00cb3e0 0x7f57e0158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58718/0 (socket says 172.21.15.6:58718) 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e49ab700 1 -- 172.21.15.6:0/3460372665 learned_addr learned my addr 172.21.15.6:0/3460372665 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57dffff700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57e00b1630 0x7f57e00b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57df7fe700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 0x7f57e01584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e49ab700 1 -- 172.21.15.6:0/3460372665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 msgr2=0x7f57e01584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e49ab700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 0x7f57e01584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e49ab700 1 -- 172.21.15.6:0/3460372665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57e00b1630 msgr2=0x7f57e00b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:57.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e49ab700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57e00b1630 0x7f57e00b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e49ab700 1 -- 172.21.15.6:0/3460372665 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d00094a0 con 0x7f57e00cb3e0 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57df7fe700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 0x7f57e01584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e49ab700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00cb3e0 0x7f57e0158bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f57d800ca80 tx=0x7f57d800cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57dcff9700 1 -- 172.21.15.6:0/3460372665 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d800a9e0 con 0x7f57e00cb3e0 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57dcff9700 1 -- 172.21.15.6:0/3460372665 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d800ab40 con 0x7f57e00cb3e0 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57e640e700 1 -- 172.21.15.6:0/3460372665 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e00b1e70 con 0x7f57e00cb3e0 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.871+0000 7f57dcff9700 1 -- 172.21.15.6:0/3460372665 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8012400 con 0x7f57e00cb3e0 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.872+0000 7f57e640e700 1 -- 172.21.15.6:0/3460372665 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e00cf0d0 con 0x7f57e00cb3e0 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.873+0000 7f57e640e700 1 -- 172.21.15.6:0/3460372665 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e00ce140 con 0x7f57e00cb3e0 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.873+0000 7f57dcff9700 1 -- 172.21.15.6:0/3460372665 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f57d8019030 con 0x7f57e00cb3e0 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.873+0000 7f57dcff9700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57cc068320 0x7f57cc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:57.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.873+0000 7f57dcff9700 1 -- 172.21.15.6:0/3460372665 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f57d80858f0 con 0x7f57e00cb3e0 2023-11-28T13:00:57.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.873+0000 7f57dffff700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57cc068320 0x7f57cc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:57.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.874+0000 7f57dffff700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57cc068320 0x7f57cc06a7d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f57d00090b0 tx=0x7f57d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:57.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:57.878+0000 7f57dcff9700 1 -- 172.21.15.6:0/3460372665 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f57d804b5e0 con 0x7f57e00cb3e0 2023-11-28T13:00:58.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.070+0000 7f57e640e700 1 -- 172.21.15.6:0/3460372665 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f57e00739f0 con 0x7f57e00cb3e0 2023-11-28T13:00:58.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.070+0000 7f57dcff9700 1 -- 172.21.15.6:0/3460372665 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f57d8051f10 con 0x7f57e00cb3e0 2023-11-28T13:00:58.071 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:00:58.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.072+0000 7f57e640e700 1 -- 172.21.15.6:0/3460372665 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57cc068320 msgr2=0x7f57cc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.072+0000 7f57e640e700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57cc068320 0x7f57cc06a7d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f57d00090b0 tx=0x7f57d0008030 comp rx=0 tx=0).stop 2023-11-28T13:00:58.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.072+0000 7f57e640e700 1 -- 172.21.15.6:0/3460372665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00cb3e0 msgr2=0x7f57e0158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.072+0000 7f57e640e700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00cb3e0 0x7f57e0158bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f57d800ca80 tx=0x7f57d800cd90 comp rx=0 tx=0).stop 2023-11-28T13:00:58.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.072+0000 7f57e640e700 1 -- 172.21.15.6:0/3460372665 shutdown_connections 2023-11-28T13:00:58.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.072+0000 7f57e640e700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f57e00cb3e0 0x7f57e0158bf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.072+0000 7f57e640e700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57e00b1630 0x7f57e00b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.072+0000 7f57e640e700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57cc068320 0x7f57cc06a7d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.072+0000 7f57e640e700 1 --2- 172.21.15.6:0/3460372665 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f57e00b1a00 0x7f57e01584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.072+0000 7f57e640e700 1 -- 172.21.15.6:0/3460372665 >> 172.21.15.6:0/3460372665 conn(0x7f57e00cc630 msgr2=0x7f57e006c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.073+0000 7f57e640e700 1 -- 172.21.15.6:0/3460372665 shutdown_connections 2023-11-28T13:00:58.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.073+0000 7f57e640e700 1 -- 172.21.15.6:0/3460372665 wait complete. 2023-11-28T13:00:58.085 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:00:58.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.208+0000 7fe04e7d3700 1 Processor -- start 2023-11-28T13:00:58.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.208+0000 7fe04e7d3700 1 -- start start 2023-11-28T13:00:58.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.209+0000 7fe04e7d3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0480b4fe0 0x7fe04815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.209+0000 7fe04e7d3700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04806abb0 con 0x7fe04815a230 2023-11-28T13:00:58.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.209+0000 7fe04e7d3700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04806a280 con 0x7fe0480cd4a0 2023-11-28T13:00:58.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.209+0000 7fe04e7d3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04815a600 con 0x7fe0480b4fe0 2023-11-28T13:00:58.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.209+0000 7fe047fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe04815a230 0x7fe0480b48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46042/0 (socket says 172.21.15.6:46042) 2023-11-28T13:00:58.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.209+0000 7fe047fff700 1 -- 172.21.15.6:0/2058732472 learned_addr learned my addr 172.21.15.6:0/2058732472 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:58.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.209+0000 7fe0477fe700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0480b4fe0 0x7fe04815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584179292 0 0) 0x7fe04806abb0 con 0x7fe04815a230 2023-11-28T13:00:58.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0300035b0 con 0x7fe04815a230 2023-11-28T13:00:58.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4196264953 0 0) 0x7fe0300035b0 con 0x7fe04815a230 2023-11-28T13:00:58.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0480b4fe0 msgr2=0x7fe04815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:58.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0480b4fe0 0x7fe04815d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 >> v1:172.21.15.61:6789/0 conn(0x7fe0480cd4a0 legacy=0x7fe0480d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04806a560 con 0x7fe04815a230 2023-11-28T13:00:58.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fe038001670 con 0x7fe04815a230 2023-11-28T13:00:58.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 >> v1:172.21.15.6:6789/0 conn(0x7fe04815a230 legacy=0x7fe0480b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe030003a30 0x7fe030003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0300043d0 0x7fe030004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0480b4fe0 0x7fe0300050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04815a600 con 0x7fe030003a30 2023-11-28T13:00:58.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03000ba10 con 0x7fe0300043d0 2023-11-28T13:00:58.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03000bba0 con 0x7fe0480b4fe0 2023-11-28T13:00:58.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe047fff700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe030003a30 0x7fe030003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe0477fe700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0300043d0 0x7fe030004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe04cd70700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0480b4fe0 0x7fe0300050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe047fff700 1 -- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0480b4fe0 msgr2=0x7fe0300050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:58.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe047fff700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0480b4fe0 0x7fe0300050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe047fff700 1 -- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0300043d0 msgr2=0x7fe030004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe047fff700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0300043d0 0x7fe030004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe047fff700 1 -- 172.21.15.6:0/2058732472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300035b0 con 0x7fe030003a30 2023-11-28T13:00:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.210+0000 7fe04cd70700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0480b4fe0 0x7fe0300050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:58.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe047fff700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe030003a30 0x7fe030003e00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c2f0 tx=0x7fe030005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0cdcb984ae90d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03800ce30 con 0x7fe030003a30 2023-11-28T13:00:58.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe038005960 con 0x7fe030003a30 2023-11-28T13:00:58.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe046ffd700 1 -- 172.21.15.6:0/2058732472 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038005cf0 con 0x7fe030003a30 2023-11-28T13:00:58.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2058732472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe030003a30 msgr2=0x7fe030003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe04e7d3700 1 --2- 172.21.15.6:0/2058732472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe030003a30 0x7fe030003e00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c2f0 tx=0x7fe030005880 comp rx=0 tx=0).stop 2023-11-28T13:00:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe04cd70700 1 -- 172.21.15.6:0/2058732472 reap_dead start 2023-11-28T13:00:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2058732472 shutdown_connections 2023-11-28T13:00:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2058732472 >> 172.21.15.6:0/2058732472 conn(0x7fe0480cf8d0 msgr2=0x7fe0480cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2058732472 shutdown_connections 2023-11-28T13:00:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.211+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2058732472 wait complete. 2023-11-28T13:00:58.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.212+0000 7fe04e7d3700 1 Processor -- start 2023-11-28T13:00:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.213+0000 7fe04e7d3700 1 -- start start 2023-11-28T13:00:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.213+0000 7fe04e7d3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe030003a30 0x7fe0480cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.213+0000 7fe04e7d3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0481f64b0 0x7fe0481560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.213+0000 7fe04e7d3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0480c8400 0x7fe04814f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.213+0000 7fe04e7d3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480cbe10 con 0x7fe0480c8400 2023-11-28T13:00:58.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.213+0000 7fe04e7d3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0481f6a00 con 0x7fe0481f64b0 2023-11-28T13:00:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.213+0000 7fe04e7d3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480cb4e0 con 0x7fe030003a30 2023-11-28T13:00:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.213+0000 7fe04cd70700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0480c8400 0x7fe04814f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.213+0000 7fe04cd70700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0480c8400 0x7fe04814f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58738/0 (socket says 172.21.15.6:58738) 2023-11-28T13:00:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.213+0000 7fe04cd70700 1 -- 172.21.15.6:0/2663873402 learned_addr learned my addr 172.21.15.6:0/2663873402 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:58.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe0477fe700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0481f64b0 0x7fe0481560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe047fff700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe030003a30 0x7fe0480cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe04cd70700 1 -- 172.21.15.6:0/2663873402 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe030003a30 msgr2=0x7fe0480cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe04cd70700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe030003a30 0x7fe0480cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe04cd70700 1 -- 172.21.15.6:0/2663873402 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0481f64b0 msgr2=0x7fe0481560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe04cd70700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0481f64b0 0x7fe0481560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe04cd70700 1 -- 172.21.15.6:0/2663873402 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0480cd920 con 0x7fe0480c8400 2023-11-28T13:00:58.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe04cd70700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0480c8400 0x7fe04814f140 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe04808b050 tx=0x7fe04000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe044ff9700 1 -- 172.21.15.6:0/2663873402 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040003c10 con 0x7fe0480c8400 2023-11-28T13:00:58.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe044ff9700 1 -- 172.21.15.6:0/2663873402 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe040003d70 con 0x7fe0480c8400 2023-11-28T13:00:58.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe044ff9700 1 -- 172.21.15.6:0/2663873402 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0400052f0 con 0x7fe0480c8400 2023-11-28T13:00:58.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2663873402 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04806a840 con 0x7fe0480c8400 2023-11-28T13:00:58.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.214+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2663873402 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04806b040 con 0x7fe0480c8400 2023-11-28T13:00:58.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.215+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2663873402 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0480cbfc0 con 0x7fe0480c8400 2023-11-28T13:00:58.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.216+0000 7fe044ff9700 1 -- 172.21.15.6:0/2663873402 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe040005450 con 0x7fe0480c8400 2023-11-28T13:00:58.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.216+0000 7fe044ff9700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0340682f0 0x7fe03406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.216+0000 7fe044ff9700 1 -- 172.21.15.6:0/2663873402 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe040086100 con 0x7fe0480c8400 2023-11-28T13:00:58.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.216+0000 7fe047fff700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0340682f0 0x7fe03406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.216+0000 7fe047fff700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0340682f0 0x7fe03406a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe038001930 tx=0x7fe038007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.220+0000 7fe044ff9700 1 -- 172.21.15.6:0/2663873402 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe0480cbfc0 con 0x7fe0480c8400 2023-11-28T13:00:58.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.416+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2663873402 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0480675a0 con 0x7fe0480c8400 2023-11-28T13:00:58.417 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.417+0000 7fe044ff9700 1 -- 172.21.15.6:0/2663873402 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7fe040052670 con 0x7fe0480c8400 2023-11-28T13:00:58.417 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:00:58.418 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2663873402 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0340682f0 msgr2=0x7fe03406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0340682f0 0x7fe03406a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe038001930 tx=0x7fe038007000 comp rx=0 tx=0).stop 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2663873402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0480c8400 msgr2=0x7fe04814f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0480c8400 0x7fe04814f140 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe04808b050 tx=0x7fe04000ef20 comp rx=0 tx=0).stop 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2663873402 shutdown_connections 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe0480c8400 0x7fe04814f140 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0340682f0 0x7fe03406a7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe030003a30 0x7fe0480cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 --2- 172.21.15.6:0/2663873402 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0481f64b0 0x7fe0481560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2663873402 >> 172.21.15.6:0/2663873402 conn(0x7fe0480cf8d0 msgr2=0x7fe0480b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2663873402 shutdown_connections 2023-11-28T13:00:58.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.419+0000 7fe04e7d3700 1 -- 172.21.15.6:0/2663873402 wait complete. 2023-11-28T13:00:58.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:00:58.432 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.559+0000 7f1c09f32700 1 Processor -- start 2023-11-28T13:00:58.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.559+0000 7f1c09f32700 1 -- start start 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c09f32700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1c040b1630 0x7f1c040cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c09f32700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c040ce550 con 0x7f1c0406afc0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c09f32700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c040ce830 con 0x7f1c040b1a00 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c09f32700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c040ceb10 con 0x7f1c040b1630 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c037fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1c040b1630 0x7f1c040cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c03fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1c0406afc0 0x7f1c0406b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46066/0 (socket says 172.21.15.6:46066) 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c03fff700 1 -- 172.21.15.6:0/3585134347 learned_addr learned my addr 172.21.15.6:0/3585134347 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c027fc700 1 -- 172.21.15.6:0/3585134347 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159650262 0 0) 0x7f1c040ce550 con 0x7f1c0406afc0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c027fc700 1 -- 172.21.15.6:0/3585134347 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bec0035b0 con 0x7f1c0406afc0 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c037fe700 1 -- 172.21.15.6:0/3585134347 >> v1:172.21.15.61:6789/0 conn(0x7f1c040b1a00 legacy=0x7f1c040caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c037fe700 1 -- 172.21.15.6:0/3585134347 >> v1:172.21.15.6:6789/0 conn(0x7f1c0406afc0 legacy=0x7f1c0406b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c037fe700 1 -- 172.21.15.6:0/3585134347 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040cedf0 con 0x7f1c040b1630 2023-11-28T13:00:58.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c037fe700 1 --2- 172.21.15.6:0/3585134347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 0x7f1c040cf5f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009870 tx=0x7f1bf4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67c08f238727c91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c027fc700 1 -- 172.21.15.6:0/3585134347 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4011830 con 0x7f1c040b1630 2023-11-28T13:00:58.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.560+0000 7f1c027fc700 1 -- 172.21.15.6:0/3585134347 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf4011990 con 0x7f1c040b1630 2023-11-28T13:00:58.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.561+0000 7f1c027fc700 1 -- 172.21.15.6:0/3585134347 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4011d20 con 0x7f1c040b1630 2023-11-28T13:00:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.561+0000 7f1c09f32700 1 -- 172.21.15.6:0/3585134347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 msgr2=0x7f1c040cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.561+0000 7f1c09f32700 1 --2- 172.21.15.6:0/3585134347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 0x7f1c040cf5f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009870 tx=0x7f1bf4009b80 comp rx=0 tx=0).stop 2023-11-28T13:00:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.561+0000 7f1c09f32700 1 -- 172.21.15.6:0/3585134347 shutdown_connections 2023-11-28T13:00:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.561+0000 7f1c09f32700 1 --2- 172.21.15.6:0/3585134347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 0x7f1c040cf5f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.561+0000 7f1c09f32700 1 -- 172.21.15.6:0/3585134347 >> 172.21.15.6:0/3585134347 conn(0x7f1c040cc630 msgr2=0x7f1c040cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.561+0000 7f1c09f32700 1 -- 172.21.15.6:0/3585134347 shutdown_connections 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.561+0000 7f1c09f32700 1 -- 172.21.15.6:0/3585134347 wait complete. 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.562+0000 7f1c09f32700 1 Processor -- start 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.562+0000 7f1c09f32700 1 -- start start 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.563+0000 7f1c09f32700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c0406afc0 0x7f1c040b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.563+0000 7f1c09f32700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 0x7f1c0415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.563+0000 7f1c09f32700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c040b1a00 0x7f1c0415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.563+0000 7f1c09f32700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c041e7080 con 0x7f1c040b1630 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.563+0000 7f1c09f32700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c041e7360 con 0x7f1c0406afc0 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.563+0000 7f1c09f32700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c041e1850 con 0x7f1c040b1a00 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.563+0000 7f1c02ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 0x7f1c0415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.563+0000 7f1c02ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 0x7f1c0415ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58750/0 (socket says 172.21.15.6:58750) 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.563+0000 7f1c02ffd700 1 -- 172.21.15.6:0/2336844041 learned_addr learned my addr 172.21.15.6:0/2336844041 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:58.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1c037fe700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c0406afc0 0x7f1c040b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1c02ffd700 1 -- 172.21.15.6:0/2336844041 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c040b1a00 msgr2=0x7f1c0415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1c02ffd700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c040b1a00 0x7f1c0415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1c02ffd700 1 -- 172.21.15.6:0/2336844041 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c0406afc0 msgr2=0x7f1c040b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1c02ffd700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c0406afc0 0x7f1c040b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1c02ffd700 1 -- 172.21.15.6:0/2336844041 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf40094a0 con 0x7f1c040b1630 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1c02ffd700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 0x7f1c0415ad20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800c450 tx=0x7f1bf800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1bebfff700 1 -- 172.21.15.6:0/2336844041 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80052d0 con 0x7f1c040b1630 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1bebfff700 1 -- 172.21.15.6:0/2336844041 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf8005430 con 0x7f1c040b1630 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1c09f32700 1 -- 172.21.15.6:0/2336844041 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c040b1ea0 con 0x7f1c040b1630 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1bebfff700 1 -- 172.21.15.6:0/2336844041 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80038c0 con 0x7f1c040b1630 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.564+0000 7f1c09f32700 1 -- 172.21.15.6:0/2336844041 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c040cf0d0 con 0x7f1c040b1630 2023-11-28T13:00:58.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.565+0000 7f1c09f32700 1 -- 172.21.15.6:0/2336844041 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c040cb5b0 con 0x7f1c040b1630 2023-11-28T13:00:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.565+0000 7f1bebfff700 1 -- 172.21.15.6:0/2336844041 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1bf8019030 con 0x7f1c040b1630 2023-11-28T13:00:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.566+0000 7f1bebfff700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1bf00682f0 0x7f1bf006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.566+0000 7f1bebfff700 1 -- 172.21.15.6:0/2336844041 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1bf8085170 con 0x7f1c040b1630 2023-11-28T13:00:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.566+0000 7f1c037fe700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1bf00682f0 0x7f1bf006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.566+0000 7f1c037fe700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1bf00682f0 0x7f1bf006a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009840 tx=0x7f1bf4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.570+0000 7f1bebfff700 1 -- 172.21.15.6:0/2336844041 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1bf80516e0 con 0x7f1c040b1630 2023-11-28T13:00:58.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.759+0000 7f1c09f32700 1 -- 172.21.15.6:0/2336844041 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1c041e1d10 con 0x7f1c040b1630 2023-11-28T13:00:58.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.760+0000 7f1bebfff700 1 -- 172.21.15.6:0/2336844041 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1bf8054d00 con 0x7f1c040b1630 2023-11-28T13:00:58.761 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:00:58.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 -- 172.21.15.6:0/2336844041 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1bf00682f0 msgr2=0x7f1bf006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1bf00682f0 0x7f1bf006a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009840 tx=0x7f1bf4008030 comp rx=0 tx=0).stop 2023-11-28T13:00:58.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 -- 172.21.15.6:0/2336844041 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 msgr2=0x7f1c0415ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 0x7f1c0415ad20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800c450 tx=0x7f1bf800c810 comp rx=0 tx=0).stop 2023-11-28T13:00:58.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 -- 172.21.15.6:0/2336844041 shutdown_connections 2023-11-28T13:00:58.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1bf00682f0 0x7f1bf006a7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1c040b1a00 0x7f1c0415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c0406afc0 0x7f1c040b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 --2- 172.21.15.6:0/2336844041 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c040b1630 0x7f1c0415ad20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 -- 172.21.15.6:0/2336844041 >> 172.21.15.6:0/2336844041 conn(0x7f1c040cc630 msgr2=0x7f1c0406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 -- 172.21.15.6:0/2336844041 shutdown_connections 2023-11-28T13:00:58.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.762+0000 7f1c09f32700 1 -- 172.21.15.6:0/2336844041 wait complete. 2023-11-28T13:00:58.775 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:00:58.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.902+0000 7fd243498700 1 Processor -- start 2023-11-28T13:00:58.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.902+0000 7fd243498700 1 -- start start 2023-11-28T13:00:58.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd243498700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd23c0cf5f0 0x7fd23c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd243498700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd23c0cb3e0 0x7fd23c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd243498700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23c0ce550 con 0x7fd23c0b1630 2023-11-28T13:00:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd243498700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23c0ce830 con 0x7fd23c0cf5f0 2023-11-28T13:00:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd243498700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23c0ceb10 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd240a33700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd23c0cf5f0 0x7fd23c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd241234700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd23c0b1630 0x7fd23c0b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46076/0 (socket says 172.21.15.6:46076) 2023-11-28T13:00:58.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd241234700 1 -- 172.21.15.6:0/2147120380 learned_addr learned my addr 172.21.15.6:0/2147120380 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:58.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd241a35700 1 --2- 172.21.15.6:0/2147120380 >> v2:172.21.15.61:3300/0 conn(0x7fd23c0cb3e0 0x7fd23c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd240a33700 1 -- 172.21.15.6:0/2147120380 >> v2:172.21.15.61:3300/0 conn(0x7fd23c0cb3e0 msgr2=0x7fd23c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:58.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd240a33700 1 --2- 172.21.15.6:0/2147120380 >> v2:172.21.15.61:3300/0 conn(0x7fd23c0cb3e0 0x7fd23c06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd240a33700 1 -- 172.21.15.6:0/2147120380 >> v1:172.21.15.6:6789/0 conn(0x7fd23c0b1630 legacy=0x7fd23c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd233fff700 1 -- 172.21.15.6:0/2147120380 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903219997 0 0) 0x7fd23c0ce550 con 0x7fd23c0b1630 2023-11-28T13:00:58.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd240a33700 1 -- 172.21.15.6:0/2147120380 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c0cedf0 con 0x7fd23c0cf5f0 2023-11-28T13:00:58.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd241a35700 1 --2- 172.21.15.6:0/2147120380 >> v2:172.21.15.61:3300/0 conn(0x7fd23c0cb3e0 0x7fd23c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:58.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd240a33700 1 --2- 172.21.15.6:0/2147120380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cf5f0 0x7fd23c0caea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd238009840 tx=0x7fd238009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f051ce3ad43196b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd233fff700 1 -- 172.21.15.6:0/2147120380 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238011830 con 0x7fd23c0cf5f0 2023-11-28T13:00:58.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd233fff700 1 -- 172.21.15.6:0/2147120380 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd238011990 con 0x7fd23c0cf5f0 2023-11-28T13:00:58.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.903+0000 7fd233fff700 1 -- 172.21.15.6:0/2147120380 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238011d20 con 0x7fd23c0cf5f0 2023-11-28T13:00:58.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.904+0000 7fd243498700 1 -- 172.21.15.6:0/2147120380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cf5f0 msgr2=0x7fd23c0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.904+0000 7fd243498700 1 --2- 172.21.15.6:0/2147120380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cf5f0 0x7fd23c0caea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd238009840 tx=0x7fd238009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:58.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.904+0000 7fd243498700 1 -- 172.21.15.6:0/2147120380 shutdown_connections 2023-11-28T13:00:58.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.904+0000 7fd243498700 1 --2- 172.21.15.6:0/2147120380 >> v2:172.21.15.61:3300/0 conn(0x7fd23c0cb3e0 0x7fd23c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.904+0000 7fd243498700 1 --2- 172.21.15.6:0/2147120380 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cf5f0 0x7fd23c0caea0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.904+0000 7fd243498700 1 -- 172.21.15.6:0/2147120380 >> 172.21.15.6:0/2147120380 conn(0x7fd23c0cc630 msgr2=0x7fd23c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:58.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.904+0000 7fd243498700 1 -- 172.21.15.6:0/2147120380 shutdown_connections 2023-11-28T13:00:58.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.904+0000 7fd243498700 1 -- 172.21.15.6:0/2147120380 wait complete. 2023-11-28T13:00:58.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.905+0000 7fd243498700 1 Processor -- start 2023-11-28T13:00:58.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.905+0000 7fd243498700 1 -- start start 2023-11-28T13:00:58.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd243498700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd23c0b1630 0x7fd23c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd243498700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cb3e0 0x7fd23c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd243498700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd23c0cf5f0 0x7fd23c0c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd243498700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0c8110 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd243498700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0c71c0 con 0x7fd23c0b1630 2023-11-28T13:00:58.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd243498700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0c7490 con 0x7fd23c0cf5f0 2023-11-28T13:00:58.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd240a33700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cb3e0 0x7fd23c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd240a33700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cb3e0 0x7fd23c0c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58764/0 (socket says 172.21.15.6:58764) 2023-11-28T13:00:58.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd240a33700 1 -- 172.21.15.6:0/2549798281 learned_addr learned my addr 172.21.15.6:0/2549798281 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:58.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd241234700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd23c0b1630 0x7fd23c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.906+0000 7fd241a35700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd23c0cf5f0 0x7fd23c0c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd240a33700 1 -- 172.21.15.6:0/2549798281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd23c0cf5f0 msgr2=0x7fd23c0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd240a33700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd23c0cf5f0 0x7fd23c0c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd240a33700 1 -- 172.21.15.6:0/2549798281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd23c0b1630 msgr2=0x7fd23c159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:58.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd240a33700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd23c0b1630 0x7fd23c159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:58.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd240a33700 1 -- 172.21.15.6:0/2549798281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380094a0 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd241234700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd23c0b1630 0x7fd23c159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:58.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd240a33700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cb3e0 0x7fd23c0c6c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd238014040 tx=0x7fd238004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd231ffb700 1 -- 172.21.15.6:0/2549798281 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2380045f0 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd231ffb700 1 -- 172.21.15.6:0/2549798281 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23801f070 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd243498700 1 -- 172.21.15.6:0/2549798281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c0ce140 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd231ffb700 1 -- 172.21.15.6:0/2549798281 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238003ae0 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.907+0000 7fd243498700 1 -- 172.21.15.6:0/2549798281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23c0cf0d0 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.908+0000 7fd243498700 1 -- 172.21.15.6:0/2549798281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23c0ce2d0 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.908+0000 7fd231ffb700 1 -- 172.21.15.6:0/2549798281 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd23800f050 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.909+0000 7fd231ffb700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd2200682f0 0x7fd22006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:58.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.909+0000 7fd231ffb700 1 -- 172.21.15.6:0/2549798281 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd238021070 con 0x7fd23c0cb3e0 2023-11-28T13:00:58.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.909+0000 7fd241234700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd2200682f0 0x7fd22006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:58.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.909+0000 7fd241234700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd2200682f0 0x7fd22006a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd22c000f60 tx=0x7fd22c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:58.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:58.913+0000 7fd231ffb700 1 -- 172.21.15.6:0/2549798281 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd238053fc0 con 0x7fd23c0cb3e0 2023-11-28T13:00:59.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.110+0000 7fd243498700 1 -- 172.21.15.6:0/2549798281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd23c0c8250 con 0x7fd23c0cb3e0 2023-11-28T13:00:59.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.110+0000 7fd231ffb700 1 -- 172.21.15.6:0/2549798281 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd23c0c8250 con 0x7fd23c0cb3e0 2023-11-28T13:00:59.112 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:00:59.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.112+0000 7fd243498700 1 -- 172.21.15.6:0/2549798281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd2200682f0 msgr2=0x7fd22006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.112+0000 7fd243498700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd2200682f0 0x7fd22006a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd22c000f60 tx=0x7fd22c009040 comp rx=0 tx=0).stop 2023-11-28T13:00:59.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.113+0000 7fd243498700 1 -- 172.21.15.6:0/2549798281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cb3e0 msgr2=0x7fd23c0c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.113+0000 7fd243498700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cb3e0 0x7fd23c0c6c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd238014040 tx=0x7fd238004750 comp rx=0 tx=0).stop 2023-11-28T13:00:59.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.113+0000 7fd243498700 1 -- 172.21.15.6:0/2549798281 shutdown_connections 2023-11-28T13:00:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.113+0000 7fd243498700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd2200682f0 0x7fd22006a7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.113+0000 7fd243498700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd23c0cf5f0 0x7fd23c0c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.113+0000 7fd243498700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd23c0b1630 0x7fd23c159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.113+0000 7fd243498700 1 --2- 172.21.15.6:0/2549798281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd23c0cb3e0 0x7fd23c0c6c80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.113+0000 7fd243498700 1 -- 172.21.15.6:0/2549798281 >> 172.21.15.6:0/2549798281 conn(0x7fd23c0cc630 msgr2=0x7fd23c0b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.113+0000 7fd243498700 1 -- 172.21.15.6:0/2549798281 shutdown_connections 2023-11-28T13:00:59.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.113+0000 7fd243498700 1 -- 172.21.15.6:0/2549798281 wait complete. 2023-11-28T13:00:59.126 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:00:59.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.252+0000 7f87b8f1c700 1 Processor -- start 2023-11-28T13:00:59.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.252+0000 7f87b8f1c700 1 -- start start 2023-11-28T13:00:59.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.253+0000 7f87b8f1c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b40cf5f0 0x7f87b40caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.253+0000 7f87b8f1c700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40ce550 con 0x7f87b40b1630 2023-11-28T13:00:59.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.253+0000 7f87b8f1c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40ce830 con 0x7f87b40cb3e0 2023-11-28T13:00:59.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.253+0000 7f87b8f1c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40cfa60 con 0x7f87b40cf5f0 2023-11-28T13:00:59.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b259c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f87b40b1630 0x7f87b40b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46088/0 (socket says 172.21.15.6:46088) 2023-11-28T13:00:59.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b259c700 1 -- 172.21.15.6:0/3917933786 learned_addr learned my addr 172.21.15.6:0/3917933786 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:59.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b1d9b700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b40cf5f0 0x7f87b40caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259707122 0 0) 0x7f87b40ce550 con 0x7f87b40b1630 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a00035b0 con 0x7f87b40b1630 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283418347 0 0) 0x7f87b40ce830 con 0x7f87b40cb3e0 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87b40ce550 con 0x7f87b40cb3e0 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1283117298 0 0) 0x7f87a00035b0 con 0x7f87b40b1630 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b40cf5f0 msgr2=0x7f87b40caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b40cf5f0 0x7f87b40caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 >> v1:172.21.15.61:6789/0 conn(0x7f87b40cb3e0 legacy=0x7f87b406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40ceb10 con 0x7f87b40b1630 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f87a4001670 con 0x7f87b40b1630 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 >> v1:172.21.15.6:6789/0 conn(0x7f87b40b1630 legacy=0x7f87b40b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b40cf5f0 0x7f87a0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 0x7f87a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a0004d90 0x7f87a0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a000baf0 con 0x7f87a0004170 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a000bc60 con 0x7f87a0004d90 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a000be20 con 0x7f87b40cf5f0 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b1d9b700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 0x7f87a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b259c700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b40cf5f0 0x7f87a0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.254+0000 7f87b2d9d700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a0004d90 0x7f87a0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b1d9b700 1 -- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b40cf5f0 msgr2=0x7f87a0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b1d9b700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b40cf5f0 0x7f87a0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b1d9b700 1 -- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a0004d90 msgr2=0x7f87a0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:59.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b1d9b700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87a0004d90 0x7f87a0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b1d9b700 1 -- 172.21.15.6:0/3917933786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a00035b0 con 0x7f87a0004170 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b1d9b700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 0x7f87a0004850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f879c009b60 tx=0x7f87a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687043b4a31aa30c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c00e6c0 con 0x7f87a0004170 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c00e820 con 0x7f87a0004170 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b159a700 1 -- 172.21.15.6:0/3917933786 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c00eb90 con 0x7f87a0004170 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b8f1c700 1 -- 172.21.15.6:0/3917933786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 msgr2=0x7f87a0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b8f1c700 1 --2- 172.21.15.6:0/3917933786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 0x7f87a0004850 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f879c009b60 tx=0x7f87a0005880 comp rx=0 tx=0).stop 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.255+0000 7f87b2d9d700 1 -- 172.21.15.6:0/3917933786 reap_dead start 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.256+0000 7f87b8f1c700 1 -- 172.21.15.6:0/3917933786 shutdown_connections 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.256+0000 7f87b8f1c700 1 -- 172.21.15.6:0/3917933786 >> 172.21.15.6:0/3917933786 conn(0x7f87b40cc630 msgr2=0x7f87b40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.256+0000 7f87b8f1c700 1 -- 172.21.15.6:0/3917933786 shutdown_connections 2023-11-28T13:00:59.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.256+0000 7f87b8f1c700 1 -- 172.21.15.6:0/3917933786 wait complete. 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.257+0000 7f87b8f1c700 1 Processor -- start 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.257+0000 7f87b8f1c700 1 -- start start 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b8f1c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 0x7f87b4157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b8f1c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b4156a40 0x7f87b41f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b8f1c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87b415f240 0x7f87b415d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b8f1c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b415fe00 con 0x7f87a0004170 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b8f1c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40ac900 con 0x7f87b415f240 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b8f1c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b41f2c50 con 0x7f87b4156a40 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 0x7f87b4157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 0x7f87b4157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58780/0 (socket says 172.21.15.6:58780) 2023-11-28T13:00:59.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b259c700 1 -- 172.21.15.6:0/329173038 learned_addr learned my addr 172.21.15.6:0/329173038 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b1d9b700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b4156a40 0x7f87b41f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b2d9d700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87b415f240 0x7f87b415d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b259c700 1 -- 172.21.15.6:0/329173038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b4156a40 msgr2=0x7f87b41f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b259c700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b4156a40 0x7f87b41f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b259c700 1 -- 172.21.15.6:0/329173038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87b415f240 msgr2=0x7f87b415d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b259c700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87b415f240 0x7f87b415d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b259c700 1 -- 172.21.15.6:0/329173038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40cf0d0 con 0x7f87a0004170 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b2d9d700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87b415f240 0x7f87b415d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87b259c700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 0x7f87b4157170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f87a40011e0 tx=0x7f87a4007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87aaffd700 1 -- 172.21.15.6:0/329173038 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4011580 con 0x7f87a0004170 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.258+0000 7f87aaffd700 1 -- 172.21.15.6:0/329173038 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a4011720 con 0x7f87a0004170 2023-11-28T13:00:59.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.259+0000 7f87aaffd700 1 -- 172.21.15.6:0/329173038 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4011ad0 con 0x7f87a0004170 2023-11-28T13:00:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.259+0000 7f87b8f1c700 1 -- 172.21.15.6:0/329173038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b40cedf0 con 0x7f87a0004170 2023-11-28T13:00:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.259+0000 7f87b8f1c700 1 -- 172.21.15.6:0/329173038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b40ce140 con 0x7f87a0004170 2023-11-28T13:00:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.260+0000 7f87b8f1c700 1 -- 172.21.15.6:0/329173038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b40aca40 con 0x7f87a0004170 2023-11-28T13:00:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.260+0000 7f87aaffd700 1 -- 172.21.15.6:0/329173038 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f87a4011c30 con 0x7f87a0004170 2023-11-28T13:00:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.261+0000 7f87aaffd700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87940682f0 0x7f879406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.261+0000 7f87b1d9b700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87940682f0 0x7f879406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.261+0000 7f87aaffd700 1 -- 172.21.15.6:0/329173038 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f87a40860a0 con 0x7f87a0004170 2023-11-28T13:00:59.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.261+0000 7f87b1d9b700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87940682f0 0x7f879406a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f879c0090b0 tx=0x7f879c01f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.265+0000 7f87aaffd700 1 -- 172.21.15.6:0/329173038 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f87b40aca40 con 0x7f87a0004170 2023-11-28T13:00:59.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.459+0000 7f87b8f1c700 1 -- 172.21.15.6:0/329173038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87b40aca40 con 0x7f87a0004170 2023-11-28T13:00:59.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.459+0000 7f87aaffd700 1 -- 172.21.15.6:0/329173038 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87b40aca40 con 0x7f87a0004170 2023-11-28T13:00:59.461 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:00:59.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 -- 172.21.15.6:0/329173038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87940682f0 msgr2=0x7f879406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87940682f0 0x7f879406a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f879c0090b0 tx=0x7f879c01f040 comp rx=0 tx=0).stop 2023-11-28T13:00:59.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 -- 172.21.15.6:0/329173038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 msgr2=0x7f87b4157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 0x7f87b4157170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f87a40011e0 tx=0x7f87a4007570 comp rx=0 tx=0).stop 2023-11-28T13:00:59.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 -- 172.21.15.6:0/329173038 shutdown_connections 2023-11-28T13:00:59.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f87b415f240 0x7f87b415d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87a0004170 0x7f87b4157170 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87940682f0 0x7f879406a7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 --2- 172.21.15.6:0/329173038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87b4156a40 0x7f87b41f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 -- 172.21.15.6:0/329173038 >> 172.21.15.6:0/329173038 conn(0x7f87b40cc630 msgr2=0x7f87b40b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 -- 172.21.15.6:0/329173038 shutdown_connections 2023-11-28T13:00:59.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.461+0000 7f87b8f1c700 1 -- 172.21.15.6:0/329173038 wait complete. 2023-11-28T13:00:59.474 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:00:59.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.601+0000 7fd4e1512700 1 Processor -- start 2023-11-28T13:00:59.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.601+0000 7fd4e1512700 1 -- start start 2023-11-28T13:00:59.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.601+0000 7fd4e1512700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 0x7fd4dc0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.601+0000 7fd4e1512700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc0d1470 con 0x7fd4dc06bf30 2023-11-28T13:00:59.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.601+0000 7fd4e1512700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc0cd4e0 con 0x7fd4dc15c640 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.601+0000 7fd4e1512700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc15ca10 con 0x7fd4dc0d2820 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4db7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd4dc06bf30 0x7fd4dc06c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46112/0 (socket says 172.21.15.6:46112) 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4db7fe700 1 -- 172.21.15.6:0/3770260705 learned_addr learned my addr 172.21.15.6:0/3770260705 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4da7fc700 1 --2- 172.21.15.6:0/3770260705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 0x7fd4dc0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4d9ffb700 1 -- 172.21.15.6:0/3770260705 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269723318 0 0) 0x7fd4dc0d1470 con 0x7fd4dc06bf30 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4d9ffb700 1 -- 172.21.15.6:0/3770260705 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4c40035b0 con 0x7fd4dc06bf30 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4d9ffb700 1 -- 172.21.15.6:0/3770260705 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830252529 0 0) 0x7fd4dc0cd4e0 con 0x7fd4dc15c640 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4d9ffb700 1 -- 172.21.15.6:0/3770260705 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4dc0d1470 con 0x7fd4dc15c640 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4da7fc700 1 -- 172.21.15.6:0/3770260705 >> v1:172.21.15.61:6789/0 conn(0x7fd4dc15c640 legacy=0x7fd4dc0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4da7fc700 1 -- 172.21.15.6:0/3770260705 >> v1:172.21.15.6:6789/0 conn(0x7fd4dc06bf30 legacy=0x7fd4dc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4da7fc700 1 -- 172.21.15.6:0/3770260705 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0cd7c0 con 0x7fd4dc0d2820 2023-11-28T13:00:59.605 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4da7fc700 1 --2- 172.21.15.6:0/3770260705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 0x7fd4dc0b5c40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0cee30 tx=0x7fd4d0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=711f0ad3958517ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.602+0000 7fd4d9ffb700 1 -- 172.21.15.6:0/3770260705 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0004020 con 0x7fd4dc0d2820 2023-11-28T13:00:59.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.603+0000 7fd4d9ffb700 1 -- 172.21.15.6:0/3770260705 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d0004180 con 0x7fd4dc0d2820 2023-11-28T13:00:59.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.603+0000 7fd4d9ffb700 1 -- 172.21.15.6:0/3770260705 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d00044f0 con 0x7fd4dc0d2820 2023-11-28T13:00:59.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.603+0000 7fd4e1512700 1 -- 172.21.15.6:0/3770260705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 msgr2=0x7fd4dc0b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.603+0000 7fd4e1512700 1 --2- 172.21.15.6:0/3770260705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 0x7fd4dc0b5c40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0cee30 tx=0x7fd4d0009b50 comp rx=0 tx=0).stop 2023-11-28T13:00:59.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.603+0000 7fd4e1512700 1 -- 172.21.15.6:0/3770260705 shutdown_connections 2023-11-28T13:00:59.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.603+0000 7fd4e1512700 1 --2- 172.21.15.6:0/3770260705 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 0x7fd4dc0b5c40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.603+0000 7fd4e1512700 1 -- 172.21.15.6:0/3770260705 >> 172.21.15.6:0/3770260705 conn(0x7fd4dc0d2df0 msgr2=0x7fd4dc0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.603+0000 7fd4e1512700 1 -- 172.21.15.6:0/3770260705 shutdown_connections 2023-11-28T13:00:59.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.603+0000 7fd4e1512700 1 -- 172.21.15.6:0/3770260705 wait complete. 2023-11-28T13:00:59.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.605+0000 7fd4e1512700 1 Processor -- start 2023-11-28T13:00:59.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.605+0000 7fd4e1512700 1 -- start start 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4e1512700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4dc06bf30 0x7fd4dc144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4e1512700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 0x7fd4dc0c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4e1512700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4dc15c640 0x7fd4dc147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4e1512700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc1597b0 con 0x7fd4dc06bf30 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4e1512700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0cbfd0 con 0x7fd4dc15c640 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4e1512700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0cc2b0 con 0x7fd4dc0d2820 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4daffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4dc06bf30 0x7fd4dc144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4daffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4dc06bf30 0x7fd4dc144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58794/0 (socket says 172.21.15.6:58794) 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4daffd700 1 -- 172.21.15.6:0/1441204134 learned_addr learned my addr 172.21.15.6:0/1441204134 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4db7fe700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4dc15c640 0x7fd4dc147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4da7fc700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 0x7fd4dc0c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4daffd700 1 -- 172.21.15.6:0/1441204134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 msgr2=0x7fd4dc0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4daffd700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 0x7fd4dc0c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4daffd700 1 -- 172.21.15.6:0/1441204134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4dc15c640 msgr2=0x7fd4dc147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4daffd700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4dc15c640 0x7fd4dc147760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4daffd700 1 -- 172.21.15.6:0/1441204134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00094a0 con 0x7fd4dc06bf30 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4db7fe700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4dc15c640 0x7fd4dc147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.606+0000 7fd4daffd700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4dc06bf30 0x7fd4dc144bd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c500 tx=0x7fd4cc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.607+0000 7fd4c37fe700 1 -- 172.21.15.6:0/1441204134 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0052d0 con 0x7fd4dc06bf30 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.607+0000 7fd4c37fe700 1 -- 172.21.15.6:0/1441204134 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc005450 con 0x7fd4dc06bf30 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.607+0000 7fd4e1512700 1 -- 172.21.15.6:0/1441204134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc0cdaa0 con 0x7fd4dc06bf30 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.607+0000 7fd4c37fe700 1 -- 172.21.15.6:0/1441204134 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0038c0 con 0x7fd4dc06bf30 2023-11-28T13:00:59.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.607+0000 7fd4e1512700 1 -- 172.21.15.6:0/1441204134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc0d1710 con 0x7fd4dc06bf30 2023-11-28T13:00:59.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.607+0000 7fd4e1512700 1 -- 172.21.15.6:0/1441204134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc0cdc30 con 0x7fd4dc06bf30 2023-11-28T13:00:59.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.608+0000 7fd4c37fe700 1 -- 172.21.15.6:0/1441204134 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd4cc019030 con 0x7fd4dc06bf30 2023-11-28T13:00:59.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.608+0000 7fd4c37fe700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4c80682f0 0x7fd4c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.608+0000 7fd4c37fe700 1 -- 172.21.15.6:0/1441204134 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd4cc0853b0 con 0x7fd4dc06bf30 2023-11-28T13:00:59.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.608+0000 7fd4da7fc700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4c80682f0 0x7fd4c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.609+0000 7fd4da7fc700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4c80682f0 0x7fd4c806a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0cee30 tx=0x7fd4d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.613+0000 7fd4c37fe700 1 -- 172.21.15.6:0/1441204134 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd4cc0518a0 con 0x7fd4dc06bf30 2023-11-28T13:00:59.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.804+0000 7fd4e1512700 1 -- 172.21.15.6:0/1441204134 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4dc0739f0 con 0x7fd4dc06bf30 2023-11-28T13:00:59.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.805+0000 7fd4c37fe700 1 -- 172.21.15.6:0/1441204134 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4cc054ec0 con 0x7fd4dc06bf30 2023-11-28T13:00:59.806 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:00:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.806+0000 7fd4e1512700 1 -- 172.21.15.6:0/1441204134 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4c80682f0 msgr2=0x7fd4c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4c80682f0 0x7fd4c806a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0cee30 tx=0x7fd4d0008030 comp rx=0 tx=0).stop 2023-11-28T13:00:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 -- 172.21.15.6:0/1441204134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4dc06bf30 msgr2=0x7fd4dc144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4dc06bf30 0x7fd4dc144bd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c500 tx=0x7fd4cc00c8c0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 -- 172.21.15.6:0/1441204134 shutdown_connections 2023-11-28T13:00:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4dc15c640 0x7fd4dc147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4dc06bf30 0x7fd4dc144bd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4c80682f0 0x7fd4c806a7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 --2- 172.21.15.6:0/1441204134 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4dc0d2820 0x7fd4dc0c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 -- 172.21.15.6:0/1441204134 >> 172.21.15.6:0/1441204134 conn(0x7fd4dc0d2df0 msgr2=0x7fd4dc159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 -- 172.21.15.6:0/1441204134 shutdown_connections 2023-11-28T13:00:59.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.807+0000 7fd4e1512700 1 -- 172.21.15.6:0/1441204134 wait complete. 2023-11-28T13:00:59.820 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:00:59.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.946+0000 7ff50dad8700 1 Processor -- start 2023-11-28T13:00:59.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.946+0000 7ff50dad8700 1 -- start start 2023-11-28T13:00:59.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff50dad8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff5080b2680 0x7ff5080b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff50dad8700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50806f940 con 0x7ff50815fc00 2023-11-28T13:00:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff50dad8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50806fc20 con 0x7ff5080cbbf0 2023-11-28T13:00:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff50dad8700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5080b3030 con 0x7ff5080b2680 2023-11-28T13:00:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff506ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff5080b2680 0x7ff5080b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff5077fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff50815fc00 0x7ff50806eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46122/0 (socket says 172.21.15.6:46122) 2023-11-28T13:00:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff5077fe700 1 -- 172.21.15.6:0/1465860215 learned_addr learned my addr 172.21.15.6:0/1465860215 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:00:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff506ffd700 1 -- 172.21.15.6:0/1465860215 >> v1:172.21.15.61:6789/0 conn(0x7ff5080cbbf0 legacy=0x7ff5080cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:00:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff506ffd700 1 -- 172.21.15.6:0/1465860215 >> v1:172.21.15.6:6789/0 conn(0x7ff50815fc00 legacy=0x7ff50806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff5067fc700 1 -- 172.21.15.6:0/1465860215 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1702075036 0 0) 0x7ff50806f940 con 0x7ff50815fc00 2023-11-28T13:00:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.947+0000 7ff506ffd700 1 -- 172.21.15.6:0/1465860215 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080cb3f0 con 0x7ff5080b2680 2023-11-28T13:00:59.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff506ffd700 1 --2- 172.21.15.6:0/1465860215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5080b2680 0x7ff5080b2af0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff5080d03c0 tx=0x7ff4f0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd85816f59d6ec72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff5067fc700 1 -- 172.21.15.6:0/1465860215 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0011830 con 0x7ff5080b2680 2023-11-28T13:00:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff5067fc700 1 -- 172.21.15.6:0/1465860215 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0011990 con 0x7ff5080b2680 2023-11-28T13:00:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff5067fc700 1 -- 172.21.15.6:0/1465860215 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0011d20 con 0x7ff5080b2680 2023-11-28T13:00:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff50dad8700 1 -- 172.21.15.6:0/1465860215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5080b2680 msgr2=0x7ff5080b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff50dad8700 1 --2- 172.21.15.6:0/1465860215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5080b2680 0x7ff5080b2af0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff5080d03c0 tx=0x7ff4f0009b20 comp rx=0 tx=0).stop 2023-11-28T13:00:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff50dad8700 1 -- 172.21.15.6:0/1465860215 shutdown_connections 2023-11-28T13:00:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff50dad8700 1 --2- 172.21.15.6:0/1465860215 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5080b2680 0x7ff5080b2af0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff50dad8700 1 -- 172.21.15.6:0/1465860215 >> 172.21.15.6:0/1465860215 conn(0x7ff5080cd680 msgr2=0x7ff5080cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:00:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff50dad8700 1 -- 172.21.15.6:0/1465860215 shutdown_connections 2023-11-28T13:00:59.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.948+0000 7ff50dad8700 1 -- 172.21.15.6:0/1465860215 wait complete. 2023-11-28T13:00:59.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.949+0000 7ff50dad8700 1 Processor -- start 2023-11-28T13:00:59.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.950+0000 7ff50dad8700 1 -- start start 2023-11-28T13:00:59.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.950+0000 7ff50dad8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff5080b2680 0x7ff5080c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.950+0000 7ff50dad8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5080cbbf0 0x7ff508158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.950+0000 7ff50dad8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff50815fc00 0x7ff508158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.950+0000 7ff50dad8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50814aa60 con 0x7ff50815fc00 2023-11-28T13:00:59.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.950+0000 7ff50dad8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080c5ca0 con 0x7ff5080b2680 2023-11-28T13:00:59.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.950+0000 7ff50dad8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080c5ed0 con 0x7ff5080cbbf0 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff507fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff50815fc00 0x7ff508158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff506ffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5080cbbf0 0x7ff508158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff5077fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff5080b2680 0x7ff5080c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff507fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff50815fc00 0x7ff508158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58812/0 (socket says 172.21.15.6:58812) 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff507fff700 1 -- 172.21.15.6:0/3776819069 learned_addr learned my addr 172.21.15.6:0/3776819069 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff507fff700 1 -- 172.21.15.6:0/3776819069 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5080cbbf0 msgr2=0x7ff508158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff507fff700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5080cbbf0 0x7ff508158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff507fff700 1 -- 172.21.15.6:0/3776819069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff5080b2680 msgr2=0x7ff5080c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff507fff700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff5080b2680 0x7ff5080c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff507fff700 1 -- 172.21.15.6:0/3776819069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00094a0 con 0x7ff50815fc00 2023-11-28T13:00:59.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff507fff700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff50815fc00 0x7ff508158560 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff50000cc10 tx=0x7ff50000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff4fffff700 1 -- 172.21.15.6:0/3776819069 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50000aa60 con 0x7ff50815fc00 2023-11-28T13:00:59.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff4fffff700 1 -- 172.21.15.6:0/3776819069 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50000abc0 con 0x7ff50815fc00 2023-11-28T13:00:59.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff50dad8700 1 -- 172.21.15.6:0/3776819069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50815ffd0 con 0x7ff50815fc00 2023-11-28T13:00:59.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff4fffff700 1 -- 172.21.15.6:0/3776819069 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500012420 con 0x7ff50815fc00 2023-11-28T13:00:59.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.951+0000 7ff50dad8700 1 -- 172.21.15.6:0/3776819069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5080cb6d0 con 0x7ff50815fc00 2023-11-28T13:00:59.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.952+0000 7ff50dad8700 1 -- 172.21.15.6:0/3776819069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50806f5b0 con 0x7ff50815fc00 2023-11-28T13:00:59.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.953+0000 7ff4fffff700 1 -- 172.21.15.6:0/3776819069 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff500010050 con 0x7ff50815fc00 2023-11-28T13:00:59.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.953+0000 7ff4fffff700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4e80682f0 0x7ff4e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:00:59.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.953+0000 7ff5077fe700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4e80682f0 0x7ff4e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:00:59.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.953+0000 7ff4fffff700 1 -- 172.21.15.6:0/3776819069 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff500086180 con 0x7ff50815fc00 2023-11-28T13:00:59.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.954+0000 7ff5077fe700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4e80682f0 0x7ff4e806a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8007fd0 tx=0x7ff4f8007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:00:59.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:00:59.958+0000 7ff4fffff700 1 -- 172.21.15.6:0/3776819069 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff5000525a0 con 0x7ff50815fc00 2023-11-28T13:01:00.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.152+0000 7ff50dad8700 1 -- 172.21.15.6:0/3776819069 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff5080739f0 con 0x7ff50815fc00 2023-11-28T13:01:00.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.152+0000 7ff4fffff700 1 -- 172.21.15.6:0/3776819069 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff500055bc0 con 0x7ff50815fc00 2023-11-28T13:01:00.154 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 -- 172.21.15.6:0/3776819069 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4e80682f0 msgr2=0x7ff4e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4e80682f0 0x7ff4e806a7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8007fd0 tx=0x7ff4f8007de0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 -- 172.21.15.6:0/3776819069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff50815fc00 msgr2=0x7ff508158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff50815fc00 0x7ff508158560 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff50000cc10 tx=0x7ff50000cf20 comp rx=0 tx=0).stop 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 -- 172.21.15.6:0/3776819069 shutdown_connections 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff50815fc00 0x7ff508158560 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff5080b2680 0x7ff5080c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff4e80682f0 0x7ff4e806a7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 --2- 172.21.15.6:0/3776819069 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff5080cbbf0 0x7ff508158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 -- 172.21.15.6:0/3776819069 >> 172.21.15.6:0/3776819069 conn(0x7ff5080cd680 msgr2=0x7ff5080b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 -- 172.21.15.6:0/3776819069 shutdown_connections 2023-11-28T13:01:00.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.154+0000 7ff50dad8700 1 -- 172.21.15.6:0/3776819069 wait complete. 2023-11-28T13:01:00.168 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:00.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.290+0000 7f48d9688700 1 Processor -- start 2023-11-28T13:01:00.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.290+0000 7f48d9688700 1 -- start start 2023-11-28T13:01:00.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.291+0000 7f48d9688700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f48d415c560 0x7f48d406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.291+0000 7f48d9688700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 0x7f48d40ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.291+0000 7f48d9688700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d40b30b0 con 0x7f48d40cecf0 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.291+0000 7f48d9688700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d406c020 con 0x7f48d415c560 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.291+0000 7f48d9688700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40cf110 con 0x7f48d40b2660 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.291+0000 7f48d27fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 0x7f48d40ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.291+0000 7f48d2ffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f48d415c560 0x7f48d406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.291+0000 7f48d37fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f48d40cecf0 0x7f48d4160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:36170/0 (socket says 172.21.15.6:36170) 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.291+0000 7f48d37fe700 1 -- 172.21.15.6:0/3879255725 learned_addr learned my addr 172.21.15.6:0/3879255725 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d1ffb700 1 -- 172.21.15.6:0/3879255725 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344274628 0 0) 0x7f48d40b30b0 con 0x7f48d40cecf0 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d1ffb700 1 -- 172.21.15.6:0/3879255725 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48bc0035b0 con 0x7f48d40cecf0 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d2ffd700 1 -- 172.21.15.6:0/3879255725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 msgr2=0x7f48d40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:00.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d2ffd700 1 --2- 172.21.15.6:0/3879255725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 0x7f48d40ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d2ffd700 1 -- 172.21.15.6:0/3879255725 >> v1:172.21.15.61:6789/0 conn(0x7f48d40cecf0 legacy=0x7f48d4160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d2ffd700 1 -- 172.21.15.6:0/3879255725 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d40d09c0 con 0x7f48d415c560 2023-11-28T13:01:00.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d27fc700 1 --2- 172.21.15.6:0/3879255725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 0x7f48d40ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:00.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d2ffd700 1 --2- 172.21.15.6:0/3879255725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48d415c560 0x7f48d406f8b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009840 tx=0x7f48c8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1503e30f9050e830 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d1ffb700 1 -- 172.21.15.6:0/3879255725 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8011830 con 0x7f48d415c560 2023-11-28T13:01:00.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d1ffb700 1 -- 172.21.15.6:0/3879255725 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c8011990 con 0x7f48d415c560 2023-11-28T13:01:00.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.292+0000 7f48d1ffb700 1 -- 172.21.15.6:0/3879255725 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8011d20 con 0x7f48d415c560 2023-11-28T13:01:00.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.293+0000 7f48d9688700 1 -- 172.21.15.6:0/3879255725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48d415c560 msgr2=0x7f48d406f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.293+0000 7f48d9688700 1 --2- 172.21.15.6:0/3879255725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48d415c560 0x7f48d406f8b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f48c8009840 tx=0x7f48c8009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:00.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.293+0000 7f48d9688700 1 -- 172.21.15.6:0/3879255725 shutdown_connections 2023-11-28T13:01:00.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.293+0000 7f48d9688700 1 --2- 172.21.15.6:0/3879255725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48d415c560 0x7f48d406f8b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.293+0000 7f48d9688700 1 --2- 172.21.15.6:0/3879255725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 0x7f48d40ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.293+0000 7f48d9688700 1 -- 172.21.15.6:0/3879255725 >> 172.21.15.6:0/3879255725 conn(0x7f48d40cd660 msgr2=0x7f48d40cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.293+0000 7f48d9688700 1 -- 172.21.15.6:0/3879255725 shutdown_connections 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.293+0000 7f48d9688700 1 -- 172.21.15.6:0/3879255725 wait complete. 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.294+0000 7f48d9688700 1 Processor -- start 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.294+0000 7f48d9688700 1 -- start start 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d9688700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 0x7f48d41f6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d9688700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48d40cecf0 0x7f48d4152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d9688700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48d415c560 0x7f48d40c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d9688700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d41f6430 con 0x7f48d415c560 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d9688700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40c5560 con 0x7f48d40cecf0 2023-11-28T13:01:00.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d9688700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40c5280 con 0x7f48d40b2660 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48d415c560 0x7f48d40c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48d415c560 0x7f48d40c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58826/0 (socket says 172.21.15.6:58826) 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d37fe700 1 -- 172.21.15.6:0/642775866 learned_addr learned my addr 172.21.15.6:0/642775866 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d2ffd700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 0x7f48d41f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d27fc700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48d40cecf0 0x7f48d4152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d37fe700 1 -- 172.21.15.6:0/642775866 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 msgr2=0x7f48d41f6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d37fe700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 0x7f48d41f6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d37fe700 1 -- 172.21.15.6:0/642775866 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48d40cecf0 msgr2=0x7f48d4152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d37fe700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48d40cecf0 0x7f48d4152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.295+0000 7f48d37fe700 1 -- 172.21.15.6:0/642775866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c40094a0 con 0x7f48d415c560 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.296+0000 7f48d37fe700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48d415c560 0x7f48d40c4710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00cb40 tx=0x7f48cc00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.296+0000 7f48bb7fe700 1 -- 172.21.15.6:0/642775866 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00a9e0 con 0x7f48d415c560 2023-11-28T13:01:00.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.296+0000 7f48bb7fe700 1 -- 172.21.15.6:0/642775866 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc00ab40 con 0x7f48d415c560 2023-11-28T13:01:00.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.296+0000 7f48bb7fe700 1 -- 172.21.15.6:0/642775866 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc012400 con 0x7f48d415c560 2023-11-28T13:01:00.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.296+0000 7f48d9688700 1 -- 172.21.15.6:0/642775866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c80094a0 con 0x7f48d415c560 2023-11-28T13:01:00.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.296+0000 7f48d9688700 1 -- 172.21.15.6:0/642775866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d40b2af0 con 0x7f48d415c560 2023-11-28T13:01:00.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.297+0000 7f48d9688700 1 -- 172.21.15.6:0/642775866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d406fdf0 con 0x7f48d415c560 2023-11-28T13:01:00.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.297+0000 7f48bb7fe700 1 -- 172.21.15.6:0/642775866 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f48cc019030 con 0x7f48d415c560 2023-11-28T13:01:00.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.298+0000 7f48bb7fe700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48c00682f0 0x7f48c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.298+0000 7f48bb7fe700 1 -- 172.21.15.6:0/642775866 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f48cc085a70 con 0x7f48d415c560 2023-11-28T13:01:00.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.298+0000 7f48d2ffd700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48c00682f0 0x7f48c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.298+0000 7f48d2ffd700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48c00682f0 0x7f48c006a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000c00 tx=0x7f48c801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.302+0000 7f48bb7fe700 1 -- 172.21.15.6:0/642775866 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f48cc04b5e0 con 0x7f48d415c560 2023-11-28T13:01:00.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.495+0000 7f48d9688700 1 -- 172.21.15.6:0/642775866 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48d40b2dd0 con 0x7f48d415c560 2023-11-28T13:01:00.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.496+0000 7f48bb7fe700 1 -- 172.21.15.6:0/642775866 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f48cc051fa0 con 0x7f48d415c560 2023-11-28T13:01:00.498 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:00.498 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:00.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 -- 172.21.15.6:0/642775866 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48c00682f0 msgr2=0x7f48c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48c00682f0 0x7f48c006a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000c00 tx=0x7f48c801c040 comp rx=0 tx=0).stop 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 -- 172.21.15.6:0/642775866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48d415c560 msgr2=0x7f48d40c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48d415c560 0x7f48d40c4710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00cb40 tx=0x7f48cc00ce50 comp rx=0 tx=0).stop 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 -- 172.21.15.6:0/642775866 shutdown_connections 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48d415c560 0x7f48d40c4710 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48c00682f0 0x7f48c006a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48d40b2660 0x7f48d41f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 --2- 172.21.15.6:0/642775866 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48d40cecf0 0x7f48d4152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 -- 172.21.15.6:0/642775866 >> 172.21.15.6:0/642775866 conn(0x7f48d40cd660 msgr2=0x7f48d4066010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 -- 172.21.15.6:0/642775866 shutdown_connections 2023-11-28T13:01:00.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.498+0000 7f48d9688700 1 -- 172.21.15.6:0/642775866 wait complete. 2023-11-28T13:01:00.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:00.512 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:00.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.639+0000 7f5ece473700 1 Processor -- start 2023-11-28T13:01:00.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.639+0000 7f5ece473700 1 -- start start 2023-11-28T13:01:00.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.639+0000 7f5ece473700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5ec80b1630 0x7f5ec80cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.639+0000 7f5ece473700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec80ce550 con 0x7f5ec806afc0 2023-11-28T13:01:00.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.639+0000 7f5ece473700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec80ce830 con 0x7f5ec80b1a00 2023-11-28T13:01:00.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.639+0000 7f5ece473700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec80ceb10 con 0x7f5ec80b1630 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ec7fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5ec80b1630 0x7f5ec80cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ecca10700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f5ec806afc0 0x7f5ec806b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46154/0 (socket says 172.21.15.6:46154) 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ecca10700 1 -- 172.21.15.6:0/982324708 learned_addr learned my addr 172.21.15.6:0/982324708 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ec6ffd700 1 -- 172.21.15.6:0/982324708 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525359903 0 0) 0x7f5ec80ce550 con 0x7f5ec806afc0 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ec6ffd700 1 -- 172.21.15.6:0/982324708 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eb00035b0 con 0x7f5ec806afc0 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ec7fff700 1 -- 172.21.15.6:0/982324708 >> v1:172.21.15.61:6789/0 conn(0x7f5ec80b1a00 legacy=0x7f5ec80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ec7fff700 1 -- 172.21.15.6:0/982324708 >> v1:172.21.15.6:6789/0 conn(0x7f5ec806afc0 legacy=0x7f5ec806b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ec7fff700 1 -- 172.21.15.6:0/982324708 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80cedf0 con 0x7f5ec80b1630 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ec7fff700 1 --2- 172.21.15.6:0/982324708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 0x7f5ec80cf5f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009870 tx=0x7f5eb8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f575924a1e5b69 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ec6ffd700 1 -- 172.21.15.6:0/982324708 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8004230 con 0x7f5ec80b1630 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ec6ffd700 1 -- 172.21.15.6:0/982324708 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb8004390 con 0x7f5ec80b1630 2023-11-28T13:01:00.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.640+0000 7f5ec6ffd700 1 -- 172.21.15.6:0/982324708 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80036a0 con 0x7f5ec80b1630 2023-11-28T13:01:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.641+0000 7f5ece473700 1 -- 172.21.15.6:0/982324708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 msgr2=0x7f5ec80cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.641+0000 7f5ece473700 1 --2- 172.21.15.6:0/982324708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 0x7f5ec80cf5f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009870 tx=0x7f5eb8009b80 comp rx=0 tx=0).stop 2023-11-28T13:01:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.641+0000 7f5ece473700 1 -- 172.21.15.6:0/982324708 shutdown_connections 2023-11-28T13:01:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.641+0000 7f5ece473700 1 --2- 172.21.15.6:0/982324708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 0x7f5ec80cf5f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.641+0000 7f5ece473700 1 -- 172.21.15.6:0/982324708 >> 172.21.15.6:0/982324708 conn(0x7f5ec80cc630 msgr2=0x7f5ec80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.641+0000 7f5ece473700 1 -- 172.21.15.6:0/982324708 shutdown_connections 2023-11-28T13:01:00.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.641+0000 7f5ece473700 1 -- 172.21.15.6:0/982324708 wait complete. 2023-11-28T13:01:00.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.642+0000 7f5ece473700 1 Processor -- start 2023-11-28T13:01:00.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.643+0000 7f5ece473700 1 -- start start 2023-11-28T13:01:00.646 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ece473700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5ec806afc0 0x7f5ec80c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ece473700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 0x7f5ec81e6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ece473700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ec80b1a00 0x7f5ec80ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ece473700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec81e6c30 con 0x7f5ec80b1630 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ece473700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec81e6f10 con 0x7f5ec806afc0 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ece473700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec81e71f0 con 0x7f5ec80b1a00 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 0x7f5ec81e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 0x7f5ec81e6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58838/0 (socket says 172.21.15.6:58838) 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec77fe700 1 -- 172.21.15.6:0/3576876137 learned_addr learned my addr 172.21.15.6:0/3576876137 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec7fff700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5ec806afc0 0x7f5ec80c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ecca10700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ec80b1a00 0x7f5ec80ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec77fe700 1 -- 172.21.15.6:0/3576876137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ec80b1a00 msgr2=0x7f5ec80ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec77fe700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ec80b1a00 0x7f5ec80ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec77fe700 1 -- 172.21.15.6:0/3576876137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5ec806afc0 msgr2=0x7f5ec80c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec77fe700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5ec806afc0 0x7f5ec80c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec77fe700 1 -- 172.21.15.6:0/3576876137 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb80094a0 con 0x7f5ec80b1630 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ecca10700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ec80b1a00 0x7f5ec80ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec77fe700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 0x7f5ec81e6530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00b4a0 tx=0x7f5ebc00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec4ff9700 1 -- 172.21.15.6:0/3576876137 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc011830 con 0x7f5ec80b1630 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec4ff9700 1 -- 172.21.15.6:0/3576876137 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc011990 con 0x7f5ec80b1630 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ece473700 1 -- 172.21.15.6:0/3576876137 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec80b1ea0 con 0x7f5ec80b1630 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.644+0000 7f5ec4ff9700 1 -- 172.21.15.6:0/3576876137 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc011d40 con 0x7f5ec80b1630 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.645+0000 7f5ece473700 1 -- 172.21.15.6:0/3576876137 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec80cf0d0 con 0x7f5ec80b1630 2023-11-28T13:01:00.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.645+0000 7f5ece473700 1 -- 172.21.15.6:0/3576876137 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec80cb5b0 con 0x7f5ec80b1630 2023-11-28T13:01:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.646+0000 7f5ec4ff9700 1 -- 172.21.15.6:0/3576876137 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ebc00f050 con 0x7f5ec80b1630 2023-11-28T13:01:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.646+0000 7f5ec4ff9700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5eb40682f0 0x7f5eb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.646+0000 7f5ec4ff9700 1 -- 172.21.15.6:0/3576876137 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5ebc085ac0 con 0x7f5ec80b1630 2023-11-28T13:01:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.646+0000 7f5ec7fff700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5eb40682f0 0x7f5eb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.647+0000 7f5ec7fff700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5eb40682f0 0x7f5eb406a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009840 tx=0x7f5eb801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.651+0000 7f5ec4ff9700 1 -- 172.21.15.6:0/3576876137 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5ebc052030 con 0x7f5ec80b1630 2023-11-28T13:01:00.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.842+0000 7f5ece473700 1 -- 172.21.15.6:0/3576876137 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5ec80739f0 con 0x7f5ec80b1630 2023-11-28T13:01:00.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.843+0000 7f5ec4ff9700 1 -- 172.21.15.6:0/3576876137 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5ebc0150a0 con 0x7f5ec80b1630 2023-11-28T13:01:00.844 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:00.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.844+0000 7f5ece473700 1 -- 172.21.15.6:0/3576876137 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5eb40682f0 msgr2=0x7f5eb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.844+0000 7f5ece473700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5eb40682f0 0x7f5eb406a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009840 tx=0x7f5eb801c040 comp rx=0 tx=0).stop 2023-11-28T13:01:00.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.845+0000 7f5ece473700 1 -- 172.21.15.6:0/3576876137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 msgr2=0x7f5ec81e6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.845+0000 7f5ece473700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 0x7f5ec81e6530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00b4a0 tx=0x7f5ebc00b7b0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.845+0000 7f5ece473700 1 -- 172.21.15.6:0/3576876137 shutdown_connections 2023-11-28T13:01:00.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.845+0000 7f5ece473700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5eb40682f0 0x7f5eb406a7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.845+0000 7f5ece473700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5ec80b1a00 0x7f5ec80ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.845+0000 7f5ece473700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5ec806afc0 0x7f5ec80c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.845+0000 7f5ece473700 1 --2- 172.21.15.6:0/3576876137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5ec80b1630 0x7f5ec81e6530 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.845+0000 7f5ece473700 1 -- 172.21.15.6:0/3576876137 >> 172.21.15.6:0/3576876137 conn(0x7f5ec80cc630 msgr2=0x7f5ec80ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.845+0000 7f5ece473700 1 -- 172.21.15.6:0/3576876137 shutdown_connections 2023-11-28T13:01:00.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.845+0000 7f5ece473700 1 -- 172.21.15.6:0/3576876137 wait complete. 2023-11-28T13:01:00.858 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:00.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.984+0000 7f7eb0e8f700 1 Processor -- start 2023-11-28T13:01:00.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.984+0000 7f7eb0e8f700 1 -- start start 2023-11-28T13:01:00.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eb0e8f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b62d0 0x7f7eac0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eb0e8f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7eac0b2d90 0x7f7eac0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eb0e8f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eac0cd2a0 con 0x7f7eac06f8b0 2023-11-28T13:01:00.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eb0e8f700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eac0d0400 con 0x7f7eac0b2d90 2023-11-28T13:01:00.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eb0e8f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac0d06e0 con 0x7f7eac0b62d0 2023-11-28T13:01:00.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eaad9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7eac0b2d90 0x7f7eac0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eaad9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7eac0b2d90 0x7f7eac0ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58848/0 (socket says 172.21.15.6:58848) 2023-11-28T13:01:00.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eaad9d700 1 -- 172.21.15.6:0/4164072924 learned_addr learned my addr 172.21.15.6:0/4164072924 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:00.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eaa59c700 1 --2- 172.21.15.6:0/4164072924 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b62d0 0x7f7eac0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eaad9d700 1 -- 172.21.15.6:0/4164072924 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b62d0 msgr2=0x7f7eac0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:00.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eaad9d700 1 --2- 172.21.15.6:0/4164072924 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b62d0 0x7f7eac0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.985+0000 7f7eaad9d700 1 -- 172.21.15.6:0/4164072924 >> v1:172.21.15.61:6789/0 conn(0x7f7eac06f8b0 legacy=0x7f7eac0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7eaad9d700 1 -- 172.21.15.6:0/4164072924 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac0cffb0 con 0x7f7eac0b2d90 2023-11-28T13:01:00.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7eaa59c700 1 --2- 172.21.15.6:0/4164072924 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b62d0 0x7f7eac0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:00.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7eaad9d700 1 --2- 172.21.15.6:0/4164072924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac0b2d90 0x7f7eac0ce7d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009e50 tx=0x7f7ea400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61960569fa5eb13 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7ea959a700 1 -- 172.21.15.6:0/4164072924 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400e070 con 0x7f7eac0b2d90 2023-11-28T13:01:00.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7ea959a700 1 -- 172.21.15.6:0/4164072924 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea400b9c0 con 0x7f7eac0b2d90 2023-11-28T13:01:00.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7ea959a700 1 -- 172.21.15.6:0/4164072924 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400bd70 con 0x7f7eac0b2d90 2023-11-28T13:01:00.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/4164072924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac0b2d90 msgr2=0x7f7eac0ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7eb0e8f700 1 --2- 172.21.15.6:0/4164072924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac0b2d90 0x7f7eac0ce7d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009e50 tx=0x7f7ea400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:00.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/4164072924 shutdown_connections 2023-11-28T13:01:00.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7eb0e8f700 1 --2- 172.21.15.6:0/4164072924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac0b2d90 0x7f7eac0ce7d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7eb0e8f700 1 --2- 172.21.15.6:0/4164072924 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b62d0 0x7f7eac0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.986+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/4164072924 >> 172.21.15.6:0/4164072924 conn(0x7f7eac0cd680 msgr2=0x7f7eac0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:00.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.987+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/4164072924 shutdown_connections 2023-11-28T13:01:00.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.987+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/4164072924 wait complete. 2023-11-28T13:01:00.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.988+0000 7f7eb0e8f700 1 Processor -- start 2023-11-28T13:01:00.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.988+0000 7f7eb0e8f700 1 -- start start 2023-11-28T13:01:00.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eb0e8f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac06f8b0 0x7f7eac14d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eb0e8f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b2d90 0x7f7eac0ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eb0e8f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7eac0b62d0 0x7f7eac0aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eb0e8f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac14d610 con 0x7f7eac06f8b0 2023-11-28T13:01:00.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eb0e8f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac0a8520 con 0x7f7eac0b62d0 2023-11-28T13:01:00.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eb0e8f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac0a8800 con 0x7f7eac0b2d90 2023-11-28T13:01:00.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eaa59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac06f8b0 0x7f7eac14d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eaa59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac06f8b0 0x7f7eac14d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58850/0 (socket says 172.21.15.6:58850) 2023-11-28T13:01:00.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eaa59c700 1 -- 172.21.15.6:0/773551154 learned_addr learned my addr 172.21.15.6:0/773551154 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:00.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eaad9d700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7eac0b62d0 0x7f7eac0aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7ea9d9b700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b2d90 0x7f7eac0ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eaa59c700 1 -- 172.21.15.6:0/773551154 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b2d90 msgr2=0x7f7eac0ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eaa59c700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b2d90 0x7f7eac0ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eaa59c700 1 -- 172.21.15.6:0/773551154 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7eac0b62d0 msgr2=0x7f7eac0aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:00.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eaa59c700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7eac0b62d0 0x7f7eac0aa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:00.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eaa59c700 1 -- 172.21.15.6:0/773551154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea4009b30 con 0x7f7eac06f8b0 2023-11-28T13:01:00.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eaa59c700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac06f8b0 0x7f7eac14d0d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000e810 tx=0x7f7ea000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7e9affd700 1 -- 172.21.15.6:0/773551154 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000c020 con 0x7f7eac06f8b0 2023-11-28T13:01:00.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7e9affd700 1 -- 172.21.15.6:0/773551154 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea0004250 con 0x7f7eac06f8b0 2023-11-28T13:01:00.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/773551154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eac06fd20 con 0x7f7eac06f8b0 2023-11-28T13:01:00.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7e9affd700 1 -- 172.21.15.6:0/773551154 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00036a0 con 0x7f7eac06f8b0 2023-11-28T13:01:00.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.989+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/773551154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eac0b66a0 con 0x7f7eac06f8b0 2023-11-28T13:01:00.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.990+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/773551154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eac0cce50 con 0x7f7eac06f8b0 2023-11-28T13:01:00.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.991+0000 7f7e9affd700 1 -- 172.21.15.6:0/773551154 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7ea0022070 con 0x7f7eac06f8b0 2023-11-28T13:01:00.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.991+0000 7f7e9affd700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e8c0682f0 0x7f7e8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:00.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.991+0000 7f7e9affd700 1 -- 172.21.15.6:0/773551154 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7ea0085e10 con 0x7f7eac06f8b0 2023-11-28T13:01:00.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.991+0000 7f7ea9d9b700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e8c0682f0 0x7f7e8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:00.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.992+0000 7f7ea9d9b700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e8c0682f0 0x7f7e8c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7eac15db50 tx=0x7f7e9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:00.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:00.996+0000 7f7e9affd700 1 -- 172.21.15.6:0/773551154 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7ea0089030 con 0x7f7eac06f8b0 2023-11-28T13:01:01.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.188+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/773551154 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7eac0cb630 con 0x7f7eac06f8b0 2023-11-28T13:01:01.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.189+0000 7f7e9affd700 1 -- 172.21.15.6:0/773551154 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ea0055cc0 con 0x7f7eac06f8b0 2023-11-28T13:01:01.190 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:01.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/773551154 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e8c0682f0 msgr2=0x7f7e8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e8c0682f0 0x7f7e8c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7eac15db50 tx=0x7f7e9c009040 comp rx=0 tx=0).stop 2023-11-28T13:01:01.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/773551154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac06f8b0 msgr2=0x7f7eac14d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac06f8b0 0x7f7eac14d0d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000e810 tx=0x7f7ea000ebd0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/773551154 shutdown_connections 2023-11-28T13:01:01.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7eac0b62d0 0x7f7eac0aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7eac06f8b0 0x7f7eac14d0d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e8c0682f0 0x7f7e8c06a7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 --2- 172.21.15.6:0/773551154 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7eac0b2d90 0x7f7eac0ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/773551154 >> 172.21.15.6:0/773551154 conn(0x7f7eac0cd680 msgr2=0x7f7eac0b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:01.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/773551154 shutdown_connections 2023-11-28T13:01:01.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.191+0000 7f7eb0e8f700 1 -- 172.21.15.6:0/773551154 wait complete. 2023-11-28T13:01:01.204 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:01.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.331+0000 7f79572bb700 1 Processor -- start 2023-11-28T13:01:01.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.331+0000 7f79572bb700 1 -- start start 2023-11-28T13:01:01.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f79572bb700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f79500cf5f0 0x7f79500caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f79572bb700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79500ce550 con 0x7f79500b1630 2023-11-28T13:01:01.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f79572bb700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79500ce830 con 0x7f79500cb3e0 2023-11-28T13:01:01.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f79572bb700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79500ceb10 con 0x7f79500cf5f0 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f7954856700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f79500cf5f0 0x7f79500caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f7955057700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f79500b1630 0x7f79500b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46178/0 (socket says 172.21.15.6:46178) 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f7955057700 1 -- 172.21.15.6:0/1212380882 learned_addr learned my addr 172.21.15.6:0/1212380882 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f7947fff700 1 -- 172.21.15.6:0/1212380882 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1867236363 0 0) 0x7f79500ce550 con 0x7f79500b1630 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f7947fff700 1 -- 172.21.15.6:0/1212380882 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f793c0035b0 con 0x7f79500b1630 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f7954856700 1 -- 172.21.15.6:0/1212380882 >> v1:172.21.15.61:6789/0 conn(0x7f79500cb3e0 legacy=0x7f795006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f7954856700 1 -- 172.21.15.6:0/1212380882 >> v1:172.21.15.6:6789/0 conn(0x7f79500b1630 legacy=0x7f79500b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f7954856700 1 -- 172.21.15.6:0/1212380882 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500cedf0 con 0x7f79500cf5f0 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f7954856700 1 --2- 172.21.15.6:0/1212380882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500cf5f0 0x7f79500caea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7940009870 tx=0x7f7940009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ec3852df1e3104 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.332+0000 7f7947fff700 1 -- 172.21.15.6:0/1212380882 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940011830 con 0x7f79500cf5f0 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.333+0000 7f7947fff700 1 -- 172.21.15.6:0/1212380882 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7940011990 con 0x7f79500cf5f0 2023-11-28T13:01:01.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.333+0000 7f7947fff700 1 -- 172.21.15.6:0/1212380882 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940011d20 con 0x7f79500cf5f0 2023-11-28T13:01:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.333+0000 7f79572bb700 1 -- 172.21.15.6:0/1212380882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500cf5f0 msgr2=0x7f79500caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.333+0000 7f79572bb700 1 --2- 172.21.15.6:0/1212380882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500cf5f0 0x7f79500caea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7940009870 tx=0x7f7940009b80 comp rx=0 tx=0).stop 2023-11-28T13:01:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.333+0000 7f79572bb700 1 -- 172.21.15.6:0/1212380882 shutdown_connections 2023-11-28T13:01:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.333+0000 7f79572bb700 1 --2- 172.21.15.6:0/1212380882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500cf5f0 0x7f79500caea0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.333+0000 7f79572bb700 1 -- 172.21.15.6:0/1212380882 >> 172.21.15.6:0/1212380882 conn(0x7f79500cc630 msgr2=0x7f79500cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.333+0000 7f79572bb700 1 -- 172.21.15.6:0/1212380882 shutdown_connections 2023-11-28T13:01:01.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.333+0000 7f79572bb700 1 -- 172.21.15.6:0/1212380882 wait complete. 2023-11-28T13:01:01.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.334+0000 7f79572bb700 1 Processor -- start 2023-11-28T13:01:01.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.334+0000 7f79572bb700 1 -- start start 2023-11-28T13:01:01.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f79572bb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500b1630 0x7f7950143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f79572bb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79500cb3e0 0x7f79500c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f79572bb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79500cf5f0 0x7f795015f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f79572bb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795015f9c0 con 0x7f79500b1630 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f79572bb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79500b3a30 con 0x7f79500cb3e0 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f79572bb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79500b3c80 con 0x7f79500cf5f0 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f7955057700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500b1630 0x7f7950143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f7955057700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500b1630 0x7f7950143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58864/0 (socket says 172.21.15.6:58864) 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f7955057700 1 -- 172.21.15.6:0/453340753 learned_addr learned my addr 172.21.15.6:0/453340753 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f7954856700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79500cb3e0 0x7f79500c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f7955057700 1 -- 172.21.15.6:0/453340753 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79500cf5f0 msgr2=0x7f795015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f7955057700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79500cf5f0 0x7f795015f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f7955057700 1 -- 172.21.15.6:0/453340753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79500cb3e0 msgr2=0x7f79500c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f7955057700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79500cb3e0 0x7f79500c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.335+0000 7f7955057700 1 -- 172.21.15.6:0/453340753 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400094a0 con 0x7f79500b1630 2023-11-28T13:01:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.336+0000 7f7955057700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500b1630 0x7f7950143bd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c700 tx=0x7f794c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.336+0000 7f7945ffb700 1 -- 172.21.15.6:0/453340753 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c005d70 con 0x7f79500b1630 2023-11-28T13:01:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.336+0000 7f7945ffb700 1 -- 172.21.15.6:0/453340753 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c0070d0 con 0x7f79500b1630 2023-11-28T13:01:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.336+0000 7f79572bb700 1 -- 172.21.15.6:0/453340753 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79500ce140 con 0x7f79500b1630 2023-11-28T13:01:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.336+0000 7f7945ffb700 1 -- 172.21.15.6:0/453340753 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c007460 con 0x7f79500b1630 2023-11-28T13:01:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.336+0000 7f79572bb700 1 -- 172.21.15.6:0/453340753 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79500cf0d0 con 0x7f79500b1630 2023-11-28T13:01:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.337+0000 7f79572bb700 1 -- 172.21.15.6:0/453340753 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79500ce2d0 con 0x7f79500b1630 2023-11-28T13:01:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.337+0000 7f7945ffb700 1 -- 172.21.15.6:0/453340753 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f794c019030 con 0x7f79500b1630 2023-11-28T13:01:01.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.338+0000 7f7945ffb700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f79340682f0 0x7f793406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.338+0000 7f7945ffb700 1 -- 172.21.15.6:0/453340753 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f794c085470 con 0x7f79500b1630 2023-11-28T13:01:01.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.338+0000 7f7954856700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f79340682f0 0x7f793406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.338+0000 7f7954856700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f79340682f0 0x7f793406a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f795014a970 tx=0x7f7940008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.342+0000 7f7945ffb700 1 -- 172.21.15.6:0/453340753 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f794c051960 con 0x7f79500b1630 2023-11-28T13:01:01.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.537+0000 7f79572bb700 1 -- 172.21.15.6:0/453340753 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79500c7780 con 0x7f79500b1630 2023-11-28T13:01:01.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.537+0000 7f7945ffb700 1 -- 172.21.15.6:0/453340753 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f794c054f80 con 0x7f79500b1630 2023-11-28T13:01:01.539 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.539+0000 7f79572bb700 1 -- 172.21.15.6:0/453340753 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f79340682f0 msgr2=0x7f793406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.539+0000 7f79572bb700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f79340682f0 0x7f793406a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f795014a970 tx=0x7f7940008030 comp rx=0 tx=0).stop 2023-11-28T13:01:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.539+0000 7f79572bb700 1 -- 172.21.15.6:0/453340753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500b1630 msgr2=0x7f7950143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.539+0000 7f79572bb700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500b1630 0x7f7950143bd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c700 tx=0x7f794c00cac0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.540+0000 7f79572bb700 1 -- 172.21.15.6:0/453340753 shutdown_connections 2023-11-28T13:01:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.540+0000 7f79572bb700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f79340682f0 0x7f793406a7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.540+0000 7f79572bb700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f79500cf5f0 0x7f795015f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.540+0000 7f79572bb700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f79500b1630 0x7f7950143bd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.540+0000 7f79572bb700 1 --2- 172.21.15.6:0/453340753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f79500cb3e0 0x7f79500c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.540+0000 7f79572bb700 1 -- 172.21.15.6:0/453340753 >> 172.21.15.6:0/453340753 conn(0x7f79500cc630 msgr2=0x7f79500cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.540+0000 7f79572bb700 1 -- 172.21.15.6:0/453340753 shutdown_connections 2023-11-28T13:01:01.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.540+0000 7f79572bb700 1 -- 172.21.15.6:0/453340753 wait complete. 2023-11-28T13:01:01.553 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:01.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.679+0000 7f4dc7242700 1 Processor -- start 2023-11-28T13:01:01.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.679+0000 7f4dc7242700 1 -- start start 2023-11-28T13:01:01.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dc7242700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4dc00cf5f0 0x7f4dc00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dc7242700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4dc00cb3e0 0x7f4dc006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dc7242700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc00ce550 con 0x7f4dc00b1630 2023-11-28T13:01:01.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dc7242700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc00ce830 con 0x7f4dc00cf5f0 2023-11-28T13:01:01.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dc7242700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc00ceb10 con 0x7f4dc00cb3e0 2023-11-28T13:01:01.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dbffff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4dc00cf5f0 0x7f4dc00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dbffff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4dc00cf5f0 0x7f4dc00caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58874/0 (socket says 172.21.15.6:58874) 2023-11-28T13:01:01.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dbffff700 1 -- 172.21.15.6:0/58739862 learned_addr learned my addr 172.21.15.6:0/58739862 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:01.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dc57df700 1 --2- 172.21.15.6:0/58739862 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4dc00cb3e0 0x7f4dc006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dbffff700 1 -- 172.21.15.6:0/58739862 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4dc00cb3e0 msgr2=0x7f4dc006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:01.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dbffff700 1 --2- 172.21.15.6:0/58739862 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4dc00cb3e0 0x7f4dc006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dbffff700 1 -- 172.21.15.6:0/58739862 >> v1:172.21.15.61:6789/0 conn(0x7f4dc00b1630 legacy=0x7f4dc00b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dbffff700 1 -- 172.21.15.6:0/58739862 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00cedf0 con 0x7f4dc00cf5f0 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.680+0000 7f4dbffff700 1 --2- 172.21.15.6:0/58739862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00cf5f0 0x7f4dc00caea0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009920 tx=0x7f4dac009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe871ffc38ac888 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dbf7fe700 1 -- 172.21.15.6:0/58739862 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00ba20 con 0x7f4dc00cf5f0 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dbf7fe700 1 -- 172.21.15.6:0/58739862 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac00bb80 con 0x7f4dc00cf5f0 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dbf7fe700 1 -- 172.21.15.6:0/58739862 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac009e70 con 0x7f4dc00cf5f0 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dc7242700 1 -- 172.21.15.6:0/58739862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00cf5f0 msgr2=0x7f4dc00caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dc7242700 1 --2- 172.21.15.6:0/58739862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00cf5f0 0x7f4dc00caea0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009920 tx=0x7f4dac009c30 comp rx=0 tx=0).stop 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dc7242700 1 -- 172.21.15.6:0/58739862 shutdown_connections 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dc7242700 1 --2- 172.21.15.6:0/58739862 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4dc00cb3e0 0x7f4dc006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dc7242700 1 --2- 172.21.15.6:0/58739862 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00cf5f0 0x7f4dc00caea0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dc7242700 1 -- 172.21.15.6:0/58739862 >> 172.21.15.6:0/58739862 conn(0x7f4dc00cc630 msgr2=0x7f4dc00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:01.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dc7242700 1 -- 172.21.15.6:0/58739862 shutdown_connections 2023-11-28T13:01:01.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.681+0000 7f4dc7242700 1 -- 172.21.15.6:0/58739862 wait complete. 2023-11-28T13:01:01.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.682+0000 7f4dc7242700 1 Processor -- start 2023-11-28T13:01:01.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.683+0000 7f4dc7242700 1 -- start start 2023-11-28T13:01:01.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.683+0000 7f4dc7242700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00b1630 0x7f4dc015f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.683+0000 7f4dc7242700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4dc00cb3e0 0x7f4dc00b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.683+0000 7f4dc7242700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dc00cf5f0 0x7f4dc0159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.683+0000 7f4dc7242700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc00c7ec0 con 0x7f4dc00b1630 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.683+0000 7f4dc7242700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc01e4ff0 con 0x7f4dc00cf5f0 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.683+0000 7f4dc7242700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc015fbf0 con 0x7f4dc00cb3e0 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.683+0000 7f4dc4fde700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00b1630 0x7f4dc015f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.683+0000 7f4dc4fde700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00b1630 0x7f4dc015f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58878/0 (socket says 172.21.15.6:58878) 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.683+0000 7f4dc4fde700 1 -- 172.21.15.6:0/1733252601 learned_addr learned my addr 172.21.15.6:0/1733252601 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dc4fde700 1 -- 172.21.15.6:0/1733252601 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4dc00cb3e0 msgr2=0x7f4dc00b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dc4fde700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4dc00cb3e0 0x7f4dc00b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dc4fde700 1 -- 172.21.15.6:0/1733252601 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dc00cf5f0 msgr2=0x7f4dc0159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dc4fde700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dc00cf5f0 0x7f4dc0159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dc4fde700 1 -- 172.21.15.6:0/1733252601 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac009550 con 0x7f4dc00b1630 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dc4fde700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00b1630 0x7f4dc015f6b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4db400c510 tx=0x7f4db400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dbd7fa700 1 -- 172.21.15.6:0/1733252601 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db40052d0 con 0x7f4dc00b1630 2023-11-28T13:01:01.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dbd7fa700 1 -- 172.21.15.6:0/1733252601 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db4005430 con 0x7f4dc00b1630 2023-11-28T13:01:01.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dc7242700 1 -- 172.21.15.6:0/1733252601 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc00ce140 con 0x7f4dc00b1630 2023-11-28T13:01:01.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dbd7fa700 1 -- 172.21.15.6:0/1733252601 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db40038c0 con 0x7f4dc00b1630 2023-11-28T13:01:01.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.684+0000 7f4dc7242700 1 -- 172.21.15.6:0/1733252601 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc00cf0d0 con 0x7f4dc00b1630 2023-11-28T13:01:01.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.685+0000 7f4dc7242700 1 -- 172.21.15.6:0/1733252601 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc00ce2d0 con 0x7f4dc00b1630 2023-11-28T13:01:01.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.685+0000 7f4dbd7fa700 1 -- 172.21.15.6:0/1733252601 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4db4019030 con 0x7f4dc00b1630 2023-11-28T13:01:01.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.686+0000 7f4dbd7fa700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4da40682f0 0x7f4da406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:01.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.686+0000 7f4dbd7fa700 1 -- 172.21.15.6:0/1733252601 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4db4085330 con 0x7f4dc00b1630 2023-11-28T13:01:01.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.686+0000 7f4dbffff700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4da40682f0 0x7f4da406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:01.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.686+0000 7f4dbffff700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4da40682f0 0x7f4da406a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009c70 tx=0x7f4dac010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:01.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.690+0000 7f4dbd7fa700 1 -- 172.21.15.6:0/1733252601 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4db40518a0 con 0x7f4dc00b1630 2023-11-28T13:01:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.886+0000 7f4dc7242700 1 -- 172.21.15.6:0/1733252601 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4dc00739f0 con 0x7f4dc00b1630 2023-11-28T13:01:01.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.887+0000 7f4dbd7fa700 1 -- 172.21.15.6:0/1733252601 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4db4054ec0 con 0x7f4dc00b1630 2023-11-28T13:01:01.888 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 -- 172.21.15.6:0/1733252601 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4da40682f0 msgr2=0x7f4da406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4da40682f0 0x7f4da406a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009c70 tx=0x7f4dac010000 comp rx=0 tx=0).stop 2023-11-28T13:01:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 -- 172.21.15.6:0/1733252601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00b1630 msgr2=0x7f4dc015f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:01.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00b1630 0x7f4dc015f6b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4db400c510 tx=0x7f4db400c8d0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 -- 172.21.15.6:0/1733252601 shutdown_connections 2023-11-28T13:01:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dc00cf5f0 0x7f4dc0159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc00b1630 0x7f4dc015f6b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4da40682f0 0x7f4da406a7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 --2- 172.21.15.6:0/1733252601 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4dc00cb3e0 0x7f4dc00b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 -- 172.21.15.6:0/1733252601 >> 172.21.15.6:0/1733252601 conn(0x7f4dc00cc630 msgr2=0x7f4dc00b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 -- 172.21.15.6:0/1733252601 shutdown_connections 2023-11-28T13:01:01.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:01.889+0000 7f4dc7242700 1 -- 172.21.15.6:0/1733252601 wait complete. 2023-11-28T13:01:01.903 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:02.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.031+0000 7f1a8ea87700 1 Processor -- start 2023-11-28T13:01:02.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.031+0000 7f1a8ea87700 1 -- start start 2023-11-28T13:01:02.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a8ea87700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a881592b0 0x7f1a880cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a8ea87700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1a88070960 0x7f1a88070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a8ea87700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8815a4a0 con 0x7f1a880ca880 2023-11-28T13:01:02.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a8ea87700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a88071310 con 0x7f1a88070960 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a8ea87700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8815a780 con 0x7f1a881592b0 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a87fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1a88070960 0x7f1a88070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a8d024700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f1a880ca880 0x7f1a880caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:36240/0 (socket says 172.21.15.6:36240) 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a8d024700 1 -- 172.21.15.6:0/4269377355 learned_addr learned my addr 172.21.15.6:0/4269377355 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a8c823700 1 --2- 172.21.15.6:0/4269377355 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a881592b0 0x7f1a880cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a877fe700 1 -- 172.21.15.6:0/4269377355 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096613832 0 0) 0x7f1a8815a4a0 con 0x7f1a880ca880 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a877fe700 1 -- 172.21.15.6:0/4269377355 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a780035b0 con 0x7f1a880ca880 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a87fff700 1 -- 172.21.15.6:0/4269377355 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a881592b0 msgr2=0x7f1a880cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a87fff700 1 --2- 172.21.15.6:0/4269377355 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a881592b0 0x7f1a880cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.032+0000 7f1a87fff700 1 -- 172.21.15.6:0/4269377355 >> v1:172.21.15.61:6789/0 conn(0x7f1a880ca880 legacy=0x7f1a880caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.033+0000 7f1a87fff700 1 -- 172.21.15.6:0/4269377355 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a880ca080 con 0x7f1a88070960 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.033+0000 7f1a87fff700 1 --2- 172.21.15.6:0/4269377355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a88070960 0x7f1a88070dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009840 tx=0x7f1a74009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4b48631fcde1d2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.033+0000 7f1a877fe700 1 -- 172.21.15.6:0/4269377355 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7400f8a0 con 0x7f1a88070960 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.033+0000 7f1a877fe700 1 -- 172.21.15.6:0/4269377355 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7400fa00 con 0x7f1a88070960 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.033+0000 7f1a877fe700 1 -- 172.21.15.6:0/4269377355 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7400fd70 con 0x7f1a88070960 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.033+0000 7f1a8ea87700 1 -- 172.21.15.6:0/4269377355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a88070960 msgr2=0x7f1a88070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.033+0000 7f1a8ea87700 1 --2- 172.21.15.6:0/4269377355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a88070960 0x7f1a88070dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009840 tx=0x7f1a74009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.033+0000 7f1a8ea87700 1 -- 172.21.15.6:0/4269377355 shutdown_connections 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.033+0000 7f1a8ea87700 1 --2- 172.21.15.6:0/4269377355 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a881592b0 0x7f1a880cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.033+0000 7f1a8ea87700 1 --2- 172.21.15.6:0/4269377355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a88070960 0x7f1a88070dd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.034+0000 7f1a8ea87700 1 -- 172.21.15.6:0/4269377355 >> 172.21.15.6:0/4269377355 conn(0x7f1a880ce9e0 msgr2=0x7f1a880cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.034+0000 7f1a8ea87700 1 -- 172.21.15.6:0/4269377355 shutdown_connections 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.034+0000 7f1a8ea87700 1 -- 172.21.15.6:0/4269377355 wait complete. 2023-11-28T13:01:02.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.035+0000 7f1a8ea87700 1 Processor -- start 2023-11-28T13:01:02.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.035+0000 7f1a8ea87700 1 -- start start 2023-11-28T13:01:02.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8ea87700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a88070960 0x7f1a880c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8ea87700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a880ca880 0x7f1a880cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8ea87700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a881592b0 0x7f1a881f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8ea87700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a880cb500 con 0x7f1a881592b0 2023-11-28T13:01:02.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8ea87700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a880cb7e0 con 0x7f1a880ca880 2023-11-28T13:01:02.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8ea87700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a880cbac0 con 0x7f1a88070960 2023-11-28T13:01:02.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8d024700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a881592b0 0x7f1a881f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8d024700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a881592b0 0x7f1a881f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58896/0 (socket says 172.21.15.6:58896) 2023-11-28T13:01:02.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8d024700 1 -- 172.21.15.6:0/70833602 learned_addr learned my addr 172.21.15.6:0/70833602 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:02.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a87fff700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a880ca880 0x7f1a880cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8c823700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a88070960 0x7f1a880c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8d024700 1 -- 172.21.15.6:0/70833602 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a88070960 msgr2=0x7f1a880c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8d024700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a88070960 0x7f1a880c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8d024700 1 -- 172.21.15.6:0/70833602 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a880ca880 msgr2=0x7f1a880cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8d024700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a880ca880 0x7f1a880cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8d024700 1 -- 172.21.15.6:0/70833602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a740094a0 con 0x7f1a881592b0 2023-11-28T13:01:02.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8d024700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a881592b0 0x7f1a881f3340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000cb40 tx=0x7f1a8000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a857fa700 1 -- 172.21.15.6:0/70833602 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000a9e0 con 0x7f1a881592b0 2023-11-28T13:01:02.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a857fa700 1 -- 172.21.15.6:0/70833602 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8000ab40 con 0x7f1a881592b0 2023-11-28T13:01:02.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a8ea87700 1 -- 172.21.15.6:0/70833602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a880c9d90 con 0x7f1a881592b0 2023-11-28T13:01:02.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.036+0000 7f1a857fa700 1 -- 172.21.15.6:0/70833602 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80012400 con 0x7f1a881592b0 2023-11-28T13:01:02.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.037+0000 7f1a8ea87700 1 -- 172.21.15.6:0/70833602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a880ca360 con 0x7f1a881592b0 2023-11-28T13:01:02.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.037+0000 7f1a8ea87700 1 -- 172.21.15.6:0/70833602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8815d1d0 con 0x7f1a881592b0 2023-11-28T13:01:02.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.038+0000 7f1a857fa700 1 -- 172.21.15.6:0/70833602 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a80019030 con 0x7f1a881592b0 2023-11-28T13:01:02.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.038+0000 7f1a857fa700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a6c0682f0 0x7f1a6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.038+0000 7f1a857fa700 1 -- 172.21.15.6:0/70833602 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1a80085a60 con 0x7f1a881592b0 2023-11-28T13:01:02.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.038+0000 7f1a8c823700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a6c0682f0 0x7f1a6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.039+0000 7f1a8c823700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a6c0682f0 0x7f1a6c06a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009b40 tx=0x7f1a7c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.043+0000 7f1a857fa700 1 -- 172.21.15.6:0/70833602 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a8004b5e0 con 0x7f1a881592b0 2023-11-28T13:01:02.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.235+0000 7f1a8ea87700 1 -- 172.21.15.6:0/70833602 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1a880739f0 con 0x7f1a881592b0 2023-11-28T13:01:02.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.236+0000 7f1a857fa700 1 -- 172.21.15.6:0/70833602 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1a80051f90 con 0x7f1a881592b0 2023-11-28T13:01:02.239 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:02.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.237+0000 7f1a8ea87700 1 -- 172.21.15.6:0/70833602 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a6c0682f0 msgr2=0x7f1a6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.237+0000 7f1a8ea87700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a6c0682f0 0x7f1a6c06a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009b40 tx=0x7f1a7c009210 comp rx=0 tx=0).stop 2023-11-28T13:01:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.238+0000 7f1a8ea87700 1 -- 172.21.15.6:0/70833602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a881592b0 msgr2=0x7f1a881f3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.238+0000 7f1a8ea87700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a881592b0 0x7f1a881f3340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000cb40 tx=0x7f1a8000ce50 comp rx=0 tx=0).stop 2023-11-28T13:01:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.238+0000 7f1a8ea87700 1 -- 172.21.15.6:0/70833602 shutdown_connections 2023-11-28T13:01:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.238+0000 7f1a8ea87700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a881592b0 0x7f1a881f3340 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.238+0000 7f1a8ea87700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a6c0682f0 0x7f1a6c06a7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.238+0000 7f1a8ea87700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a88070960 0x7f1a880c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.238+0000 7f1a8ea87700 1 --2- 172.21.15.6:0/70833602 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a880ca880 0x7f1a880cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.238+0000 7f1a8ea87700 1 -- 172.21.15.6:0/70833602 >> 172.21.15.6:0/70833602 conn(0x7f1a880ce9e0 msgr2=0x7f1a88159680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.238+0000 7f1a8ea87700 1 -- 172.21.15.6:0/70833602 shutdown_connections 2023-11-28T13:01:02.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.238+0000 7f1a8ea87700 1 -- 172.21.15.6:0/70833602 wait complete. 2023-11-28T13:01:02.251 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:02.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.374+0000 7f7aacc55700 1 Processor -- start 2023-11-28T13:01:02.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.374+0000 7f7aacc55700 1 -- start start 2023-11-28T13:01:02.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.375+0000 7f7aacc55700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7aa8158ad0 0x7f7aa80711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.375+0000 7f7aacc55700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7aa80cd620 0x7f7aa8071730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.375+0000 7f7aacc55700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa80cc860 con 0x7f7aa815c8b0 2023-11-28T13:01:02.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.375+0000 7f7aacc55700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa80ccb40 con 0x7f7aa80cd620 2023-11-28T13:01:02.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.375+0000 7f7aacc55700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80cce20 con 0x7f7aa8158ad0 2023-11-28T13:01:02.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.375+0000 7f7aa5d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7aa80cd620 0x7f7aa8071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.375+0000 7f7aa6d9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f7aa815c8b0 0x7f7aa815cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46224/0 (socket says 172.21.15.6:46224) 2023-11-28T13:01:02.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.375+0000 7f7aa6d9d700 1 -- 172.21.15.6:0/2085314745 learned_addr learned my addr 172.21.15.6:0/2085314745 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:02.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aa659c700 1 --2- 172.21.15.6:0/2085314745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7aa8158ad0 0x7f7aa80711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aa5d9b700 1 -- 172.21.15.6:0/2085314745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7aa8158ad0 msgr2=0x7f7aa80711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:02.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aa5d9b700 1 --2- 172.21.15.6:0/2085314745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7aa8158ad0 0x7f7aa80711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aa5d9b700 1 -- 172.21.15.6:0/2085314745 >> v1:172.21.15.6:6789/0 conn(0x7f7aa815c8b0 legacy=0x7f7aa815cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aa5d9b700 1 -- 172.21.15.6:0/2085314745 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa80cd100 con 0x7f7aa80cd620 2023-11-28T13:01:02.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aa559a700 1 -- 172.21.15.6:0/2085314745 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470197269 0 0) 0x7f7aa80cc860 con 0x7f7aa815c8b0 2023-11-28T13:01:02.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aa5d9b700 1 --2- 172.21.15.6:0/2085314745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 0x7f7aa8071730 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009840 tx=0x7f7a9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ada53ac9c20eb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aa559a700 1 -- 172.21.15.6:0/2085314745 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c010800 con 0x7f7aa80cd620 2023-11-28T13:01:02.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aa559a700 1 -- 172.21.15.6:0/2085314745 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c010960 con 0x7f7aa80cd620 2023-11-28T13:01:02.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aa559a700 1 -- 172.21.15.6:0/2085314745 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c010cf0 con 0x7f7aa80cd620 2023-11-28T13:01:02.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aacc55700 1 -- 172.21.15.6:0/2085314745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 msgr2=0x7f7aa8071730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.376+0000 7f7aacc55700 1 --2- 172.21.15.6:0/2085314745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 0x7f7aa8071730 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009840 tx=0x7f7a9c009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:02.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.377+0000 7f7aacc55700 1 -- 172.21.15.6:0/2085314745 shutdown_connections 2023-11-28T13:01:02.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.377+0000 7f7aacc55700 1 --2- 172.21.15.6:0/2085314745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7aa8158ad0 0x7f7aa80711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.377+0000 7f7aacc55700 1 --2- 172.21.15.6:0/2085314745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 0x7f7aa8071730 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.377+0000 7f7aacc55700 1 -- 172.21.15.6:0/2085314745 >> 172.21.15.6:0/2085314745 conn(0x7f7aa80ce9e0 msgr2=0x7f7aa80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:02.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.377+0000 7f7aacc55700 1 -- 172.21.15.6:0/2085314745 shutdown_connections 2023-11-28T13:01:02.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.377+0000 7f7aacc55700 1 -- 172.21.15.6:0/2085314745 wait complete. 2023-11-28T13:01:02.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.378+0000 7f7aacc55700 1 Processor -- start 2023-11-28T13:01:02.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.378+0000 7f7aacc55700 1 -- start start 2023-11-28T13:01:02.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.379+0000 7f7aacc55700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 0x7f7aa81f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.379+0000 7f7aacc55700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7aa8158ad0 0x7f7aa8140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.379+0000 7f7aacc55700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7aa815c8b0 0x7f7aa81550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.379+0000 7f7aacc55700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa81522d0 con 0x7f7aa80cd620 2023-11-28T13:01:02.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.379+0000 7f7aacc55700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa8143b50 con 0x7f7aa8158ad0 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.379+0000 7f7aacc55700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa8143e30 con 0x7f7aa815c8b0 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.379+0000 7f7aa659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 0x7f7aa81f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.379+0000 7f7aa659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 0x7f7aa81f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58906/0 (socket says 172.21.15.6:58906) 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7aa659c700 1 -- 172.21.15.6:0/419412249 learned_addr learned my addr 172.21.15.6:0/419412249 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7aa659c700 1 -- 172.21.15.6:0/419412249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7aa815c8b0 msgr2=0x7f7aa81550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7aa659c700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7aa815c8b0 0x7f7aa81550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7aa659c700 1 -- 172.21.15.6:0/419412249 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7aa8158ad0 msgr2=0x7f7aa8140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7aa659c700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7aa8158ad0 0x7f7aa8140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7aa659c700 1 -- 172.21.15.6:0/419412249 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c0094a0 con 0x7f7aa80cd620 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7aa659c700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 0x7f7aa81f2d60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800e810 tx=0x7f7a9800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7a96ffd700 1 -- 172.21.15.6:0/419412249 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98013610 con 0x7f7aa80cd620 2023-11-28T13:01:02.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7a96ffd700 1 -- 172.21.15.6:0/419412249 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a98013770 con 0x7f7aa80cd620 2023-11-28T13:01:02.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7aacc55700 1 -- 172.21.15.6:0/419412249 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa8158ea0 con 0x7f7aa80cd620 2023-11-28T13:01:02.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7a96ffd700 1 -- 172.21.15.6:0/419412249 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98013b20 con 0x7f7aa80cd620 2023-11-28T13:01:02.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.380+0000 7f7aacc55700 1 -- 172.21.15.6:0/419412249 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa815d610 con 0x7f7aa80cd620 2023-11-28T13:01:02.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.381+0000 7f7aacc55700 1 -- 172.21.15.6:0/419412249 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa8159030 con 0x7f7aa80cd620 2023-11-28T13:01:02.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.381+0000 7f7a96ffd700 1 -- 172.21.15.6:0/419412249 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a98023070 con 0x7f7aa80cd620 2023-11-28T13:01:02.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.382+0000 7f7a96ffd700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a880682f0 0x7f7a8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.382+0000 7f7a96ffd700 1 -- 172.21.15.6:0/419412249 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7a98085c60 con 0x7f7aa80cd620 2023-11-28T13:01:02.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.382+0000 7f7aa5d9b700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a880682f0 0x7f7a8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.382+0000 7f7aa5d9b700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a880682f0 0x7f7a8806a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0090b0 tx=0x7f7a9c011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.386+0000 7f7a96ffd700 1 -- 172.21.15.6:0/419412249 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a98052150 con 0x7f7aa80cd620 2023-11-28T13:01:02.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.580+0000 7f7aacc55700 1 -- 172.21.15.6:0/419412249 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7aa80c3a90 con 0x7f7aa80cd620 2023-11-28T13:01:02.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.581+0000 7f7a96ffd700 1 -- 172.21.15.6:0/419412249 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f7aa80c3a90 con 0x7f7aa80cd620 2023-11-28T13:01:02.583 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:02.584 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:02.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 -- 172.21.15.6:0/419412249 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a880682f0 msgr2=0x7f7a8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a880682f0 0x7f7a8806a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0090b0 tx=0x7f7a9c011000 comp rx=0 tx=0).stop 2023-11-28T13:01:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 -- 172.21.15.6:0/419412249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 msgr2=0x7f7aa81f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 0x7f7aa81f2d60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800e810 tx=0x7f7a9800ebd0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 -- 172.21.15.6:0/419412249 shutdown_connections 2023-11-28T13:01:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7a880682f0 0x7f7a8806a7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7aa815c8b0 0x7f7aa81550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aa80cd620 0x7f7aa81f2d60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 --2- 172.21.15.6:0/419412249 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7aa8158ad0 0x7f7aa8140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 -- 172.21.15.6:0/419412249 >> 172.21.15.6:0/419412249 conn(0x7f7aa80ce9e0 msgr2=0x7f7aa80ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 -- 172.21.15.6:0/419412249 shutdown_connections 2023-11-28T13:01:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.584+0000 7f7aacc55700 1 -- 172.21.15.6:0/419412249 wait complete. 2023-11-28T13:01:02.598 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:02.599 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.725+0000 7f8a721f2700 1 Processor -- start 2023-11-28T13:01:02.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.725+0000 7f8a721f2700 1 -- start start 2023-11-28T13:01:02.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.725+0000 7f8a721f2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8a6c159500 0x7f8a6c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.725+0000 7f8a721f2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8a6c15dbf0 0x7f8a6c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.725+0000 7f8a721f2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6c15e040 con 0x7f8a6c0b1630 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a721f2700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6c0cede0 con 0x7f8a6c159500 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a721f2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6c0ceff0 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8a6c159500 0x7f8a6c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8a6c159500 0x7f8a6c0cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58914/0 (socket says 172.21.15.6:58914) 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6b7fe700 1 -- 172.21.15.6:0/1411397547 learned_addr learned my addr 172.21.15.6:0/1411397547 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6affd700 1 --2- 172.21.15.6:0/1411397547 >> v2:172.21.15.61:3300/0 conn(0x7f8a6c15dbf0 0x7f8a6c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6b7fe700 1 -- 172.21.15.6:0/1411397547 >> v2:172.21.15.61:3300/0 conn(0x7f8a6c15dbf0 msgr2=0x7f8a6c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:02.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6b7fe700 1 --2- 172.21.15.6:0/1411397547 >> v2:172.21.15.61:3300/0 conn(0x7f8a6c15dbf0 0x7f8a6c0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6b7fe700 1 -- 172.21.15.6:0/1411397547 >> v1:172.21.15.61:6789/0 conn(0x7f8a6c0b1630 legacy=0x7f8a6c06bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6b7fe700 1 -- 172.21.15.6:0/1411397547 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c0cf2d0 con 0x7f8a6c159500 2023-11-28T13:01:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6b7fe700 1 --2- 172.21.15.6:0/1411397547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c159500 0x7f8a6c0cd140 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0d0c50 tx=0x7f8a6000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565cd642787d218 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6a7fc700 1 -- 172.21.15.6:0/1411397547 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000dcf0 con 0x7f8a6c159500 2023-11-28T13:01:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6a7fc700 1 -- 172.21.15.6:0/1411397547 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000de50 con 0x7f8a6c159500 2023-11-28T13:01:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.726+0000 7f8a6a7fc700 1 -- 172.21.15.6:0/1411397547 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60004250 con 0x7f8a6c159500 2023-11-28T13:01:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.727+0000 7f8a721f2700 1 -- 172.21.15.6:0/1411397547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c159500 msgr2=0x7f8a6c0cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.727+0000 7f8a721f2700 1 --2- 172.21.15.6:0/1411397547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c159500 0x7f8a6c0cd140 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0d0c50 tx=0x7f8a6000be70 comp rx=0 tx=0).stop 2023-11-28T13:01:02.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.727+0000 7f8a721f2700 1 -- 172.21.15.6:0/1411397547 shutdown_connections 2023-11-28T13:01:02.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.727+0000 7f8a721f2700 1 --2- 172.21.15.6:0/1411397547 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c159500 0x7f8a6c0cd140 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.727+0000 7f8a721f2700 1 --2- 172.21.15.6:0/1411397547 >> v2:172.21.15.61:3300/0 conn(0x7f8a6c15dbf0 0x7f8a6c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.727+0000 7f8a721f2700 1 -- 172.21.15.6:0/1411397547 >> 172.21.15.6:0/1411397547 conn(0x7f8a6c0cc630 msgr2=0x7f8a6c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:02.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.727+0000 7f8a721f2700 1 -- 172.21.15.6:0/1411397547 shutdown_connections 2023-11-28T13:01:02.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.727+0000 7f8a721f2700 1 -- 172.21.15.6:0/1411397547 wait complete. 2023-11-28T13:01:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.728+0000 7f8a721f2700 1 Processor -- start 2023-11-28T13:01:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.728+0000 7f8a721f2700 1 -- start start 2023-11-28T13:01:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a721f2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a6c0b1630 0x7f8a6c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a721f2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a6c159500 0x7f8a6c0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a721f2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c15dbf0 0x7f8a6c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a721f2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c14b790 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a721f2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c0c9a70 con 0x7f8a6c159500 2023-11-28T13:01:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a721f2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c0ac0e0 con 0x7f8a6c0b1630 2023-11-28T13:01:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a6bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c15dbf0 0x7f8a6c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a6bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c15dbf0 0x7f8a6c0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58924/0 (socket says 172.21.15.6:58924) 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a6bfff700 1 -- 172.21.15.6:0/4154427400 learned_addr learned my addr 172.21.15.6:0/4154427400 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a6b7fe700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a6c0b1630 0x7f8a6c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a6affd700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a6c159500 0x7f8a6c0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.729+0000 7f8a6bfff700 1 -- 172.21.15.6:0/4154427400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a6c0b1630 msgr2=0x7f8a6c0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a6bfff700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a6c0b1630 0x7f8a6c0c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a6bfff700 1 -- 172.21.15.6:0/4154427400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a6c159500 msgr2=0x7f8a6c0c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a6bfff700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a6c159500 0x7f8a6c0c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a6bfff700 1 -- 172.21.15.6:0/4154427400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6000b7f0 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a6affd700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a6c159500 0x7f8a6c0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a6bfff700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c15dbf0 0x7f8a6c0c8d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400cc10 tx=0x7f8a6400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a53fff700 1 -- 172.21.15.6:0/4154427400 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400ab10 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a53fff700 1 -- 172.21.15.6:0/4154427400 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6400ac70 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a721f2700 1 -- 172.21.15.6:0/4154427400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c15d3f0 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a53fff700 1 -- 172.21.15.6:0/4154427400 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64012420 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.730+0000 7f8a721f2700 1 -- 172.21.15.6:0/4154427400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c15d110 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.731+0000 7f8a721f2700 1 -- 172.21.15.6:0/4154427400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c15d6d0 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.731+0000 7f8a53fff700 1 -- 172.21.15.6:0/4154427400 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8a64010050 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.732+0000 7f8a53fff700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8a580682f0 0x7f8a5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.732+0000 7f8a53fff700 1 -- 172.21.15.6:0/4154427400 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8a64086160 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.732+0000 7f8a6b7fe700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8a580682f0 0x7f8a5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:02.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.732+0000 7f8a6b7fe700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8a580682f0 0x7f8a5806a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000b340 tx=0x7f8a6001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:02.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.736+0000 7f8a53fff700 1 -- 172.21.15.6:0/4154427400 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8a640526d0 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.930+0000 7f8a721f2700 1 -- 172.21.15.6:0/4154427400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8a6c0715c0 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.930+0000 7f8a53fff700 1 -- 172.21.15.6:0/4154427400 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8a64055cf0 con 0x7f8a6c15dbf0 2023-11-28T13:01:02.932 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:02.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.932+0000 7f8a721f2700 1 -- 172.21.15.6:0/4154427400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8a580682f0 msgr2=0x7f8a5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8a580682f0 0x7f8a5806a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000b340 tx=0x7f8a6001c040 comp rx=0 tx=0).stop 2023-11-28T13:01:02.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 -- 172.21.15.6:0/4154427400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c15dbf0 msgr2=0x7f8a6c0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c15dbf0 0x7f8a6c0c8d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400cc10 tx=0x7f8a6400cf20 comp rx=0 tx=0).stop 2023-11-28T13:01:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 -- 172.21.15.6:0/4154427400 shutdown_connections 2023-11-28T13:01:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a6c15dbf0 0x7f8a6c0c8d90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8a580682f0 0x7f8a5806a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8a6c0b1630 0x7f8a6c0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 --2- 172.21.15.6:0/4154427400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a6c159500 0x7f8a6c0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 -- 172.21.15.6:0/4154427400 >> 172.21.15.6:0/4154427400 conn(0x7f8a6c0cc630 msgr2=0x7f8a6c0cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 -- 172.21.15.6:0/4154427400 shutdown_connections 2023-11-28T13:01:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:02.933+0000 7f8a721f2700 1 -- 172.21.15.6:0/4154427400 wait complete. 2023-11-28T13:01:02.946 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:03.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.074+0000 7fa60fa41700 1 Processor -- start 2023-11-28T13:01:03.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.074+0000 7fa60fa41700 1 -- start start 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60fa41700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa60815a490 0x7fa60806f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60fa41700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa6080b2150 0x7fa60806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60fa41700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60815a860 con 0x7fa60806bcd0 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60fa41700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60806b510 con 0x7fa6080b2150 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60fa41700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6080c9a80 con 0x7fa60815a490 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60cfdc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa6080b2150 0x7fa60806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60cfdc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa6080b2150 0x7fa60806d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58930/0 (socket says 172.21.15.6:58930) 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60cfdc700 1 -- 172.21.15.6:0/1725634689 learned_addr learned my addr 172.21.15.6:0/1725634689 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60d7dd700 1 --2- 172.21.15.6:0/1725634689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa60815a490 0x7fa60806f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60cfdc700 1 -- 172.21.15.6:0/1725634689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa60815a490 msgr2=0x7fa60806f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60cfdc700 1 --2- 172.21.15.6:0/1725634689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa60815a490 0x7fa60806f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60cfdc700 1 -- 172.21.15.6:0/1725634689 >> v1:172.21.15.61:6789/0 conn(0x7fa60806bcd0 legacy=0x7fa60815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60cfdc700 1 -- 172.21.15.6:0/1725634689 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60806b230 con 0x7fa6080b2150 2023-11-28T13:01:03.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa60cfdc700 1 --2- 172.21.15.6:0/1725634689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 0x7fa60806d630 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80098f0 tx=0x7fa5f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6241bda80b49add server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa5fffff700 1 -- 172.21.15.6:0/1725634689 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800b8a0 con 0x7fa6080b2150 2023-11-28T13:01:03.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.075+0000 7fa5fffff700 1 -- 172.21.15.6:0/1725634689 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f800ba00 con 0x7fa6080b2150 2023-11-28T13:01:03.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.076+0000 7fa5fffff700 1 -- 172.21.15.6:0/1725634689 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800bd90 con 0x7fa6080b2150 2023-11-28T13:01:03.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.076+0000 7fa60fa41700 1 -- 172.21.15.6:0/1725634689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 msgr2=0x7fa60806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.076+0000 7fa60fa41700 1 --2- 172.21.15.6:0/1725634689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 0x7fa60806d630 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80098f0 tx=0x7fa5f8009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:03.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.076+0000 7fa60fa41700 1 -- 172.21.15.6:0/1725634689 shutdown_connections 2023-11-28T13:01:03.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.076+0000 7fa60fa41700 1 --2- 172.21.15.6:0/1725634689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa60815a490 0x7fa60806f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.076+0000 7fa60fa41700 1 --2- 172.21.15.6:0/1725634689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 0x7fa60806d630 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.076+0000 7fa60fa41700 1 -- 172.21.15.6:0/1725634689 >> 172.21.15.6:0/1725634689 conn(0x7fa6080b0df0 msgr2=0x7fa6080b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:03.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.076+0000 7fa60fa41700 1 -- 172.21.15.6:0/1725634689 shutdown_connections 2023-11-28T13:01:03.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.076+0000 7fa60fa41700 1 -- 172.21.15.6:0/1725634689 wait complete. 2023-11-28T13:01:03.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.078+0000 7fa60fa41700 1 Processor -- start 2023-11-28T13:01:03.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.078+0000 7fa60fa41700 1 -- start start 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60fa41700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa60806bcd0 0x7fa608158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60fa41700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 0x7fa608158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60fa41700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa60815a490 0x7fa6080c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60fa41700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60813de00 con 0x7fa6080b2150 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60fa41700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60813d1f0 con 0x7fa60815a490 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60fa41700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60813d4d0 con 0x7fa60806bcd0 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60cfdc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 0x7fa608158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60cfdc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 0x7fa608158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58936/0 (socket says 172.21.15.6:58936) 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60cfdc700 1 -- 172.21.15.6:0/895856392 learned_addr learned my addr 172.21.15.6:0/895856392 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60d7dd700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa60806bcd0 0x7fa608158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60dfde700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa60815a490 0x7fa6080c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60cfdc700 1 -- 172.21.15.6:0/895856392 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa60806bcd0 msgr2=0x7fa608158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60cfdc700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa60806bcd0 0x7fa608158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60cfdc700 1 -- 172.21.15.6:0/895856392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa60815a490 msgr2=0x7fa6080c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60cfdc700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa60815a490 0x7fa6080c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60cfdc700 1 -- 172.21.15.6:0/895856392 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8009550 con 0x7fa6080b2150 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60dfde700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa60815a490 0x7fa6080c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.079+0000 7fa60cfdc700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 0x7fa608158ec0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009c40 tx=0x7fa5f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.080+0000 7fa5fdffb700 1 -- 172.21.15.6:0/895856392 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80043c0 con 0x7fa6080b2150 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.080+0000 7fa5fdffb700 1 -- 172.21.15.6:0/895856392 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f801f070 con 0x7fa6080b2150 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.080+0000 7fa60fa41700 1 -- 172.21.15.6:0/895856392 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6080ca000 con 0x7fa6080b2150 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.080+0000 7fa5fdffb700 1 -- 172.21.15.6:0/895856392 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80036a0 con 0x7fa6080b2150 2023-11-28T13:01:03.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.080+0000 7fa60fa41700 1 -- 172.21.15.6:0/895856392 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6080c9d20 con 0x7fa6080b2150 2023-11-28T13:01:03.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.081+0000 7fa60fa41700 1 -- 172.21.15.6:0/895856392 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60806ae90 con 0x7fa6080b2150 2023-11-28T13:01:03.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.081+0000 7fa5fdffb700 1 -- 172.21.15.6:0/895856392 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa5f802a020 con 0x7fa6080b2150 2023-11-28T13:01:03.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.082+0000 7fa5fdffb700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa5ec0682f0 0x7fa5ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.082+0000 7fa5fdffb700 1 -- 172.21.15.6:0/895856392 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa5f80874c0 con 0x7fa6080b2150 2023-11-28T13:01:03.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.082+0000 7fa60d7dd700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa5ec0682f0 0x7fa5ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.082+0000 7fa60d7dd700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa5ec0682f0 0x7fa5ec06a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa604006b20 tx=0x7fa604008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.086+0000 7fa5fdffb700 1 -- 172.21.15.6:0/895856392 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa5f8053a30 con 0x7fa6080b2150 2023-11-28T13:01:03.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.280+0000 7fa60fa41700 1 -- 172.21.15.6:0/895856392 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6080739f0 con 0x7fa6080b2150 2023-11-28T13:01:03.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.280+0000 7fa5fdffb700 1 -- 172.21.15.6:0/895856392 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5f8015020 con 0x7fa6080b2150 2023-11-28T13:01:03.282 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:03.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 -- 172.21.15.6:0/895856392 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa5ec0682f0 msgr2=0x7fa5ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa5ec0682f0 0x7fa5ec06a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa604006b20 tx=0x7fa604008040 comp rx=0 tx=0).stop 2023-11-28T13:01:03.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 -- 172.21.15.6:0/895856392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 msgr2=0x7fa608158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 0x7fa608158ec0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009c40 tx=0x7fa5f8004750 comp rx=0 tx=0).stop 2023-11-28T13:01:03.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 -- 172.21.15.6:0/895856392 shutdown_connections 2023-11-28T13:01:03.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa60815a490 0x7fa6080c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa5ec0682f0 0x7fa5ec06a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa60806bcd0 0x7fa608158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 --2- 172.21.15.6:0/895856392 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa6080b2150 0x7fa608158ec0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 -- 172.21.15.6:0/895856392 >> 172.21.15.6:0/895856392 conn(0x7fa6080b0df0 msgr2=0x7fa6080b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:03.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 -- 172.21.15.6:0/895856392 shutdown_connections 2023-11-28T13:01:03.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.282+0000 7fa60fa41700 1 -- 172.21.15.6:0/895856392 wait complete. 2023-11-28T13:01:03.296 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:03.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.422+0000 7f669f0ae700 1 Processor -- start 2023-11-28T13:01:03.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.422+0000 7f669f0ae700 1 -- start start 2023-11-28T13:01:03.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.423+0000 7f669f0ae700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6698158d90 0x7f66980ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.423+0000 7f669f0ae700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f669806e900 0x7f669806ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.423+0000 7f669f0ae700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669806e470 con 0x7f66981589c0 2023-11-28T13:01:03.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.423+0000 7f669f0ae700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669806de60 con 0x7f669806e900 2023-11-28T13:01:03.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.423+0000 7f669f0ae700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66980ceb40 con 0x7f6698158d90 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.423+0000 7f669d64b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f669806e900 0x7f669806ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.423+0000 7f669d64b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f669806e900 0x7f669806ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58944/0 (socket says 172.21.15.6:58944) 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.423+0000 7f669d64b700 1 -- 172.21.15.6:0/3423300514 learned_addr learned my addr 172.21.15.6:0/3423300514 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.424+0000 7f6697fff700 1 --2- 172.21.15.6:0/3423300514 >> v2:172.21.15.61:3300/0 conn(0x7f6698158d90 0x7f66980ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.424+0000 7f669d64b700 1 -- 172.21.15.6:0/3423300514 >> v2:172.21.15.61:3300/0 conn(0x7f6698158d90 msgr2=0x7f66980ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.424+0000 7f669d64b700 1 --2- 172.21.15.6:0/3423300514 >> v2:172.21.15.61:3300/0 conn(0x7f6698158d90 0x7f66980ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.424+0000 7f669d64b700 1 -- 172.21.15.6:0/3423300514 >> v1:172.21.15.61:6789/0 conn(0x7f66981589c0 legacy=0x7f669806aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.424+0000 7f669d64b700 1 -- 172.21.15.6:0/3423300514 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66980cee20 con 0x7f669806e900 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.424+0000 7f669d64b700 1 --2- 172.21.15.6:0/3423300514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f669806e900 0x7f669806ed60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6698071010 tx=0x7f669000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78a548b5743a2e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.424+0000 7f66977fe700 1 -- 172.21.15.6:0/3423300514 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669000e070 con 0x7f669806e900 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.424+0000 7f66977fe700 1 -- 172.21.15.6:0/3423300514 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669000b9c0 con 0x7f669806e900 2023-11-28T13:01:03.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.424+0000 7f66977fe700 1 -- 172.21.15.6:0/3423300514 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669000bd70 con 0x7f669806e900 2023-11-28T13:01:03.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.425+0000 7f669f0ae700 1 -- 172.21.15.6:0/3423300514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f669806e900 msgr2=0x7f669806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.425+0000 7f669f0ae700 1 --2- 172.21.15.6:0/3423300514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f669806e900 0x7f669806ed60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6698071010 tx=0x7f669000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:03.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.425+0000 7f669f0ae700 1 -- 172.21.15.6:0/3423300514 shutdown_connections 2023-11-28T13:01:03.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.425+0000 7f669f0ae700 1 --2- 172.21.15.6:0/3423300514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f669806e900 0x7f669806ed60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.425+0000 7f669f0ae700 1 --2- 172.21.15.6:0/3423300514 >> v2:172.21.15.61:3300/0 conn(0x7f6698158d90 0x7f66980ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.425+0000 7f669f0ae700 1 -- 172.21.15.6:0/3423300514 >> 172.21.15.6:0/3423300514 conn(0x7f66980b0df0 msgr2=0x7f66980b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:03.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.425+0000 7f669f0ae700 1 -- 172.21.15.6:0/3423300514 shutdown_connections 2023-11-28T13:01:03.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.425+0000 7f669f0ae700 1 -- 172.21.15.6:0/3423300514 wait complete. 2023-11-28T13:01:03.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.426+0000 7f669f0ae700 1 Processor -- start 2023-11-28T13:01:03.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.426+0000 7f669f0ae700 1 -- start start 2023-11-28T13:01:03.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669f0ae700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f669806e900 0x7f6698156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669f0ae700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66981589c0 0x7f66980c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669f0ae700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6698158d90 0x7f66980c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669f0ae700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66980c6c20 con 0x7f6698158d90 2023-11-28T13:01:03.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669f0ae700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66980c6f00 con 0x7f669806e900 2023-11-28T13:01:03.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669f0ae700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66980c71e0 con 0x7f66981589c0 2023-11-28T13:01:03.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669d64b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6698158d90 0x7f66980c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669d64b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6698158d90 0x7f66980c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58950/0 (socket says 172.21.15.6:58950) 2023-11-28T13:01:03.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669d64b700 1 -- 172.21.15.6:0/2829870040 learned_addr learned my addr 172.21.15.6:0/2829870040 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:03.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669ce4a700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f669806e900 0x7f6698156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f6697fff700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66981589c0 0x7f66980c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669d64b700 1 -- 172.21.15.6:0/2829870040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66981589c0 msgr2=0x7f66980c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669d64b700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66981589c0 0x7f66980c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669d64b700 1 -- 172.21.15.6:0/2829870040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f669806e900 msgr2=0x7f6698156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669d64b700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f669806e900 0x7f6698156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669d64b700 1 -- 172.21.15.6:0/2829870040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6690009b30 con 0x7f6698158d90 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.427+0000 7f669d64b700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6698158d90 0x7f66980c8b60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6698072280 tx=0x7f6690004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.428+0000 7f66957fa700 1 -- 172.21.15.6:0/2829870040 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690004880 con 0x7f6698158d90 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.428+0000 7f66957fa700 1 -- 172.21.15.6:0/2829870040 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66900092e0 con 0x7f6698158d90 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.428+0000 7f669f0ae700 1 -- 172.21.15.6:0/2829870040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66980c9d90 con 0x7f6698158d90 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.428+0000 7f66957fa700 1 -- 172.21.15.6:0/2829870040 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669001c660 con 0x7f6698158d90 2023-11-28T13:01:03.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.428+0000 7f669f0ae700 1 -- 172.21.15.6:0/2829870040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669806e100 con 0x7f6698158d90 2023-11-28T13:01:03.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.429+0000 7f669f0ae700 1 -- 172.21.15.6:0/2829870040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669806f2a0 con 0x7f6698158d90 2023-11-28T13:01:03.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.429+0000 7f66957fa700 1 -- 172.21.15.6:0/2829870040 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f669002c070 con 0x7f6698158d90 2023-11-28T13:01:03.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.429+0000 7f66957fa700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f667c0682f0 0x7f667c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.430+0000 7f66957fa700 1 -- 172.21.15.6:0/2829870040 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6690087a00 con 0x7f6698158d90 2023-11-28T13:01:03.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.430+0000 7f669ce4a700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f667c0682f0 0x7f667c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.430+0000 7f669ce4a700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f667c0682f0 0x7f667c06a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f668c007fd0 tx=0x7f668c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.434+0000 7f66957fa700 1 -- 172.21.15.6:0/2829870040 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6690053f70 con 0x7f6698158d90 2023-11-28T13:01:03.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.625+0000 7f669f0ae700 1 -- 172.21.15.6:0/2829870040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66980739f0 con 0x7f6698158d90 2023-11-28T13:01:03.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.626+0000 7f66957fa700 1 -- 172.21.15.6:0/2829870040 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6690057590 con 0x7f6698158d90 2023-11-28T13:01:03.627 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:03.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.627+0000 7f669f0ae700 1 -- 172.21.15.6:0/2829870040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f667c0682f0 msgr2=0x7f667c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.627+0000 7f669f0ae700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f667c0682f0 0x7f667c06a7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f668c007fd0 tx=0x7f668c009040 comp rx=0 tx=0).stop 2023-11-28T13:01:03.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.627+0000 7f669f0ae700 1 -- 172.21.15.6:0/2829870040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6698158d90 msgr2=0x7f66980c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.627+0000 7f669f0ae700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6698158d90 0x7f66980c8b60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6698072280 tx=0x7f6690004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.628+0000 7f669f0ae700 1 -- 172.21.15.6:0/2829870040 shutdown_connections 2023-11-28T13:01:03.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.628+0000 7f669f0ae700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6698158d90 0x7f66980c8b60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.628+0000 7f669f0ae700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f669806e900 0x7f6698156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.628+0000 7f669f0ae700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f667c0682f0 0x7f667c06a7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.628+0000 7f669f0ae700 1 --2- 172.21.15.6:0/2829870040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f66981589c0 0x7f66980c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.628+0000 7f669f0ae700 1 -- 172.21.15.6:0/2829870040 >> 172.21.15.6:0/2829870040 conn(0x7f66980b0df0 msgr2=0x7f66980b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:03.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.628+0000 7f669f0ae700 1 -- 172.21.15.6:0/2829870040 shutdown_connections 2023-11-28T13:01:03.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.628+0000 7f669f0ae700 1 -- 172.21.15.6:0/2829870040 wait complete. 2023-11-28T13:01:03.644 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:03.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.770+0000 7ff861bf1700 1 Processor -- start 2023-11-28T13:01:03.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.770+0000 7ff861bf1700 1 -- start start 2023-11-28T13:01:03.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff861bf1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff85c159500 0x7ff85c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff861bf1700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff85c15dbf0 0x7ff85c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff861bf1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85c15e040 con 0x7ff85c0b1630 2023-11-28T13:01:03.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff861bf1700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85c0cede0 con 0x7ff85c15dbf0 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff861bf1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85c0ceff0 con 0x7ff85c159500 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff85affd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff85c15dbf0 0x7ff85c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff85affd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff85c15dbf0 0x7ff85c0ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58956/0 (socket says 172.21.15.6:58956) 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff85affd700 1 -- 172.21.15.6:0/2921535370 learned_addr learned my addr 172.21.15.6:0/2921535370 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff85b7fe700 1 --2- 172.21.15.6:0/2921535370 >> v2:172.21.15.61:3300/0 conn(0x7ff85c159500 0x7ff85c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff85affd700 1 -- 172.21.15.6:0/2921535370 >> v2:172.21.15.61:3300/0 conn(0x7ff85c159500 msgr2=0x7ff85c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff85affd700 1 --2- 172.21.15.6:0/2921535370 >> v2:172.21.15.61:3300/0 conn(0x7ff85c159500 0x7ff85c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff85affd700 1 -- 172.21.15.6:0/2921535370 >> v1:172.21.15.61:6789/0 conn(0x7ff85c0b1630 legacy=0x7ff85c06bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.771+0000 7ff85affd700 1 -- 172.21.15.6:0/2921535370 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c0cf2d0 con 0x7ff85c15dbf0 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff85affd700 1 --2- 172.21.15.6:0/2921535370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c15dbf0 0x7ff85c0ce8a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0d0bc0 tx=0x7ff84400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2658498ac327cc2a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff85a7fc700 1 -- 172.21.15.6:0/2921535370 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84400dd30 con 0x7ff85c15dbf0 2023-11-28T13:01:03.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff85a7fc700 1 -- 172.21.15.6:0/2921535370 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84400de90 con 0x7ff85c15dbf0 2023-11-28T13:01:03.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff85a7fc700 1 -- 172.21.15.6:0/2921535370 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844010690 con 0x7ff85c15dbf0 2023-11-28T13:01:03.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff861bf1700 1 -- 172.21.15.6:0/2921535370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c15dbf0 msgr2=0x7ff85c0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff861bf1700 1 --2- 172.21.15.6:0/2921535370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c15dbf0 0x7ff85c0ce8a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0d0bc0 tx=0x7ff84400be70 comp rx=0 tx=0).stop 2023-11-28T13:01:03.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff861bf1700 1 -- 172.21.15.6:0/2921535370 shutdown_connections 2023-11-28T13:01:03.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff861bf1700 1 --2- 172.21.15.6:0/2921535370 >> v2:172.21.15.61:3300/0 conn(0x7ff85c159500 0x7ff85c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff861bf1700 1 --2- 172.21.15.6:0/2921535370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c15dbf0 0x7ff85c0ce8a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff861bf1700 1 -- 172.21.15.6:0/2921535370 >> 172.21.15.6:0/2921535370 conn(0x7ff85c0cc630 msgr2=0x7ff85c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:03.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff861bf1700 1 -- 172.21.15.6:0/2921535370 shutdown_connections 2023-11-28T13:01:03.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.772+0000 7ff861bf1700 1 -- 172.21.15.6:0/2921535370 wait complete. 2023-11-28T13:01:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.774+0000 7ff861bf1700 1 Processor -- start 2023-11-28T13:01:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.774+0000 7ff861bf1700 1 -- start start 2023-11-28T13:01:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff861bf1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c0b1630 0x7ff85c1ee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff861bf1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff85c159500 0x7ff85c1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff861bf1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff85c15dbf0 0x7ff85c15b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff861bf1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c1f19c0 con 0x7ff85c0b1630 2023-11-28T13:01:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff861bf1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c1f1ca0 con 0x7ff85c159500 2023-11-28T13:01:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff861bf1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c1f1f80 con 0x7ff85c15dbf0 2023-11-28T13:01:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff85b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c0b1630 0x7ff85c1ee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff85b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c0b1630 0x7ff85c1ee9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58960/0 (socket says 172.21.15.6:58960) 2023-11-28T13:01:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff85b7fe700 1 -- 172.21.15.6:0/995037530 learned_addr learned my addr 172.21.15.6:0/995037530 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:03.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff85affd700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff85c159500 0x7ff85c1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.775+0000 7ff85bfff700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff85c15dbf0 0x7ff85c15b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff85b7fe700 1 -- 172.21.15.6:0/995037530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff85c15dbf0 msgr2=0x7ff85c15b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff85b7fe700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff85c15dbf0 0x7ff85c15b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff85b7fe700 1 -- 172.21.15.6:0/995037530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff85c159500 msgr2=0x7ff85c1560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff85b7fe700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff85c159500 0x7ff85c1560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff85b7fe700 1 -- 172.21.15.6:0/995037530 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84400b7f0 con 0x7ff85c0b1630 2023-11-28T13:01:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff85b7fe700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c0b1630 0x7ff85c1ee9e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00e810 tx=0x7ff84c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff853fff700 1 -- 172.21.15.6:0/995037530 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c013610 con 0x7ff85c0b1630 2023-11-28T13:01:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff853fff700 1 -- 172.21.15.6:0/995037530 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c013770 con 0x7ff85c0b1630 2023-11-28T13:01:03.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff861bf1700 1 -- 172.21.15.6:0/995037530 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85c15d3f0 con 0x7ff85c0b1630 2023-11-28T13:01:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff853fff700 1 -- 172.21.15.6:0/995037530 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c013b20 con 0x7ff85c0b1630 2023-11-28T13:01:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.776+0000 7ff861bf1700 1 -- 172.21.15.6:0/995037530 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85c15d110 con 0x7ff85c0b1630 2023-11-28T13:01:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.777+0000 7ff861bf1700 1 -- 172.21.15.6:0/995037530 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85c15d6d0 con 0x7ff85c0b1630 2023-11-28T13:01:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.777+0000 7ff853fff700 1 -- 172.21.15.6:0/995037530 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff84c023070 con 0x7ff85c0b1630 2023-11-28T13:01:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.778+0000 7ff853fff700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff83c068320 0x7ff83c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.778+0000 7ff853fff700 1 -- 172.21.15.6:0/995037530 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff84c085b50 con 0x7ff85c0b1630 2023-11-28T13:01:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.778+0000 7ff85affd700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff83c068320 0x7ff83c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:03.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.778+0000 7ff85affd700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff83c068320 0x7ff83c06a7d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0d0bc0 tx=0x7ff844012000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:03.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.782+0000 7ff853fff700 1 -- 172.21.15.6:0/995037530 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff84c052040 con 0x7ff85c0b1630 2023-11-28T13:01:03.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.974+0000 7ff861bf1700 1 -- 172.21.15.6:0/995037530 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff85c1f2430 con 0x7ff85c0b1630 2023-11-28T13:01:03.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.975+0000 7ff853fff700 1 -- 172.21.15.6:0/995037530 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff84c055660 con 0x7ff85c0b1630 2023-11-28T13:01:03.976 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:03.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 -- 172.21.15.6:0/995037530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff83c068320 msgr2=0x7ff83c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff83c068320 0x7ff83c06a7d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0d0bc0 tx=0x7ff844012000 comp rx=0 tx=0).stop 2023-11-28T13:01:03.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 -- 172.21.15.6:0/995037530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c0b1630 msgr2=0x7ff85c1ee9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:03.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c0b1630 0x7ff85c1ee9e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00e810 tx=0x7ff84c00eb20 comp rx=0 tx=0).stop 2023-11-28T13:01:03.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 -- 172.21.15.6:0/995037530 shutdown_connections 2023-11-28T13:01:03.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff83c068320 0x7ff83c06a7d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff85c15dbf0 0x7ff85c15b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff85c0b1630 0x7ff85c1ee9e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 --2- 172.21.15.6:0/995037530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff85c159500 0x7ff85c1560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:03.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 -- 172.21.15.6:0/995037530 >> 172.21.15.6:0/995037530 conn(0x7ff85c0cc630 msgr2=0x7ff85c0af710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:03.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 -- 172.21.15.6:0/995037530 shutdown_connections 2023-11-28T13:01:03.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:03.977+0000 7ff861bf1700 1 -- 172.21.15.6:0/995037530 wait complete. 2023-11-28T13:01:03.990 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:04.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.117+0000 7f895a2f4700 1 Processor -- start 2023-11-28T13:01:04.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.118+0000 7f895a2f4700 1 -- start start 2023-11-28T13:01:04.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.118+0000 7f895a2f4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b2d90 0x7f89540ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.118+0000 7f895a2f4700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540cd2a0 con 0x7f89540cb630 2023-11-28T13:01:04.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.118+0000 7f895a2f4700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540d0400 con 0x7f89540b62d0 2023-11-28T13:01:04.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.118+0000 7f895a2f4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89540d06e0 con 0x7f89540b2d90 2023-11-28T13:01:04.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.118+0000 7f89537fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f89540cb630 0x7f89540b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46288/0 (socket says 172.21.15.6:46288) 2023-11-28T13:01:04.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.118+0000 7f89537fe700 1 -- 172.21.15.6:0/1309180553 learned_addr learned my addr 172.21.15.6:0/1309180553 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:04.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8958891700 1 --2- 172.21.15.6:0/1309180553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b2d90 0x7f89540ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8952ffd700 1 -- 172.21.15.6:0/1309180553 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456618414 0 0) 0x7f89540cd2a0 con 0x7f89540cb630 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8952ffd700 1 -- 172.21.15.6:0/1309180553 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f893c0035b0 con 0x7f89540cb630 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8952ffd700 1 -- 172.21.15.6:0/1309180553 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327776604 0 0) 0x7f89540d0400 con 0x7f89540b62d0 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8952ffd700 1 -- 172.21.15.6:0/1309180553 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89540cd2a0 con 0x7f89540b62d0 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8952ffd700 1 -- 172.21.15.6:0/1309180553 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4113525237 0 0) 0x7f893c0035b0 con 0x7f89540cb630 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8958891700 1 -- 172.21.15.6:0/1309180553 >> v1:172.21.15.61:6789/0 conn(0x7f89540b62d0 legacy=0x7f895406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8958891700 1 -- 172.21.15.6:0/1309180553 >> v1:172.21.15.6:6789/0 conn(0x7f89540cb630 legacy=0x7f89540b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8958891700 1 -- 172.21.15.6:0/1309180553 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540cffb0 con 0x7f89540b2d90 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8958891700 1 --2- 172.21.15.6:0/1309180553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b2d90 0x7f89540ce7d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f894c009dc0 tx=0x7f894c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7168b33b5c34626 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8952ffd700 1 -- 172.21.15.6:0/1309180553 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00e070 con 0x7f89540b2d90 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8952ffd700 1 -- 172.21.15.6:0/1309180553 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c003a20 con 0x7f89540b2d90 2023-11-28T13:01:04.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.119+0000 7f8952ffd700 1 -- 172.21.15.6:0/1309180553 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948001670 con 0x7f89540b2d90 2023-11-28T13:01:04.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.120+0000 7f895a2f4700 1 -- 172.21.15.6:0/1309180553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b2d90 msgr2=0x7f89540ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.120+0000 7f895a2f4700 1 --2- 172.21.15.6:0/1309180553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b2d90 0x7f89540ce7d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f894c009dc0 tx=0x7f894c00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:04.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.120+0000 7f895a2f4700 1 -- 172.21.15.6:0/1309180553 shutdown_connections 2023-11-28T13:01:04.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.120+0000 7f895a2f4700 1 --2- 172.21.15.6:0/1309180553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b2d90 0x7f89540ce7d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.120+0000 7f895a2f4700 1 -- 172.21.15.6:0/1309180553 >> 172.21.15.6:0/1309180553 conn(0x7f89540cd680 msgr2=0x7f89540cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:04.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.120+0000 7f895a2f4700 1 -- 172.21.15.6:0/1309180553 shutdown_connections 2023-11-28T13:01:04.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.120+0000 7f895a2f4700 1 -- 172.21.15.6:0/1309180553 wait complete. 2023-11-28T13:01:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.121+0000 7f895a2f4700 1 Processor -- start 2023-11-28T13:01:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.121+0000 7f895a2f4700 1 -- start start 2023-11-28T13:01:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f895a2f4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89540b2d90 0x7f895415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f895a2f4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b62d0 0x7f89541576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f895a2f4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89540cb630 0x7f895415bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f895a2f4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895415c520 con 0x7f89540b2d90 2023-11-28T13:01:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f895a2f4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895415c800 con 0x7f89540cb630 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f895a2f4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895415cae0 con 0x7f89540b62d0 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f8953fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89540b2d90 0x7f895415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f8953fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89540b2d90 0x7f895415d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58974/0 (socket says 172.21.15.6:58974) 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f8953fff700 1 -- 172.21.15.6:0/10702110 learned_addr learned my addr 172.21.15.6:0/10702110 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f89537fe700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b62d0 0x7f89541576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.122+0000 7f8958891700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89540cb630 0x7f895415bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f8953fff700 1 -- 172.21.15.6:0/10702110 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b62d0 msgr2=0x7f89541576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f8953fff700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b62d0 0x7f89541576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f8953fff700 1 -- 172.21.15.6:0/10702110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89540cb630 msgr2=0x7f895415bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f8953fff700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89540cb630 0x7f895415bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f8953fff700 1 -- 172.21.15.6:0/10702110 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c009a70 con 0x7f89540b2d90 2023-11-28T13:01:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f89537fe700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b62d0 0x7f89541576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f8958891700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89540cb630 0x7f895415bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f8953fff700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89540b2d90 0x7f895415d5c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f894400c520 tx=0x7f894400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f8950ff9700 1 -- 172.21.15.6:0/10702110 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440052d0 con 0x7f89540b2d90 2023-11-28T13:01:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f8950ff9700 1 -- 172.21.15.6:0/10702110 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8944005430 con 0x7f89540b2d90 2023-11-28T13:01:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f895a2f4700 1 -- 172.21.15.6:0/10702110 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89540cce50 con 0x7f89540b2d90 2023-11-28T13:01:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f8950ff9700 1 -- 172.21.15.6:0/10702110 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440038c0 con 0x7f89540b2d90 2023-11-28T13:01:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.123+0000 7f895a2f4700 1 -- 172.21.15.6:0/10702110 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89540b66a0 con 0x7f89540b2d90 2023-11-28T13:01:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.124+0000 7f895a2f4700 1 -- 172.21.15.6:0/10702110 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89540ccfe0 con 0x7f89540b2d90 2023-11-28T13:01:04.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.125+0000 7f8950ff9700 1 -- 172.21.15.6:0/10702110 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8944019030 con 0x7f89540b2d90 2023-11-28T13:01:04.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.125+0000 7f8950ff9700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f89400682f0 0x7f894006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.125+0000 7f8950ff9700 1 -- 172.21.15.6:0/10702110 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8944085750 con 0x7f89540b2d90 2023-11-28T13:01:04.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.125+0000 7f89537fe700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f89400682f0 0x7f894006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.126+0000 7f89537fe700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f89400682f0 0x7f894006a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89480017a0 tx=0x7f8948001550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:04.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.129+0000 7f8950ff9700 1 -- 172.21.15.6:0/10702110 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8944051d60 con 0x7f89540b2d90 2023-11-28T13:01:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.323+0000 7f895a2f4700 1 -- 172.21.15.6:0/10702110 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f89540739f0 con 0x7f89540b2d90 2023-11-28T13:01:04.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.324+0000 7f8950ff9700 1 -- 172.21.15.6:0/10702110 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8944055380 con 0x7f89540b2d90 2023-11-28T13:01:04.326 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:04.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 -- 172.21.15.6:0/10702110 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f89400682f0 msgr2=0x7f894006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f89400682f0 0x7f894006a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89480017a0 tx=0x7f8948001550 comp rx=0 tx=0).stop 2023-11-28T13:01:04.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 -- 172.21.15.6:0/10702110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89540b2d90 msgr2=0x7f895415d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89540b2d90 0x7f895415d5c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f894400c520 tx=0x7f894400c8e0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 -- 172.21.15.6:0/10702110 shutdown_connections 2023-11-28T13:01:04.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f89540cb630 0x7f895415bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89540b2d90 0x7f895415d5c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f89400682f0 0x7f894006a7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 --2- 172.21.15.6:0/10702110 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f89540b62d0 0x7f89541576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 -- 172.21.15.6:0/10702110 >> 172.21.15.6:0/10702110 conn(0x7f89540cd680 msgr2=0x7f89540b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:04.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 -- 172.21.15.6:0/10702110 shutdown_connections 2023-11-28T13:01:04.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.326+0000 7f895a2f4700 1 -- 172.21.15.6:0/10702110 wait complete. 2023-11-28T13:01:04.347 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:01:04.347 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T13:01:04.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.471+0000 7fb738f54700 1 Processor -- start 2023-11-28T13:01:04.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.471+0000 7fb738f54700 1 -- start start 2023-11-28T13:01:04.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb738f54700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb734161090 0x7fb7340ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb738f54700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7340caeb0 0x7fb7340cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb738f54700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7340ce340 con 0x7fb73415c990 2023-11-28T13:01:04.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb738f54700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7340cebe0 con 0x7fb734161090 2023-11-28T13:01:04.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb738f54700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340cf780 con 0x7fb7340caeb0 2023-11-28T13:01:04.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb73259c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb73415c990 0x7fb7340cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46302/0 (socket says 172.21.15.6:46302) 2023-11-28T13:01:04.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb73259c700 1 -- 172.21.15.6:0/2582928999 learned_addr learned my addr 172.21.15.6:0/2582928999 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:04.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb731d9b700 1 --2- 172.21.15.6:0/2582928999 >> v2:172.21.15.61:3300/0 conn(0x7fb734161090 0x7fb7340ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb732d9d700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7340caeb0 0x7fb7340cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042748846 0 0) 0x7fb7340ce340 con 0x7fb73415c990 2023-11-28T13:01:04.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb71c0035b0 con 0x7fb73415c990 2023-11-28T13:01:04.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.472+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255335523 0 0) 0x7fb71c0035b0 con 0x7fb73415c990 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7340caeb0 msgr2=0x7fb7340cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7340caeb0 0x7fb7340cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 >> v2:172.21.15.61:3300/0 conn(0x7fb734161090 msgr2=0x7fb7340ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 --2- 172.21.15.6:0/2582928999 >> v2:172.21.15.61:3300/0 conn(0x7fb734161090 0x7fb7340ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340cb3f0 con 0x7fb73415c990 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fb728001670 con 0x7fb73415c990 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 >> v1:172.21.15.6:6789/0 conn(0x7fb73415c990 legacy=0x7fb7340cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb71c003a00 0x7fb71c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7340caeb0 0x7fb71c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 0x7fb71c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340cebe0 con 0x7fb71c004d90 2023-11-28T13:01:04.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340cf780 con 0x7fb71c003a00 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c00bac0 con 0x7fb7340caeb0 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb732d9d700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 0x7fb71c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73259c700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb71c003a00 0x7fb71c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb731d9b700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7340caeb0 0x7fb71c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb732d9d700 1 -- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7340caeb0 msgr2=0x7fb71c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb732d9d700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7340caeb0 0x7fb71c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb732d9d700 1 -- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb71c003a00 msgr2=0x7fb71c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb732d9d700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb71c003a00 0x7fb71c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb732d9d700 1 -- 172.21.15.6:0/2582928999 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c0035b0 con 0x7fb71c004d90 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb732d9d700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 0x7fb71c005200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009fc0 tx=0x7fb71c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd3d3331850d6e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00ec30 con 0x7fb71c004d90 2023-11-28T13:01:04.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c00f040 con 0x7fb71c004d90 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.473+0000 7fb73159a700 1 -- 172.21.15.6:0/2582928999 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c014480 con 0x7fb71c004d90 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.474+0000 7fb738f54700 1 -- 172.21.15.6:0/2582928999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 msgr2=0x7fb71c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.474+0000 7fb738f54700 1 --2- 172.21.15.6:0/2582928999 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 0x7fb71c005200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009fc0 tx=0x7fb71c005880 comp rx=0 tx=0).stop 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.474+0000 7fb732d9d700 1 -- 172.21.15.6:0/2582928999 reap_dead start 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.474+0000 7fb738f54700 1 -- 172.21.15.6:0/2582928999 shutdown_connections 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.474+0000 7fb738f54700 1 -- 172.21.15.6:0/2582928999 >> 172.21.15.6:0/2582928999 conn(0x7fb7340b1650 msgr2=0x7fb7340b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.474+0000 7fb738f54700 1 -- 172.21.15.6:0/2582928999 shutdown_connections 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.474+0000 7fb738f54700 1 -- 172.21.15.6:0/2582928999 wait complete. 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.475+0000 7fb738f54700 1 Processor -- start 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.475+0000 7fb738f54700 1 -- start start 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb738f54700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 0x7fb7341567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb738f54700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb734147800 0x7fb7340b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb738f54700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7340c58a0 0x7fb73406c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb738f54700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb734147c70 con 0x7fb71c004d90 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb738f54700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340b5670 con 0x7fb7340c58a0 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb738f54700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7341456c0 con 0x7fb734147800 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb73259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 0x7fb7341567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb73259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 0x7fb7341567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:58994/0 (socket says 172.21.15.6:58994) 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb73259c700 1 -- 172.21.15.6:0/873176519 learned_addr learned my addr 172.21.15.6:0/873176519 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb732d9d700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7340c58a0 0x7fb73406c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb731d9b700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb734147800 0x7fb7340b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.476+0000 7fb73259c700 1 -- 172.21.15.6:0/873176519 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb734147800 msgr2=0x7fb7340b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb73259c700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb734147800 0x7fb7340b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb73259c700 1 -- 172.21.15.6:0/873176519 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7340c58a0 msgr2=0x7fb73406c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb73259c700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7340c58a0 0x7fb73406c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:04.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb73259c700 1 -- 172.21.15.6:0/873176519 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340ce870 con 0x7fb71c004d90 2023-11-28T13:01:04.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb731d9b700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb734147800 0x7fb7340b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:04.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb732d9d700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7340c58a0 0x7fb73406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:04.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb73259c700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 0x7fb7341567f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb728000fb0 tx=0x7fb7280076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:04.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb722ffd700 1 -- 172.21.15.6:0/873176519 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7280077c0 con 0x7fb71c004d90 2023-11-28T13:01:04.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb722ffd700 1 -- 172.21.15.6:0/873176519 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72800f8a0 con 0x7fb71c004d90 2023-11-28T13:01:04.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb738f54700 1 -- 172.21.15.6:0/873176519 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7340ce590 con 0x7fb71c004d90 2023-11-28T13:01:04.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb722ffd700 1 -- 172.21.15.6:0/873176519 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72800fc50 con 0x7fb71c004d90 2023-11-28T13:01:04.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.477+0000 7fb738f54700 1 -- 172.21.15.6:0/873176519 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7340cc310 con 0x7fb71c004d90 2023-11-28T13:01:04.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.478+0000 7fb738f54700 1 -- 172.21.15.6:0/873176519 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7340675a0 con 0x7fb71c004d90 2023-11-28T13:01:04.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.478+0000 7fb722ffd700 1 -- 172.21.15.6:0/873176519 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb72800fdb0 con 0x7fb71c004d90 2023-11-28T13:01:04.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.478+0000 7fb722ffd700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7140682f0 0x7fb71406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:04.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.479+0000 7fb722ffd700 1 -- 172.21.15.6:0/873176519 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb7280860c0 con 0x7fb71c004d90 2023-11-28T13:01:04.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.479+0000 7fb731d9b700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7140682f0 0x7fb71406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:04.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.479+0000 7fb731d9b700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7140682f0 0x7fb71406a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7340c6340 tx=0x7fb724008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:04.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.483+0000 7fb722ffd700 1 -- 172.21.15.6:0/873176519 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb728052630 con 0x7fb71c004d90 2023-11-28T13:01:04.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:04.760+0000 7fb738f54700 1 -- 172.21.15.6:0/873176519 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fb7340cdc50 con 0x7fb71c004d90 2023-11-28T13:01:05.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.038+0000 7fb722ffd700 1 -- 172.21.15.6:0/873176519 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v58) v1 ==== 145+0+0 (secure 0 0 0) 0x7fb728055c50 con 0x7fb71c004d90 2023-11-28T13:01:05.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 -- 172.21.15.6:0/873176519 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7140682f0 msgr2=0x7fb71406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7140682f0 0x7fb71406a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7340c6340 tx=0x7fb724008040 comp rx=0 tx=0).stop 2023-11-28T13:01:05.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 -- 172.21.15.6:0/873176519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 msgr2=0x7fb7341567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 0x7fb7341567f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb728000fb0 tx=0x7fb7280076a0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 -- 172.21.15.6:0/873176519 shutdown_connections 2023-11-28T13:01:05.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7340c58a0 0x7fb73406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb71c004d90 0x7fb7341567f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7140682f0 0x7fb71406a7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 --2- 172.21.15.6:0/873176519 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb734147800 0x7fb7340b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 -- 172.21.15.6:0/873176519 >> 172.21.15.6:0/873176519 conn(0x7fb7340b1650 msgr2=0x7fb73406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:05.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 -- 172.21.15.6:0/873176519 shutdown_connections 2023-11-28T13:01:05.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.041+0000 7fb738f54700 1 -- 172.21.15.6:0/873176519 wait complete. 2023-11-28T13:01:05.055 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T13:01:05.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.184+0000 7f58f2af1700 1 Processor -- start 2023-11-28T13:01:05.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.184+0000 7f58f2af1700 1 -- start start 2023-11-28T13:01:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.185+0000 7f58f2af1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58ec0b1630 0x7f58ec0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.185+0000 7f58f2af1700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ec0ce550 con 0x7f58ec0b1a00 2023-11-28T13:01:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.185+0000 7f58f2af1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ec0ce830 con 0x7f58ec06afc0 2023-11-28T13:01:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.185+0000 7f58f2af1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0ceb10 con 0x7f58ec0b1630 2023-11-28T13:01:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.185+0000 7f58ebfff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f58ec0b1a00 0x7f58ec0caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46324/0 (socket says 172.21.15.6:46324) 2023-11-28T13:01:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.185+0000 7f58ebfff700 1 -- 172.21.15.6:0/528091561 learned_addr learned my addr 172.21.15.6:0/528091561 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.185+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910290376 0 0) 0x7f58ec0ce550 con 0x7f58ec0b1a00 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.185+0000 7f58f088d700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58ec0b1630 0x7f58ec0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.185+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58dc0035b0 con 0x7f58ec0b1a00 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 954399121 0 0) 0x7f58dc0035b0 con 0x7f58ec0b1a00 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58ec0b1630 msgr2=0x7f58ec0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58ec0b1630 0x7f58ec0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 >> v1:172.21.15.61:6789/0 conn(0x7f58ec06afc0 legacy=0x7f58ec06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0cedf0 con 0x7f58ec0b1a00 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f58d8001670 con 0x7f58ec0b1a00 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 >> v1:172.21.15.6:6789/0 conn(0x7f58ec0b1a00 legacy=0x7f58ec0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58dc003a30 0x7f58dc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58dc004850 0x7f58dc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58ec0b1630 0x7f58dc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0ceb10 con 0x7f58dc003a30 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc00bac0 con 0x7f58dc004850 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc00bc50 con 0x7f58ec0b1630 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58f088d700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58dc003a30 0x7f58dc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58f108e700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58ec0b1630 0x7f58dc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58ebfff700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58dc004850 0x7f58dc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58f088d700 1 -- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58ec0b1630 msgr2=0x7f58dc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58f088d700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58ec0b1630 0x7f58dc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58f088d700 1 -- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58dc004850 msgr2=0x7f58dc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58f088d700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58dc004850 0x7f58dc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58f088d700 1 -- 172.21.15.6:0/528091561 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc0035b0 con 0x7f58dc003a30 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58f108e700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58ec0b1630 0x7f58dc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58f088d700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58dc003a30 0x7f58dc003e40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009b60 tx=0x7f58dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f507c385cea8ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.186+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e000e6c0 con 0x7f58dc003a30 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.187+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e000e820 con 0x7f58dc003a30 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.187+0000 7f58eb7fe700 1 -- 172.21.15.6:0/528091561 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e000eb90 con 0x7f58dc003a30 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.187+0000 7f58f2af1700 1 -- 172.21.15.6:0/528091561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58dc003a30 msgr2=0x7f58dc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.187+0000 7f58f2af1700 1 --2- 172.21.15.6:0/528091561 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58dc003a30 0x7f58dc003e40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009b60 tx=0x7f58dc005880 comp rx=0 tx=0).stop 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.187+0000 7f58f108e700 1 -- 172.21.15.6:0/528091561 reap_dead start 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.187+0000 7f58f2af1700 1 -- 172.21.15.6:0/528091561 shutdown_connections 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.187+0000 7f58f2af1700 1 -- 172.21.15.6:0/528091561 >> 172.21.15.6:0/528091561 conn(0x7f58ec0cc630 msgr2=0x7f58ec0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.187+0000 7f58f2af1700 1 -- 172.21.15.6:0/528091561 shutdown_connections 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.187+0000 7f58f2af1700 1 -- 172.21.15.6:0/528091561 wait complete. 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.189+0000 7f58f2af1700 1 Processor -- start 2023-11-28T13:01:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.189+0000 7f58f2af1700 1 -- start start 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58f2af1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58dc003a30 0x7f58ec15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58f2af1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58ec0c6e20 0x7f58ec15efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58f2af1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58ec0c5770 0x7f58ec0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58f2af1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0c6120 con 0x7f58ec0c6e20 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58f2af1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0b2b80 con 0x7f58ec0c5770 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58f2af1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0b2e60 con 0x7f58dc003a30 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58ebfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58ec0c6e20 0x7f58ec15efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58ebfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58ec0c6e20 0x7f58ec15efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59016/0 (socket says 172.21.15.6:59016) 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58ebfff700 1 -- 172.21.15.6:0/1211720038 learned_addr learned my addr 172.21.15.6:0/1211720038 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58f108e700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58ec0c5770 0x7f58ec0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58f088d700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58dc003a30 0x7f58ec15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58ebfff700 1 -- 172.21.15.6:0/1211720038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58dc003a30 msgr2=0x7f58ec15b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58ebfff700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58dc003a30 0x7f58ec15b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58ebfff700 1 -- 172.21.15.6:0/1211720038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58ec0c5770 msgr2=0x7f58ec0c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58ebfff700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58ec0c5770 0x7f58ec0c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.190+0000 7f58ebfff700 1 -- 172.21.15.6:0/1211720038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0b1e70 con 0x7f58ec0c6e20 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.191+0000 7f58ebfff700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58ec0c6e20 0x7f58ec15efe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f58d800eaf0 tx=0x7f58d800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.191+0000 7f58e97fa700 1 -- 172.21.15.6:0/1211720038 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d800c5d0 con 0x7f58ec0c6e20 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.191+0000 7f58e97fa700 1 -- 172.21.15.6:0/1211720038 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d800c730 con 0x7f58ec0c6e20 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.191+0000 7f58f2af1700 1 -- 172.21.15.6:0/1211720038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec0cf0d0 con 0x7f58ec0c6e20 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.191+0000 7f58e97fa700 1 -- 172.21.15.6:0/1211720038 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d800cae0 con 0x7f58ec0c6e20 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.191+0000 7f58f2af1700 1 -- 172.21.15.6:0/1211720038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec0cb5b0 con 0x7f58ec0c6e20 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.192+0000 7f58f2af1700 1 -- 172.21.15.6:0/1211720038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec0739f0 con 0x7f58ec0c6e20 2023-11-28T13:01:05.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.192+0000 7f58e97fa700 1 -- 172.21.15.6:0/1211720038 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f58d800cc40 con 0x7f58ec0c6e20 2023-11-28T13:01:05.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.192+0000 7f58e97fa700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58d00682f0 0x7f58d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.193+0000 7f58e97fa700 1 -- 172.21.15.6:0/1211720038 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f58d8085d70 con 0x7f58ec0c6e20 2023-11-28T13:01:05.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.193+0000 7f58f088d700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58d00682f0 0x7f58d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.193+0000 7f58f088d700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58d00682f0 0x7f58d006a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f58e00090b0 tx=0x7f58e0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:05.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.197+0000 7f58e97fa700 1 -- 172.21.15.6:0/1211720038 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f58d80522e0 con 0x7f58ec0c6e20 2023-11-28T13:01:05.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.391+0000 7f58f2af1700 1 -- 172.21.15.6:0/1211720038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f58ec0cac80 con 0x7f58ec0c6e20 2023-11-28T13:01:05.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.397+0000 7f58e97fa700 1 -- 172.21.15.6:0/1211720038 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f58d8055900 con 0x7f58ec0c6e20 2023-11-28T13:01:05.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 -- 172.21.15.6:0/1211720038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58d00682f0 msgr2=0x7f58d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58d00682f0 0x7f58d006a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f58e00090b0 tx=0x7f58e0008030 comp rx=0 tx=0).stop 2023-11-28T13:01:05.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 -- 172.21.15.6:0/1211720038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58ec0c6e20 msgr2=0x7f58ec15efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58ec0c6e20 0x7f58ec15efe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f58d800eaf0 tx=0x7f58d800ee00 comp rx=0 tx=0).stop 2023-11-28T13:01:05.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 -- 172.21.15.6:0/1211720038 shutdown_connections 2023-11-28T13:01:05.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58ec0c5770 0x7f58ec0c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58d00682f0 0x7f58d006a7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58dc003a30 0x7f58ec15b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 --2- 172.21.15.6:0/1211720038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58ec0c6e20 0x7f58ec15efe0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 -- 172.21.15.6:0/1211720038 >> 172.21.15.6:0/1211720038 conn(0x7f58ec0cc630 msgr2=0x7f58ec0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:05.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 -- 172.21.15.6:0/1211720038 shutdown_connections 2023-11-28T13:01:05.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.399+0000 7f58f2af1700 1 -- 172.21.15.6:0/1211720038 wait complete. 2023-11-28T13:01:05.412 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:01:05.413 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:01:05.453 DEBUG:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:01:05.512 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:01:05.512 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:01:05.512 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:05.530 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:01:05.530 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:05.587 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:05.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.750+0000 7f27219d5700 1 Processor -- start 2023-11-28T13:01:05.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.750+0000 7f27219d5700 1 -- start start 2023-11-28T13:01:05.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f27219d5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f271c1592b0 0x7f271c0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f27219d5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0caf90 0x7f271c070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f27219d5700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271c15a4a0 con 0x7f271c070960 2023-11-28T13:01:05.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f27219d5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c071320 con 0x7f271c1592b0 2023-11-28T13:01:05.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f27219d5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c15a780 con 0x7f271c0caf90 2023-11-28T13:01:05.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f271a7fc700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f271c070960 0x7f271c0ca880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46344/0 (socket says 172.21.15.6:46344) 2023-11-28T13:01:05.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f271a7fc700 1 -- 172.21.15.6:0/913924001 learned_addr learned my addr 172.21.15.6:0/913924001 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:05.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f271b7fe700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0caf90 0x7f271c070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f271affd700 1 --2- 172.21.15.6:0/913924001 >> v2:172.21.15.61:3300/0 conn(0x7f271c1592b0 0x7f271c0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451230725 0 0) 0x7f271c15a4a0 con 0x7f271c070960 2023-11-28T13:01:05.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27040035b0 con 0x7f271c070960 2023-11-28T13:01:05.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 887319939 0 0) 0x7f27040035b0 con 0x7f271c070960 2023-11-28T13:01:05.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0caf90 msgr2=0x7f271c070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f2719ffb700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0caf90 0x7f271c070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 >> v2:172.21.15.61:3300/0 conn(0x7f271c1592b0 msgr2=0x7f271c0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f2719ffb700 1 --2- 172.21.15.6:0/913924001 >> v2:172.21.15.61:3300/0 conn(0x7f271c1592b0 0x7f271c0cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c0ca080 con 0x7f271c070960 2023-11-28T13:01:05.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.751+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f270c001670 con 0x7f271c070960 2023-11-28T13:01:05.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 >> v1:172.21.15.6:6789/0 conn(0x7f271c070960 legacy=0x7f271c0ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f2719ffb700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2704003a00 0x7f2704003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f2719ffb700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0caf90 0x7f2704004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f2719ffb700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2704004d90 0x7f2704005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c071320 con 0x7f2704004d90 2023-11-28T13:01:05.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c15a780 con 0x7f2704003a00 2023-11-28T13:01:05.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270400baf0 con 0x7f271c0caf90 2023-11-28T13:01:05.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f271b7fe700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2704004d90 0x7f2704005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f271a7fc700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0caf90 0x7f2704004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f271affd700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2704003a00 0x7f2704003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f271b7fe700 1 -- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0caf90 msgr2=0x7f2704004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f271b7fe700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0caf90 0x7f2704004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f271b7fe700 1 -- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2704003a00 msgr2=0x7f2704003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f271b7fe700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2704003a00 0x7f2704003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f271b7fe700 1 -- 172.21.15.6:0/913924001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27040035b0 con 0x7f2704004d90 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f271affd700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2704003a00 0x7f2704003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f271b7fe700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2704004d90 0x7f2704005200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2714009fc0 tx=0x7f2704005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e56630ef9cac4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271400ec30 con 0x7f2704004d90 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271400f040 con 0x7f2704004d90 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.752+0000 7f2719ffb700 1 -- 172.21.15.6:0/913924001 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714014480 con 0x7f2704004d90 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.753+0000 7f27219d5700 1 -- 172.21.15.6:0/913924001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2704004d90 msgr2=0x7f2704005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.753+0000 7f27219d5700 1 --2- 172.21.15.6:0/913924001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2704004d90 0x7f2704005200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2714009fc0 tx=0x7f2704005880 comp rx=0 tx=0).stop 2023-11-28T13:01:05.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.753+0000 7f271b7fe700 1 -- 172.21.15.6:0/913924001 reap_dead start 2023-11-28T13:01:05.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.753+0000 7f27219d5700 1 -- 172.21.15.6:0/913924001 shutdown_connections 2023-11-28T13:01:05.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.753+0000 7f27219d5700 1 -- 172.21.15.6:0/913924001 >> 172.21.15.6:0/913924001 conn(0x7f271c0ce9e0 msgr2=0x7f271c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:05.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.753+0000 7f27219d5700 1 -- 172.21.15.6:0/913924001 shutdown_connections 2023-11-28T13:01:05.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.753+0000 7f27219d5700 1 -- 172.21.15.6:0/913924001 wait complete. 2023-11-28T13:01:05.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.754+0000 7f27219d5700 1 Processor -- start 2023-11-28T13:01:05.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.754+0000 7f27219d5700 1 -- start start 2023-11-28T13:01:05.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f27219d5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f271c1ef5c0 0x7f271c0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f27219d5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c1ef990 0x7f271c0c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f27219d5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0c35a0 0x7f271c0c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f27219d5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0c4720 con 0x7f271c1ef990 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f27219d5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0b4760 con 0x7f271c1ef5c0 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f27219d5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0b1df0 con 0x7f271c0c35a0 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f271a7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c1ef990 0x7f271c0c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f271a7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c1ef990 0x7f271c0c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59036/0 (socket says 172.21.15.6:59036) 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f271a7fc700 1 -- 172.21.15.6:0/3085785767 learned_addr learned my addr 172.21.15.6:0/3085785767 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f271affd700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f271c1ef5c0 0x7f271c0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.755+0000 7f271b7fe700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0c35a0 0x7f271c0c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f271a7fc700 1 -- 172.21.15.6:0/3085785767 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0c35a0 msgr2=0x7f271c0c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f271a7fc700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0c35a0 0x7f271c0c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f271a7fc700 1 -- 172.21.15.6:0/3085785767 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f271c1ef5c0 msgr2=0x7f271c0b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f271a7fc700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f271c1ef5c0 0x7f271c0b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f271a7fc700 1 -- 172.21.15.6:0/3085785767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c0c9d90 con 0x7f271c1ef990 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f271b7fe700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0c35a0 0x7f271c0c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f271a7fc700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c1ef990 0x7f271c0c7cc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f270c000c00 tx=0x7f270c0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f2703fff700 1 -- 172.21.15.6:0/3085785767 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c0077d0 con 0x7f271c1ef990 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f2703fff700 1 -- 172.21.15.6:0/3085785767 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270c00f8a0 con 0x7f271c1ef990 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f27219d5700 1 -- 172.21.15.6:0/3085785767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c0ca360 con 0x7f271c1ef990 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f2703fff700 1 -- 172.21.15.6:0/3085785767 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c00fc30 con 0x7f271c1ef990 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.756+0000 7f27219d5700 1 -- 172.21.15.6:0/3085785767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c15d1d0 con 0x7f271c1ef990 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.757+0000 7f27219d5700 1 -- 172.21.15.6:0/3085785767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c0c4860 con 0x7f271c1ef990 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.757+0000 7f2703fff700 1 -- 172.21.15.6:0/3085785767 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f270c00fd90 con 0x7f271c1ef990 2023-11-28T13:01:05.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.758+0000 7f2703fff700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27080682f0 0x7f270806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:05.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.758+0000 7f2703fff700 1 -- 172.21.15.6:0/3085785767 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f270c0860d0 con 0x7f271c1ef990 2023-11-28T13:01:05.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.758+0000 7f271affd700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27080682f0 0x7f270806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:05.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.758+0000 7f271affd700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27080682f0 0x7f270806a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2710009bc0 tx=0x7f2710008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:05.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.762+0000 7f2703fff700 1 -- 172.21.15.6:0/3085785767 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f271c0c4860 con 0x7f271c1ef990 2023-11-28T13:01:05.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.956+0000 7f27219d5700 1 -- 172.21.15.6:0/3085785767 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f271c0c4860 con 0x7f271c1ef990 2023-11-28T13:01:05.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.957+0000 7f2703fff700 1 -- 172.21.15.6:0/3085785767 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f271c0c4860 con 0x7f271c1ef990 2023-11-28T13:01:05.959 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:05.959 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:05.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 -- 172.21.15.6:0/3085785767 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27080682f0 msgr2=0x7f270806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27080682f0 0x7f270806a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2710009bc0 tx=0x7f2710008040 comp rx=0 tx=0).stop 2023-11-28T13:01:05.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 -- 172.21.15.6:0/3085785767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c1ef990 msgr2=0x7f271c0c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:05.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c1ef990 0x7f271c0c7cc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f270c000c00 tx=0x7f270c0075f0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 -- 172.21.15.6:0/3085785767 shutdown_connections 2023-11-28T13:01:05.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27080682f0 0x7f270806a7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f271c0c35a0 0x7f271c0c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f271c1ef5c0 0x7f271c0b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 --2- 172.21.15.6:0/3085785767 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f271c1ef990 0x7f271c0c7cc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:05.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 -- 172.21.15.6:0/3085785767 >> 172.21.15.6:0/3085785767 conn(0x7f271c0ce9e0 msgr2=0x7f271c15aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:05.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 -- 172.21.15.6:0/3085785767 shutdown_connections 2023-11-28T13:01:05.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:05.959+0000 7f27219d5700 1 -- 172.21.15.6:0/3085785767 wait complete. 2023-11-28T13:01:05.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:05.973 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:06.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.100+0000 7fc88f120700 1 Processor -- start 2023-11-28T13:01:06.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.100+0000 7fc88f120700 1 -- start start 2023-11-28T13:01:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.101+0000 7fc88f120700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc888159500 0x7fc8880cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.101+0000 7fc88f120700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88815dbf0 0x7fc8880ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.101+0000 7fc88f120700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88815e040 con 0x7fc8880b1630 2023-11-28T13:01:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.101+0000 7fc88f120700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8880cede0 con 0x7fc888159500 2023-11-28T13:01:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.101+0000 7fc88f120700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8880ceff0 con 0x7fc88815dbf0 2023-11-28T13:01:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.101+0000 7fc88cebc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc888159500 0x7fc8880cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.101+0000 7fc88cebc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc888159500 0x7fc8880cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59042/0 (socket says 172.21.15.6:59042) 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.101+0000 7fc88cebc700 1 -- 172.21.15.6:0/1329314853 learned_addr learned my addr 172.21.15.6:0/1329314853 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.101+0000 7fc887fff700 1 --2- 172.21.15.6:0/1329314853 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88815dbf0 0x7fc8880ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc88cebc700 1 -- 172.21.15.6:0/1329314853 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88815dbf0 msgr2=0x7fc8880ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc88cebc700 1 --2- 172.21.15.6:0/1329314853 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88815dbf0 0x7fc8880ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc88cebc700 1 -- 172.21.15.6:0/1329314853 >> v1:172.21.15.61:6789/0 conn(0x7fc8880b1630 legacy=0x7fc88806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc8877fe700 1 -- 172.21.15.6:0/1329314853 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417681721 0 0) 0x7fc88815e040 con 0x7fc8880b1630 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc88cebc700 1 -- 172.21.15.6:0/1329314853 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8880cf2d0 con 0x7fc888159500 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc88cebc700 1 --2- 172.21.15.6:0/1329314853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 0x7fc8880cd140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00bbc0 tx=0x7fc87c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aba19a274eaf422 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc8877fe700 1 -- 172.21.15.6:0/1329314853 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c00dcf0 con 0x7fc888159500 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc8877fe700 1 -- 172.21.15.6:0/1329314853 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c00de50 con 0x7fc888159500 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc8877fe700 1 -- 172.21.15.6:0/1329314853 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c010690 con 0x7fc888159500 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc88f120700 1 -- 172.21.15.6:0/1329314853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 msgr2=0x7fc8880cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.102+0000 7fc88f120700 1 --2- 172.21.15.6:0/1329314853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 0x7fc8880cd140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00bbc0 tx=0x7fc87c00bed0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.103+0000 7fc88f120700 1 -- 172.21.15.6:0/1329314853 shutdown_connections 2023-11-28T13:01:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.103+0000 7fc88f120700 1 --2- 172.21.15.6:0/1329314853 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 0x7fc8880cd140 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.103+0000 7fc88f120700 1 --2- 172.21.15.6:0/1329314853 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88815dbf0 0x7fc8880ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.103+0000 7fc88f120700 1 -- 172.21.15.6:0/1329314853 >> 172.21.15.6:0/1329314853 conn(0x7fc8880cc630 msgr2=0x7fc8880cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.103+0000 7fc88f120700 1 -- 172.21.15.6:0/1329314853 shutdown_connections 2023-11-28T13:01:06.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.103+0000 7fc88f120700 1 -- 172.21.15.6:0/1329314853 wait complete. 2023-11-28T13:01:06.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.104+0000 7fc88f120700 1 Processor -- start 2023-11-28T13:01:06.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.104+0000 7fc88f120700 1 -- start start 2023-11-28T13:01:06.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc88f120700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8880b1630 0x7fc8880b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc88f120700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 0x7fc8881f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc88f120700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc88815dbf0 0x7fc888154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc88f120700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8880b2f20 con 0x7fc888159500 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc88f120700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8881f3260 con 0x7fc88815dbf0 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc88f120700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8880b3d20 con 0x7fc8880b1630 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc887fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 0x7fc8881f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc887fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 0x7fc8881f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59050/0 (socket says 172.21.15.6:59050) 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc887fff700 1 -- 172.21.15.6:0/2242193927 learned_addr learned my addr 172.21.15.6:0/2242193927 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc88cebc700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8880b1630 0x7fc8880b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc88d6bd700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc88815dbf0 0x7fc888154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc887fff700 1 -- 172.21.15.6:0/2242193927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8880b1630 msgr2=0x7fc8880b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc887fff700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8880b1630 0x7fc8880b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc887fff700 1 -- 172.21.15.6:0/2242193927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc88815dbf0 msgr2=0x7fc888154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc887fff700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc88815dbf0 0x7fc888154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc887fff700 1 -- 172.21.15.6:0/2242193927 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c00b7f0 con 0x7fc888159500 2023-11-28T13:01:06.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.105+0000 7fc887fff700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 0x7fc8881f2d20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc87400e980 tx=0x7fc87400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.106+0000 7fc8857fa700 1 -- 172.21.15.6:0/2242193927 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874009710 con 0x7fc888159500 2023-11-28T13:01:06.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.106+0000 7fc88f120700 1 -- 172.21.15.6:0/2242193927 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88815d3f0 con 0x7fc888159500 2023-11-28T13:01:06.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.106+0000 7fc8857fa700 1 -- 172.21.15.6:0/2242193927 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87400c690 con 0x7fc888159500 2023-11-28T13:01:06.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.106+0000 7fc8857fa700 1 -- 172.21.15.6:0/2242193927 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87400ca20 con 0x7fc888159500 2023-11-28T13:01:06.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.106+0000 7fc88f120700 1 -- 172.21.15.6:0/2242193927 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88815d110 con 0x7fc888159500 2023-11-28T13:01:06.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.106+0000 7fc88f120700 1 -- 172.21.15.6:0/2242193927 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88815d6d0 con 0x7fc888159500 2023-11-28T13:01:06.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.107+0000 7fc8857fa700 1 -- 172.21.15.6:0/2242193927 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc874019030 con 0x7fc888159500 2023-11-28T13:01:06.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.107+0000 7fc8857fa700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 0x7fc86c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.107+0000 7fc8857fa700 1 -- 172.21.15.6:0/2242193927 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc874085e50 con 0x7fc888159500 2023-11-28T13:01:06.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.107+0000 7fc88cebc700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 0x7fc86c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.108+0000 7fc88cebc700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 0x7fc86c06a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc8881efe00 tx=0x7fc87c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.112+0000 7fc8857fa700 1 -- 172.21.15.6:0/2242193927 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc874052340 con 0x7fc888159500 2023-11-28T13:01:06.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.303+0000 7fc88f120700 1 -- 172.21.15.6:0/2242193927 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc8880739f0 con 0x7fc888159500 2023-11-28T13:01:06.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.303+0000 7fc8857fa700 1 -- 172.21.15.6:0/2242193927 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc874055960 con 0x7fc888159500 2023-11-28T13:01:06.305 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:06.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 -- 172.21.15.6:0/2242193927 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 msgr2=0x7fc86c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 0x7fc86c06a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc8881efe00 tx=0x7fc87c01c040 comp rx=0 tx=0).stop 2023-11-28T13:01:06.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 -- 172.21.15.6:0/2242193927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 msgr2=0x7fc8881f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 0x7fc8881f2d20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc87400e980 tx=0x7fc87400ec90 comp rx=0 tx=0).stop 2023-11-28T13:01:06.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 -- 172.21.15.6:0/2242193927 shutdown_connections 2023-11-28T13:01:06.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc88815dbf0 0x7fc888154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 0x7fc86c06a7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc8880b1630 0x7fc8880b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 --2- 172.21.15.6:0/2242193927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc888159500 0x7fc8881f2d20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 -- 172.21.15.6:0/2242193927 >> 172.21.15.6:0/2242193927 conn(0x7fc8880cc630 msgr2=0x7fc8880cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:06.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 -- 172.21.15.6:0/2242193927 shutdown_connections 2023-11-28T13:01:06.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.305+0000 7fc88f120700 1 -- 172.21.15.6:0/2242193927 wait complete. 2023-11-28T13:01:06.319 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:06.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.446+0000 7f5255782700 1 Processor -- start 2023-11-28T13:01:06.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.446+0000 7f5255782700 1 -- start start 2023-11-28T13:01:06.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.447+0000 7f5255782700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5250165670 0x7f5250167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.447+0000 7f5255782700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52500ce550 con 0x7f52500cf5f0 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.447+0000 7f5255782700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52500ce830 con 0x7f52500b1630 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.447+0000 7f5255782700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52500ceb10 con 0x7f5250165670 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.447+0000 7f524e7fc700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f52500cf5f0 0x7f52500caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46376/0 (socket says 172.21.15.6:46376) 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.447+0000 7f524f7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5250165670 0x7f5250167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.447+0000 7f524e7fc700 1 -- 172.21.15.6:0/2594121856 learned_addr learned my addr 172.21.15.6:0/2594121856 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f524f7fe700 1 -- 172.21.15.6:0/2594121856 >> v1:172.21.15.61:6789/0 conn(0x7f52500b1630 legacy=0x7f52500b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f524f7fe700 1 -- 172.21.15.6:0/2594121856 >> v1:172.21.15.6:6789/0 conn(0x7f52500cf5f0 legacy=0x7f52500caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f524f7fe700 1 -- 172.21.15.6:0/2594121856 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500cedf0 con 0x7f5250165670 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f524dffb700 1 -- 172.21.15.6:0/2594121856 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645488947 0 0) 0x7f52500ce550 con 0x7f52500cf5f0 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f524f7fe700 1 --2- 172.21.15.6:0/2594121856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5250165670 0x7f5250167a50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5248009dc0 tx=0x7f524800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca92e3d1b9197cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f524dffb700 1 -- 172.21.15.6:0/2594121856 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800e070 con 0x7f5250165670 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f524dffb700 1 -- 172.21.15.6:0/2594121856 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524800bb50 con 0x7f5250165670 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f524dffb700 1 -- 172.21.15.6:0/2594121856 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5248003a20 con 0x7f5250165670 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f5255782700 1 -- 172.21.15.6:0/2594121856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5250165670 msgr2=0x7f5250167a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f5255782700 1 --2- 172.21.15.6:0/2594121856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5250165670 0x7f5250167a50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5248009dc0 tx=0x7f524800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f5255782700 1 -- 172.21.15.6:0/2594121856 shutdown_connections 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f5255782700 1 --2- 172.21.15.6:0/2594121856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5250165670 0x7f5250167a50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.448+0000 7f5255782700 1 -- 172.21.15.6:0/2594121856 >> 172.21.15.6:0/2594121856 conn(0x7f52500cc630 msgr2=0x7f52500cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.449+0000 7f5255782700 1 -- 172.21.15.6:0/2594121856 shutdown_connections 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.449+0000 7f5255782700 1 -- 172.21.15.6:0/2594121856 wait complete. 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.450+0000 7f5255782700 1 Processor -- start 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.450+0000 7f5255782700 1 -- start start 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f5255782700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52500b1630 0x7f5250158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f5255782700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52500cf5f0 0x7f52500c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f5255782700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5250165670 0x7f52500b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f5255782700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52500c7cc0 con 0x7f52500cf5f0 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f5255782700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52500c7040 con 0x7f52500b1630 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f5255782700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52500c7320 con 0x7f5250165670 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52500cf5f0 0x7f52500c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52500cf5f0 0x7f52500c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59064/0 (socket says 172.21.15.6:59064) 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524e7fc700 1 -- 172.21.15.6:0/1083188706 learned_addr learned my addr 172.21.15.6:0/1083188706 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524f7fe700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5250165670 0x7f52500b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524effd700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52500b1630 0x7f5250158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524e7fc700 1 -- 172.21.15.6:0/1083188706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5250165670 msgr2=0x7f52500b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524e7fc700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5250165670 0x7f52500b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524e7fc700 1 -- 172.21.15.6:0/1083188706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52500b1630 msgr2=0x7f5250158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524e7fc700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52500b1630 0x7f5250158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524e7fc700 1 -- 172.21.15.6:0/1083188706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248009a70 con 0x7f52500cf5f0 2023-11-28T13:01:06.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f524e7fc700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52500cf5f0 0x7f52500c80e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52400011e0 tx=0x7f524000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f52377fe700 1 -- 172.21.15.6:0/1083188706 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240004c80 con 0x7f52500cf5f0 2023-11-28T13:01:06.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.451+0000 7f52377fe700 1 -- 172.21.15.6:0/1083188706 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5240004e00 con 0x7f52500cf5f0 2023-11-28T13:01:06.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.452+0000 7f52377fe700 1 -- 172.21.15.6:0/1083188706 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240007310 con 0x7f52500cf5f0 2023-11-28T13:01:06.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.452+0000 7f5255782700 1 -- 172.21.15.6:0/1083188706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52500cb5b0 con 0x7f52500cf5f0 2023-11-28T13:01:06.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.452+0000 7f5255782700 1 -- 172.21.15.6:0/1083188706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52500cf0d0 con 0x7f52500cf5f0 2023-11-28T13:01:06.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.452+0000 7f5255782700 1 -- 172.21.15.6:0/1083188706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52500ce140 con 0x7f52500cf5f0 2023-11-28T13:01:06.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.453+0000 7f52377fe700 1 -- 172.21.15.6:0/1083188706 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f524001a020 con 0x7f52500cf5f0 2023-11-28T13:01:06.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.453+0000 7f52377fe700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f523c0682f0 0x7f523c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.453+0000 7f52377fe700 1 -- 172.21.15.6:0/1083188706 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5240085b90 con 0x7f52500cf5f0 2023-11-28T13:01:06.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.453+0000 7f524effd700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f523c0682f0 0x7f523c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.454+0000 7f524effd700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f523c0682f0 0x7f523c06a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f525015c200 tx=0x7f5244007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.457+0000 7f52377fe700 1 -- 172.21.15.6:0/1083188706 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5240051f90 con 0x7f52500cf5f0 2023-11-28T13:01:06.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.648+0000 7f5255782700 1 -- 172.21.15.6:0/1083188706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52500739f0 con 0x7f52500cf5f0 2023-11-28T13:01:06.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.648+0000 7f52377fe700 1 -- 172.21.15.6:0/1083188706 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52400555b0 con 0x7f52500cf5f0 2023-11-28T13:01:06.650 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:06.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 -- 172.21.15.6:0/1083188706 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f523c0682f0 msgr2=0x7f523c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f523c0682f0 0x7f523c06a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f525015c200 tx=0x7f5244007ec0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 -- 172.21.15.6:0/1083188706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52500cf5f0 msgr2=0x7f52500c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52500cf5f0 0x7f52500c80e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52400011e0 tx=0x7f524000ca30 comp rx=0 tx=0).stop 2023-11-28T13:01:06.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 -- 172.21.15.6:0/1083188706 shutdown_connections 2023-11-28T13:01:06.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f523c0682f0 0x7f523c06a7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5250165670 0x7f52500b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52500b1630 0x7f5250158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 --2- 172.21.15.6:0/1083188706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52500cf5f0 0x7f52500c80e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 -- 172.21.15.6:0/1083188706 >> 172.21.15.6:0/1083188706 conn(0x7f52500cc630 msgr2=0x7f525006bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:06.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 -- 172.21.15.6:0/1083188706 shutdown_connections 2023-11-28T13:01:06.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.650+0000 7f5255782700 1 -- 172.21.15.6:0/1083188706 wait complete. 2023-11-28T13:01:06.664 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:06.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.790+0000 7f82e9d90700 1 Processor -- start 2023-11-28T13:01:06.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.790+0000 7f82e9d90700 1 -- start start 2023-11-28T13:01:06.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.791+0000 7f82e9d90700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f82e40cf5f0 0x7f82e40caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.791+0000 7f82e9d90700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40cb3e0 0x7f82e406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.791+0000 7f82e9d90700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e40ce550 con 0x7f82e40b1630 2023-11-28T13:01:06.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.791+0000 7f82e9d90700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e40ce830 con 0x7f82e40cf5f0 2023-11-28T13:01:06.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.791+0000 7f82e9d90700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40ceb10 con 0x7f82e40cb3e0 2023-11-28T13:01:06.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.791+0000 7f82e37fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f82e40b1630 0x7f82e40b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46390/0 (socket says 172.21.15.6:46390) 2023-11-28T13:01:06.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.791+0000 7f82e37fe700 1 -- 172.21.15.6:0/3082852321 learned_addr learned my addr 172.21.15.6:0/3082852321 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:06.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.791+0000 7f82e2ffd700 1 --2- 172.21.15.6:0/3082852321 >> v2:172.21.15.61:3300/0 conn(0x7f82e40cf5f0 0x7f82e40caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e3fff700 1 --2- 172.21.15.6:0/3082852321 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40cb3e0 0x7f82e406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e27fc700 1 -- 172.21.15.6:0/3082852321 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966754925 0 0) 0x7f82e40ce550 con 0x7f82e40b1630 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e27fc700 1 -- 172.21.15.6:0/3082852321 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d00035b0 con 0x7f82e40b1630 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e2ffd700 1 -- 172.21.15.6:0/3082852321 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40cb3e0 msgr2=0x7f82e406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e2ffd700 1 --2- 172.21.15.6:0/3082852321 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40cb3e0 0x7f82e406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e2ffd700 1 -- 172.21.15.6:0/3082852321 >> v1:172.21.15.6:6789/0 conn(0x7f82e40b1630 legacy=0x7f82e40b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e2ffd700 1 -- 172.21.15.6:0/3082852321 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e40cedf0 con 0x7f82e40cf5f0 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e3fff700 1 --2- 172.21.15.6:0/3082852321 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40cb3e0 0x7f82e406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e2ffd700 1 --2- 172.21.15.6:0/3082852321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82e40cf5f0 0x7f82e40caea0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009870 tx=0x7f82cc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b98e116155f9fe8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e27fc700 1 -- 172.21.15.6:0/3082852321 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc011830 con 0x7f82e40cf5f0 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e27fc700 1 -- 172.21.15.6:0/3082852321 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc011990 con 0x7f82e40cf5f0 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.792+0000 7f82e27fc700 1 -- 172.21.15.6:0/3082852321 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc011d20 con 0x7f82e40cf5f0 2023-11-28T13:01:06.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.793+0000 7f82e9d90700 1 -- 172.21.15.6:0/3082852321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82e40cf5f0 msgr2=0x7f82e40caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.793+0000 7f82e9d90700 1 --2- 172.21.15.6:0/3082852321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82e40cf5f0 0x7f82e40caea0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009870 tx=0x7f82cc009b80 comp rx=0 tx=0).stop 2023-11-28T13:01:06.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.793+0000 7f82e9d90700 1 -- 172.21.15.6:0/3082852321 shutdown_connections 2023-11-28T13:01:06.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.793+0000 7f82e9d90700 1 --2- 172.21.15.6:0/3082852321 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40cb3e0 0x7f82e406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.793+0000 7f82e9d90700 1 --2- 172.21.15.6:0/3082852321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82e40cf5f0 0x7f82e40caea0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.793+0000 7f82e9d90700 1 -- 172.21.15.6:0/3082852321 >> 172.21.15.6:0/3082852321 conn(0x7f82e40cc630 msgr2=0x7f82e40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:06.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.793+0000 7f82e9d90700 1 -- 172.21.15.6:0/3082852321 shutdown_connections 2023-11-28T13:01:06.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.793+0000 7f82e9d90700 1 -- 172.21.15.6:0/3082852321 wait complete. 2023-11-28T13:01:06.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.794+0000 7f82e9d90700 1 Processor -- start 2023-11-28T13:01:06.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.794+0000 7f82e9d90700 1 -- start start 2023-11-28T13:01:06.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e9d90700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40b1630 0x7f82e40b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e9d90700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82e40cb3e0 0x7f82e414d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e9d90700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82e40cf5f0 0x7f82e40c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e9d90700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e414d5f0 con 0x7f82e40cf5f0 2023-11-28T13:01:06.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e9d90700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40b2fc0 con 0x7f82e40cb3e0 2023-11-28T13:01:06.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e9d90700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40b32a0 con 0x7f82e40b1630 2023-11-28T13:01:06.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82e40cf5f0 0x7f82e40c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82e40cf5f0 0x7f82e40c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59080/0 (socket says 172.21.15.6:59080) 2023-11-28T13:01:06.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e3fff700 1 -- 172.21.15.6:0/4197967978 learned_addr learned my addr 172.21.15.6:0/4197967978 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e37fe700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40b1630 0x7f82e40b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e2ffd700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82e40cb3e0 0x7f82e414d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e3fff700 1 -- 172.21.15.6:0/4197967978 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40b1630 msgr2=0x7f82e40b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e3fff700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40b1630 0x7f82e40b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e3fff700 1 -- 172.21.15.6:0/4197967978 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82e40cb3e0 msgr2=0x7f82e414d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e3fff700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82e40cb3e0 0x7f82e414d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e3fff700 1 -- 172.21.15.6:0/4197967978 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc009a70 con 0x7f82e40cf5f0 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.795+0000 7f82e2ffd700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82e40cb3e0 0x7f82e414d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.796+0000 7f82e3fff700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82e40cf5f0 0x7f82e40c8fa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00efc0 tx=0x7f82dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.796+0000 7f82dbfff700 1 -- 172.21.15.6:0/4197967978 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00cb40 con 0x7f82e40cf5f0 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.796+0000 7f82dbfff700 1 -- 172.21.15.6:0/4197967978 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc00f040 con 0x7f82e40cf5f0 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.796+0000 7f82e9d90700 1 -- 172.21.15.6:0/4197967978 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc0094a0 con 0x7f82e40cf5f0 2023-11-28T13:01:06.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.796+0000 7f82dbfff700 1 -- 172.21.15.6:0/4197967978 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc003a20 con 0x7f82e40cf5f0 2023-11-28T13:01:06.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.796+0000 7f82e9d90700 1 -- 172.21.15.6:0/4197967978 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e40ce140 con 0x7f82e40cf5f0 2023-11-28T13:01:06.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.797+0000 7f82e9d90700 1 -- 172.21.15.6:0/4197967978 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e40cf0d0 con 0x7f82e40cf5f0 2023-11-28T13:01:06.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.797+0000 7f82dbfff700 1 -- 172.21.15.6:0/4197967978 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f82dc014480 con 0x7f82e40cf5f0 2023-11-28T13:01:06.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.798+0000 7f82dbfff700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82c40682f0 0x7f82c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:06.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.798+0000 7f82dbfff700 1 -- 172.21.15.6:0/4197967978 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f82dc086a30 con 0x7f82e40cf5f0 2023-11-28T13:01:06.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.798+0000 7f82e37fe700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82c40682f0 0x7f82c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:06.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.798+0000 7f82e37fe700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82c40682f0 0x7f82c406a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f82d4007fd0 tx=0x7f82d4007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:06.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.802+0000 7f82dbfff700 1 -- 172.21.15.6:0/4197967978 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f82dc052f40 con 0x7f82e40cf5f0 2023-11-28T13:01:06.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.997+0000 7f82e9d90700 1 -- 172.21.15.6:0/4197967978 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82e40ce2d0 con 0x7f82e40cf5f0 2023-11-28T13:01:06.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:06.997+0000 7f82dbfff700 1 -- 172.21.15.6:0/4197967978 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f82dc056560 con 0x7f82e40cf5f0 2023-11-28T13:01:06.999 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:07.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 -- 172.21.15.6:0/4197967978 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82c40682f0 msgr2=0x7f82c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82c40682f0 0x7f82c406a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f82d4007fd0 tx=0x7f82d4007dc0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 -- 172.21.15.6:0/4197967978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82e40cf5f0 msgr2=0x7f82e40c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82e40cf5f0 0x7f82e40c8fa0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00efc0 tx=0x7f82dc00c3a0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 -- 172.21.15.6:0/4197967978 shutdown_connections 2023-11-28T13:01:07.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82e40cf5f0 0x7f82e40c8fa0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82c40682f0 0x7f82c406a7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82e40b1630 0x7f82e40b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 --2- 172.21.15.6:0/4197967978 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f82e40cb3e0 0x7f82e414d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 -- 172.21.15.6:0/4197967978 >> 172.21.15.6:0/4197967978 conn(0x7f82e40cc630 msgr2=0x7f82e40cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:07.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 -- 172.21.15.6:0/4197967978 shutdown_connections 2023-11-28T13:01:07.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.000+0000 7f82e9d90700 1 -- 172.21.15.6:0/4197967978 wait complete. 2023-11-28T13:01:07.013 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:07.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.140+0000 7f7894ae7700 1 Processor -- start 2023-11-28T13:01:07.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.140+0000 7f7894ae7700 1 -- start start 2023-11-28T13:01:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f7894ae7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f78900cd140 0x7f789006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f7894ae7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f789006d140 0x7f789006b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f7894ae7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78900ae730 con 0x7f789015a640 2023-11-28T13:01:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f7894ae7700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78900cb210 con 0x7f789006d140 2023-11-28T13:01:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f7894ae7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78900c3e10 con 0x7f78900cd140 2023-11-28T13:01:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f788ed9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f789006d140 0x7f789006b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f788ed9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f789006d140 0x7f789006b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59088/0 (socket says 172.21.15.6:59088) 2023-11-28T13:01:07.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f788ed9d700 1 -- 172.21.15.6:0/2130023506 learned_addr learned my addr 172.21.15.6:0/2130023506 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f788dd9b700 1 --2- 172.21.15.6:0/2130023506 >> v2:172.21.15.61:3300/0 conn(0x7f78900cd140 0x7f789006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f788ed9d700 1 -- 172.21.15.6:0/2130023506 >> v2:172.21.15.61:3300/0 conn(0x7f78900cd140 msgr2=0x7f789006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f788ed9d700 1 --2- 172.21.15.6:0/2130023506 >> v2:172.21.15.61:3300/0 conn(0x7f78900cd140 0x7f789006b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f788ed9d700 1 -- 172.21.15.6:0/2130023506 >> v1:172.21.15.61:6789/0 conn(0x7f789015a640 legacy=0x7f78900c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f788ed9d700 1 -- 172.21.15.6:0/2130023506 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789006e9e0 con 0x7f789006d140 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.141+0000 7f788dd9b700 1 --2- 172.21.15.6:0/2130023506 >> v2:172.21.15.61:3300/0 conn(0x7f78900cd140 0x7f789006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f788ed9d700 1 --2- 172.21.15.6:0/2130023506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f789006d140 0x7f789006b730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f78900d0c50 tx=0x7f788800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a9e84cccfe4a18 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f788d59a700 1 -- 172.21.15.6:0/2130023506 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788800e070 con 0x7f789006d140 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f788d59a700 1 -- 172.21.15.6:0/2130023506 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788800b9c0 con 0x7f789006d140 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f788d59a700 1 -- 172.21.15.6:0/2130023506 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788800bd70 con 0x7f789006d140 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f7894ae7700 1 -- 172.21.15.6:0/2130023506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f789006d140 msgr2=0x7f789006b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f7894ae7700 1 --2- 172.21.15.6:0/2130023506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f789006d140 0x7f789006b730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f78900d0c50 tx=0x7f788800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f7894ae7700 1 -- 172.21.15.6:0/2130023506 shutdown_connections 2023-11-28T13:01:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f7894ae7700 1 --2- 172.21.15.6:0/2130023506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f789006d140 0x7f789006b730 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f7894ae7700 1 --2- 172.21.15.6:0/2130023506 >> v2:172.21.15.61:3300/0 conn(0x7f78900cd140 0x7f789006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f7894ae7700 1 -- 172.21.15.6:0/2130023506 >> 172.21.15.6:0/2130023506 conn(0x7f78900cc630 msgr2=0x7f78900cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f7894ae7700 1 -- 172.21.15.6:0/2130023506 shutdown_connections 2023-11-28T13:01:07.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.142+0000 7f7894ae7700 1 -- 172.21.15.6:0/2130023506 wait complete. 2023-11-28T13:01:07.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.144+0000 7f7894ae7700 1 Processor -- start 2023-11-28T13:01:07.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.144+0000 7f7894ae7700 1 -- start start 2023-11-28T13:01:07.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f7894ae7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789006d140 0x7f789015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f7894ae7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78900cd140 0x7f78900b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f7894ae7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789015a640 0x7f78900b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f7894ae7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78900b3360 con 0x7f78900cd140 2023-11-28T13:01:07.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f7894ae7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78900b29e0 con 0x7f789015a640 2023-11-28T13:01:07.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f7894ae7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78900b4b20 con 0x7f789006d140 2023-11-28T13:01:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78900cd140 0x7f78900b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78900cd140 0x7f78900b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59092/0 (socket says 172.21.15.6:59092) 2023-11-28T13:01:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788dd9b700 1 -- 172.21.15.6:0/525890914 learned_addr learned my addr 172.21.15.6:0/525890914 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788ed9d700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789015a640 0x7f78900b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788e59c700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789006d140 0x7f789015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788dd9b700 1 -- 172.21.15.6:0/525890914 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789006d140 msgr2=0x7f789015ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788dd9b700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789006d140 0x7f789015ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788dd9b700 1 -- 172.21.15.6:0/525890914 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789015a640 msgr2=0x7f78900b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788dd9b700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789015a640 0x7f78900b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788dd9b700 1 -- 172.21.15.6:0/525890914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7888009b30 con 0x7f78900cd140 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f788dd9b700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78900cd140 0x7f78900b3780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f788000e810 tx=0x7f788000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f787effd700 1 -- 172.21.15.6:0/525890914 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880004230 con 0x7f78900cd140 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f787effd700 1 -- 172.21.15.6:0/525890914 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7880004390 con 0x7f78900cd140 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f7894ae7700 1 -- 172.21.15.6:0/525890914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78900aec60 con 0x7f78900cd140 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f787effd700 1 -- 172.21.15.6:0/525890914 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78800036a0 con 0x7f78900cd140 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.145+0000 7f7894ae7700 1 -- 172.21.15.6:0/525890914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78900ae980 con 0x7f78900cd140 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.146+0000 7f7894ae7700 1 -- 172.21.15.6:0/525890914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78900aef40 con 0x7f78900cd140 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.147+0000 7f787effd700 1 -- 172.21.15.6:0/525890914 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7880022070 con 0x7f78900cd140 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.147+0000 7f787effd700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78700682f0 0x7f787006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.147+0000 7f787effd700 1 -- 172.21.15.6:0/525890914 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7880085b80 con 0x7f78900cd140 2023-11-28T13:01:07.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.147+0000 7f788e59c700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78700682f0 0x7f787006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.148+0000 7f788e59c700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78700682f0 0x7f787006a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7884000f60 tx=0x7f7884009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.151+0000 7f787effd700 1 -- 172.21.15.6:0/525890914 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7880051fa0 con 0x7f78900cd140 2023-11-28T13:01:07.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.348+0000 7f7894ae7700 1 -- 172.21.15.6:0/525890914 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78900b34a0 con 0x7f78900cd140 2023-11-28T13:01:07.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.348+0000 7f787effd700 1 -- 172.21.15.6:0/525890914 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78800555c0 con 0x7f78900cd140 2023-11-28T13:01:07.350 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:07.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 -- 172.21.15.6:0/525890914 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78700682f0 msgr2=0x7f787006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78700682f0 0x7f787006a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7884000f60 tx=0x7f7884009040 comp rx=0 tx=0).stop 2023-11-28T13:01:07.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 -- 172.21.15.6:0/525890914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78900cd140 msgr2=0x7f78900b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78900cd140 0x7f78900b3780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f788000e810 tx=0x7f788000ebd0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 -- 172.21.15.6:0/525890914 shutdown_connections 2023-11-28T13:01:07.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f789015a640 0x7f78900b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f78700682f0 0x7f787006a7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f789006d140 0x7f789015ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 --2- 172.21.15.6:0/525890914 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78900cd140 0x7f78900b3780 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 -- 172.21.15.6:0/525890914 >> 172.21.15.6:0/525890914 conn(0x7f78900cc630 msgr2=0x7f78900cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:07.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 -- 172.21.15.6:0/525890914 shutdown_connections 2023-11-28T13:01:07.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.350+0000 7f7894ae7700 1 -- 172.21.15.6:0/525890914 wait complete. 2023-11-28T13:01:07.364 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:07.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.491+0000 7fefb1f76700 1 Processor -- start 2023-11-28T13:01:07.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.491+0000 7fefb1f76700 1 -- start start 2023-11-28T13:01:07.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.491+0000 7fefb1f76700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac15a640 0x7fefac0c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.491+0000 7fefb1f76700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fefac0cd140 0x7fefac06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.491+0000 7fefb1f76700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefac0ae730 con 0x7fefac06d140 2023-11-28T13:01:07.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.491+0000 7fefb1f76700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefac0cb210 con 0x7fefac0cd140 2023-11-28T13:01:07.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.491+0000 7fefb1f76700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac06e9e0 con 0x7fefac15a640 2023-11-28T13:01:07.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefabfff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fefac06d140 0x7fefac0cb630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46422/0 (socket says 172.21.15.6:46422) 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefabfff700 1 -- 172.21.15.6:0/56880328 learned_addr learned my addr 172.21.15.6:0/56880328 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaaffd700 1 --2- 172.21.15.6:0/56880328 >> v2:172.21.15.61:3300/0 conn(0x7fefac0cd140 0x7fefac06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefab7fe700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac15a640 0x7fefac0c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944975283 0 0) 0x7fefac0ae730 con 0x7fefac06d140 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef940035b0 con 0x7fefac06d140 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3758451706 0 0) 0x7fef940035b0 con 0x7fefac06d140 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac15a640 msgr2=0x7fefac0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac15a640 0x7fefac0c3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 >> v2:172.21.15.61:3300/0 conn(0x7fefac0cd140 msgr2=0x7fefac06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 --2- 172.21.15.6:0/56880328 >> v2:172.21.15.61:3300/0 conn(0x7fefac0cd140 0x7fefac06b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0ae980 con 0x7fefac06d140 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fefa40019f0 con 0x7fefac06d140 2023-11-28T13:01:07.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 >> v1:172.21.15.6:6789/0 conn(0x7fefac06d140 legacy=0x7fefac0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 0x7fef94003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef94004430 0x7fef940048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac15a640 0x7fef94004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0cb210 con 0x7fef94003a00 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac06e9e0 con 0x7fef94004430 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9400ba10 con 0x7fefac15a640 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefab7fe700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 0x7fef94003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefabfff700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac15a640 0x7fef94004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.492+0000 7fefaaffd700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef94004430 0x7fef940048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefab7fe700 1 -- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac15a640 msgr2=0x7fef94004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefab7fe700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac15a640 0x7fef94004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefab7fe700 1 -- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef94004430 msgr2=0x7fef940048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefab7fe700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef94004430 0x7fef940048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefab7fe700 1 -- 172.21.15.6:0/56880328 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940035b0 con 0x7fef94003a00 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefaaffd700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef94004430 0x7fef940048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefab7fe700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 0x7fef94003e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009b60 tx=0x7fef94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b8012151c73cb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00e6c0 con 0x7fef94003a00 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c00e820 con 0x7fef94003a00 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefaa7fc700 1 -- 172.21.15.6:0/56880328 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00eb90 con 0x7fef94003a00 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefb1f76700 1 -- 172.21.15.6:0/56880328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 msgr2=0x7fef94003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefb1f76700 1 --2- 172.21.15.6:0/56880328 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 0x7fef94003e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009b60 tx=0x7fef94005880 comp rx=0 tx=0).stop 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefabfff700 1 -- 172.21.15.6:0/56880328 reap_dead start 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefb1f76700 1 -- 172.21.15.6:0/56880328 shutdown_connections 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefb1f76700 1 -- 172.21.15.6:0/56880328 >> 172.21.15.6:0/56880328 conn(0x7fefac0cc630 msgr2=0x7fefac0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:07.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefb1f76700 1 -- 172.21.15.6:0/56880328 shutdown_connections 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.493+0000 7fefb1f76700 1 -- 172.21.15.6:0/56880328 wait complete. 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.495+0000 7fefb1f76700 1 Processor -- start 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.495+0000 7fefb1f76700 1 -- start start 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefb1f76700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 0x7fefac14bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefb1f76700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefac0c49c0 0x7fefac14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefb1f76700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac14d2c0 0x7fefac158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefb1f76700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0c5f40 con 0x7fef94003a00 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefb1f76700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac1efd30 con 0x7fefac0c49c0 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefb1f76700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0c4ea0 con 0x7fefac14d2c0 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefab7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 0x7fefac14bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefab7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 0x7fefac14bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59110/0 (socket says 172.21.15.6:59110) 2023-11-28T13:01:07.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefab7fe700 1 -- 172.21.15.6:0/1641861352 learned_addr learned my addr 172.21.15.6:0/1641861352 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefabfff700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac14d2c0 0x7fefac158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefaaffd700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefac0c49c0 0x7fefac14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefab7fe700 1 -- 172.21.15.6:0/1641861352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac14d2c0 msgr2=0x7fefac158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefab7fe700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac14d2c0 0x7fefac158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefab7fe700 1 -- 172.21.15.6:0/1641861352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefac0c49c0 msgr2=0x7fefac14cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefab7fe700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefac0c49c0 0x7fefac14cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefab7fe700 1 -- 172.21.15.6:0/1641861352 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0aef40 con 0x7fef94003a00 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefab7fe700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 0x7fefac14bbc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0090b0 tx=0x7fef9c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fef93fff700 1 -- 172.21.15.6:0/1641861352 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c021070 con 0x7fef94003a00 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fef93fff700 1 -- 172.21.15.6:0/1641861352 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c004590 con 0x7fef94003a00 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefb1f76700 1 -- 172.21.15.6:0/1641861352 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefac0aec60 con 0x7fef94003a00 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fef93fff700 1 -- 172.21.15.6:0/1641861352 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00e6c0 con 0x7fef94003a00 2023-11-28T13:01:07.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.496+0000 7fefb1f76700 1 -- 172.21.15.6:0/1641861352 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefac0af220 con 0x7fef94003a00 2023-11-28T13:01:07.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.497+0000 7fefb1f76700 1 -- 172.21.15.6:0/1641861352 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefac0739f0 con 0x7fef94003a00 2023-11-28T13:01:07.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.498+0000 7fef93fff700 1 -- 172.21.15.6:0/1641861352 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fef9c02e030 con 0x7fef94003a00 2023-11-28T13:01:07.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.498+0000 7fef93fff700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef980682f0 0x7fef9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.498+0000 7fef93fff700 1 -- 172.21.15.6:0/1641861352 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fef9c087450 con 0x7fef94003a00 2023-11-28T13:01:07.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.498+0000 7fefaaffd700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef980682f0 0x7fef9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.499+0000 7fefaaffd700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef980682f0 0x7fef9806a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fefac159620 tx=0x7fefa0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.503+0000 7fef93fff700 1 -- 172.21.15.6:0/1641861352 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fef9c0539c0 con 0x7fef94003a00 2023-11-28T13:01:07.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.695+0000 7fefb1f76700 1 -- 172.21.15.6:0/1641861352 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fefac1e3f30 con 0x7fef94003a00 2023-11-28T13:01:07.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.696+0000 7fef93fff700 1 -- 172.21.15.6:0/1641861352 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fef9c056fe0 con 0x7fef94003a00 2023-11-28T13:01:07.697 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:07.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 -- 172.21.15.6:0/1641861352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef980682f0 msgr2=0x7fef9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef980682f0 0x7fef9806a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fefac159620 tx=0x7fefa0008040 comp rx=0 tx=0).stop 2023-11-28T13:01:07.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 -- 172.21.15.6:0/1641861352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 msgr2=0x7fefac14bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 0x7fefac14bbc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0090b0 tx=0x7fef9c0047c0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 -- 172.21.15.6:0/1641861352 shutdown_connections 2023-11-28T13:01:07.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fef980682f0 0x7fef9806a7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fefac14d2c0 0x7fefac158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef94003a00 0x7fefac14bbc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 --2- 172.21.15.6:0/1641861352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fefac0c49c0 0x7fefac14cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 -- 172.21.15.6:0/1641861352 >> 172.21.15.6:0/1641861352 conn(0x7fefac0cc630 msgr2=0x7fefac1e3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:07.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 -- 172.21.15.6:0/1641861352 shutdown_connections 2023-11-28T13:01:07.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.698+0000 7fefb1f76700 1 -- 172.21.15.6:0/1641861352 wait complete. 2023-11-28T13:01:07.712 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:07.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.837+0000 7fa442dd6700 1 Processor -- start 2023-11-28T13:01:07.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.837+0000 7fa442dd6700 1 -- start start 2023-11-28T13:01:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa442dd6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa43c159590 0x7fa43c06e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa442dd6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa43c0ce890 0x7fa43c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa442dd6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43c15d120 con 0x7fa43c15db70 2023-11-28T13:01:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa442dd6700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c0cf250 con 0x7fa43c0ce890 2023-11-28T13:01:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa442dd6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c15d6e0 con 0x7fa43c159590 2023-11-28T13:01:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa441373700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa43c0ce890 0x7fa43c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa441373700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa43c0ce890 0x7fa43c0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59122/0 (socket says 172.21.15.6:59122) 2023-11-28T13:01:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa441373700 1 -- 172.21.15.6:0/4214678040 learned_addr learned my addr 172.21.15.6:0/4214678040 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:07.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa440b72700 1 --2- 172.21.15.6:0/4214678040 >> v2:172.21.15.61:3300/0 conn(0x7fa43c159590 0x7fa43c06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa441373700 1 -- 172.21.15.6:0/4214678040 >> v2:172.21.15.61:3300/0 conn(0x7fa43c159590 msgr2=0x7fa43c06e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa441373700 1 --2- 172.21.15.6:0/4214678040 >> v2:172.21.15.61:3300/0 conn(0x7fa43c159590 0x7fa43c06e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa441373700 1 -- 172.21.15.6:0/4214678040 >> v1:172.21.15.61:6789/0 conn(0x7fa43c15db70 legacy=0x7fa43c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa441373700 1 -- 172.21.15.6:0/4214678040 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c0aef60 con 0x7fa43c0ce890 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.838+0000 7fa440b72700 1 --2- 172.21.15.6:0/4214678040 >> v2:172.21.15.61:3300/0 conn(0x7fa43c159590 0x7fa43c06e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa441373700 1 --2- 172.21.15.6:0/4214678040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 0x7fa43c0ced10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa43c06bd10 tx=0x7fa43400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecfebfe6ddf2cfa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa43b7fe700 1 -- 172.21.15.6:0/4214678040 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43400e070 con 0x7fa43c0ce890 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa43b7fe700 1 -- 172.21.15.6:0/4214678040 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43400b9c0 con 0x7fa43c0ce890 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa43b7fe700 1 -- 172.21.15.6:0/4214678040 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43400bd70 con 0x7fa43c0ce890 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa442dd6700 1 -- 172.21.15.6:0/4214678040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 msgr2=0x7fa43c0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa442dd6700 1 --2- 172.21.15.6:0/4214678040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 0x7fa43c0ced10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa43c06bd10 tx=0x7fa43400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:07.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa442dd6700 1 -- 172.21.15.6:0/4214678040 shutdown_connections 2023-11-28T13:01:07.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa442dd6700 1 --2- 172.21.15.6:0/4214678040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 0x7fa43c0ced10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa442dd6700 1 --2- 172.21.15.6:0/4214678040 >> v2:172.21.15.61:3300/0 conn(0x7fa43c159590 0x7fa43c06e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa442dd6700 1 -- 172.21.15.6:0/4214678040 >> 172.21.15.6:0/4214678040 conn(0x7fa43c0b1650 msgr2=0x7fa43c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:07.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa442dd6700 1 -- 172.21.15.6:0/4214678040 shutdown_connections 2023-11-28T13:01:07.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.839+0000 7fa442dd6700 1 -- 172.21.15.6:0/4214678040 wait complete. 2023-11-28T13:01:07.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.841+0000 7fa442dd6700 1 Processor -- start 2023-11-28T13:01:07.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.841+0000 7fa442dd6700 1 -- start start 2023-11-28T13:01:07.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa442dd6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 0x7fa43c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa442dd6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa43c159590 0x7fa43c1575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa442dd6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa43c15db70 0x7fa43c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa442dd6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c1583e0 con 0x7fa43c0ce890 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa442dd6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c157e20 con 0x7fa43c15db70 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa442dd6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c158100 con 0x7fa43c159590 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa440b72700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 0x7fa43c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa440b72700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 0x7fa43c1552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59124/0 (socket says 172.21.15.6:59124) 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa440b72700 1 -- 172.21.15.6:0/2829714654 learned_addr learned my addr 172.21.15.6:0/2829714654 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa441373700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa43c15db70 0x7fa43c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa440b72700 1 -- 172.21.15.6:0/2829714654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa43c159590 msgr2=0x7fa43c1575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.842+0000 7fa43bfff700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa43c159590 0x7fa43c1575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.843+0000 7fa440b72700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa43c159590 0x7fa43c1575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.843+0000 7fa440b72700 1 -- 172.21.15.6:0/2829714654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa43c15db70 msgr2=0x7fa43c1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:07.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.843+0000 7fa440b72700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa43c15db70 0x7fa43c1524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:07.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.843+0000 7fa440b72700 1 -- 172.21.15.6:0/2829714654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa434009b30 con 0x7fa43c0ce890 2023-11-28T13:01:07.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.843+0000 7fa440b72700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 0x7fa43c1552b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa430010700 tx=0x7fa430010ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.843+0000 7fa4397fa700 1 -- 172.21.15.6:0/2829714654 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa430011920 con 0x7fa43c0ce890 2023-11-28T13:01:07.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.843+0000 7fa4397fa700 1 -- 172.21.15.6:0/2829714654 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa430011a80 con 0x7fa43c0ce890 2023-11-28T13:01:07.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.843+0000 7fa442dd6700 1 -- 172.21.15.6:0/2829714654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c15d370 con 0x7fa43c0ce890 2023-11-28T13:01:07.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.843+0000 7fa4397fa700 1 -- 172.21.15.6:0/2829714654 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa430011e30 con 0x7fa43c0ce890 2023-11-28T13:01:07.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.843+0000 7fa442dd6700 1 -- 172.21.15.6:0/2829714654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c0af240 con 0x7fa43c0ce890 2023-11-28T13:01:07.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.844+0000 7fa442dd6700 1 -- 172.21.15.6:0/2829714654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c15e010 con 0x7fa43c0ce890 2023-11-28T13:01:07.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.844+0000 7fa4397fa700 1 -- 172.21.15.6:0/2829714654 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa430019430 con 0x7fa43c0ce890 2023-11-28T13:01:07.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.845+0000 7fa4397fa700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa420068320 0x7fa42006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:07.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.845+0000 7fa4397fa700 1 -- 172.21.15.6:0/2829714654 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa430088240 con 0x7fa43c0ce890 2023-11-28T13:01:07.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.845+0000 7fa43bfff700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa420068320 0x7fa42006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:07.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.845+0000 7fa43bfff700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa420068320 0x7fa42006a7d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa43c1f2900 tx=0x7fa428009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:07.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:07.849+0000 7fa4397fa700 1 -- 172.21.15.6:0/2829714654 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa4300515e0 con 0x7fa43c0ce890 2023-11-28T13:01:08.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.061+0000 7fa442dd6700 1 -- 172.21.15.6:0/2829714654 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa43c0675a0 con 0x7fa43c0ce890 2023-11-28T13:01:08.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.061+0000 7fa4397fa700 1 -- 172.21.15.6:0/2829714654 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa4300546d0 con 0x7fa43c0ce890 2023-11-28T13:01:08.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.063+0000 7fa442dd6700 1 -- 172.21.15.6:0/2829714654 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa420068320 msgr2=0x7fa42006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa420068320 0x7fa42006a7d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa43c1f2900 tx=0x7fa428009040 comp rx=0 tx=0).stop 2023-11-28T13:01:08.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 -- 172.21.15.6:0/2829714654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 msgr2=0x7fa43c1552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 0x7fa43c1552b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa430010700 tx=0x7fa430010ac0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 -- 172.21.15.6:0/2829714654 shutdown_connections 2023-11-28T13:01:08.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa43c15db70 0x7fa43c1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa43c0ce890 0x7fa43c1552b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa420068320 0x7fa42006a7d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 --2- 172.21.15.6:0/2829714654 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa43c159590 0x7fa43c1575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 -- 172.21.15.6:0/2829714654 >> 172.21.15.6:0/2829714654 conn(0x7fa43c0b1650 msgr2=0x7fa43c0afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 -- 172.21.15.6:0/2829714654 shutdown_connections 2023-11-28T13:01:08.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.064+0000 7fa442dd6700 1 -- 172.21.15.6:0/2829714654 wait complete. 2023-11-28T13:01:08.068 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:01:08.078 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:01:08.078 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:08.098 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:08.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.261+0000 7f73103c4700 1 Processor -- start 2023-11-28T13:01:08.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.261+0000 7f73103c4700 1 -- start start 2023-11-28T13:01:08.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.262+0000 7f73103c4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f730815a5a0 0x7f730806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.262+0000 7f73103c4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f73080ca280 0x7f73080cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.262+0000 7f73103c4700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73080cb570 con 0x7f73080cbf40 2023-11-28T13:01:08.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.262+0000 7f73103c4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73080ca6f0 con 0x7f73080ca280 2023-11-28T13:01:08.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.262+0000 7f73103c4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73080ca960 con 0x7f730815a5a0 2023-11-28T13:01:08.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.262+0000 7f730e961700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f73080cbf40 0x7f73080cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46454/0 (socket says 172.21.15.6:46454) 2023-11-28T13:01:08.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.262+0000 7f730e961700 1 -- 172.21.15.6:0/2665443399 learned_addr learned my addr 172.21.15.6:0/2665443399 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:08.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998034145 0 0) 0x7f73080cb570 con 0x7f73080cbf40 2023-11-28T13:01:08.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72f40035b0 con 0x7f73080cbf40 2023-11-28T13:01:08.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730e160700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f730815a5a0 0x7f730806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d95f700 1 --2- 172.21.15.6:0/2665443399 >> v2:172.21.15.61:3300/0 conn(0x7f73080ca280 0x7f73080cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1666242914 0 0) 0x7f72f40035b0 con 0x7f73080cbf40 2023-11-28T13:01:08.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f730815a5a0 msgr2=0x7f730806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f730815a5a0 0x7f730806b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 >> v2:172.21.15.61:3300/0 conn(0x7f73080ca280 msgr2=0x7f73080cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 --2- 172.21.15.6:0/2665443399 >> v2:172.21.15.61:3300/0 conn(0x7f73080ca280 0x7f73080cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080cac40 con 0x7f73080cbf40 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f73000019f0 con 0x7f73080cbf40 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 >> v1:172.21.15.6:6789/0 conn(0x7f73080cbf40 legacy=0x7f73080cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72f4003a00 0x7f72f4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f730815a5a0 0x7f72f4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 0x7f72f4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080ca6f0 con 0x7f72f4004d90 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080ca960 con 0x7f72f4003a00 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f400ba10 con 0x7f730815a5a0 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730e961700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 0x7f72f4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730e160700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72f4003a00 0x7f72f4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730d95f700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f730815a5a0 0x7f72f4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.263+0000 7f730e961700 1 -- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f730815a5a0 msgr2=0x7f72f4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f730e961700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f730815a5a0 0x7f72f4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f730e961700 1 -- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72f4003a00 msgr2=0x7f72f4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f730e961700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72f4003a00 0x7f72f4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f730e961700 1 -- 172.21.15.6:0/2665443399 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f40035b0 con 0x7f72f4004d90 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f730d95f700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f730815a5a0 0x7f72f4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f730e961700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 0x7f72f4005200 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f730000a8d0 tx=0x7f72f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752c883928964be9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730000a340 con 0x7f72f4004d90 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7300012600 con 0x7f72f4004d90 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f730d15e700 1 -- 172.21.15.6:0/2665443399 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000129b0 con 0x7f72f4004d90 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f73103c4700 1 -- 172.21.15.6:0/2665443399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 msgr2=0x7f72f4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f73103c4700 1 --2- 172.21.15.6:0/2665443399 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 0x7f72f4005200 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f730000a8d0 tx=0x7f72f4005880 comp rx=0 tx=0).stop 2023-11-28T13:01:08.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f730e961700 1 -- 172.21.15.6:0/2665443399 reap_dead start 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f73103c4700 1 -- 172.21.15.6:0/2665443399 shutdown_connections 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f73103c4700 1 -- 172.21.15.6:0/2665443399 >> 172.21.15.6:0/2665443399 conn(0x7f73080b1650 msgr2=0x7f73080b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.264+0000 7f73103c4700 1 -- 172.21.15.6:0/2665443399 shutdown_connections 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.265+0000 7f73103c4700 1 -- 172.21.15.6:0/2665443399 wait complete. 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.266+0000 7f73103c4700 1 Processor -- start 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.266+0000 7f73103c4700 1 -- start start 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f73103c4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 0x7f73080b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f73103c4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7308155890 0x7f7308155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f73103c4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73080b32f0 0x7f73080b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f73103c4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080b2950 con 0x7f72f4004d90 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f73103c4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080b3ca0 con 0x7f7308155890 2023-11-28T13:01:08.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f73103c4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73080b2b60 con 0x7f73080b32f0 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730e160700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 0x7f73080b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730e160700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 0x7f73080b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59142/0 (socket says 172.21.15.6:59142) 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730e160700 1 -- 172.21.15.6:0/2109604107 learned_addr learned my addr 172.21.15.6:0/2109604107 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730d95f700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7308155890 0x7f7308155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730e961700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73080b32f0 0x7f73080b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730e160700 1 -- 172.21.15.6:0/2109604107 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73080b32f0 msgr2=0x7f73080b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730e160700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73080b32f0 0x7f73080b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730e160700 1 -- 172.21.15.6:0/2109604107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7308155890 msgr2=0x7f7308155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730e160700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7308155890 0x7f7308155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730e160700 1 -- 172.21.15.6:0/2109604107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080cb200 con 0x7f72f4004d90 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f730e160700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 0x7f73080b5310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f73040119a0 tx=0x7f7304011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f72feffd700 1 -- 172.21.15.6:0/2109604107 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400f940 con 0x7f72f4004d90 2023-11-28T13:01:08.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f72feffd700 1 -- 172.21.15.6:0/2109604107 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730400faa0 con 0x7f72f4004d90 2023-11-28T13:01:08.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f73103c4700 1 -- 172.21.15.6:0/2109604107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73080caf20 con 0x7f72f4004d90 2023-11-28T13:01:08.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f72feffd700 1 -- 172.21.15.6:0/2109604107 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400fe30 con 0x7f72f4004d90 2023-11-28T13:01:08.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.267+0000 7f73103c4700 1 -- 172.21.15.6:0/2109604107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730815a970 con 0x7f72f4004d90 2023-11-28T13:01:08.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.268+0000 7f73103c4700 1 -- 172.21.15.6:0/2109604107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73080675a0 con 0x7f72f4004d90 2023-11-28T13:01:08.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.269+0000 7f72feffd700 1 -- 172.21.15.6:0/2109604107 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7304014440 con 0x7f72f4004d90 2023-11-28T13:01:08.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.269+0000 7f72feffd700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72ec0682f0 0x7f72ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.269+0000 7f72feffd700 1 -- 172.21.15.6:0/2109604107 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7304086900 con 0x7f72f4004d90 2023-11-28T13:01:08.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.269+0000 7f730d95f700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72ec0682f0 0x7f72ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.270+0000 7f730d95f700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72ec0682f0 0x7f72ec06a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f73080c9770 tx=0x7f72f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.274+0000 7f72feffd700 1 -- 172.21.15.6:0/2109604107 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7304052d20 con 0x7f72f4004d90 2023-11-28T13:01:08.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.467+0000 7f73103c4700 1 -- 172.21.15.6:0/2109604107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73080af940 con 0x7f72f4004d90 2023-11-28T13:01:08.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.468+0000 7f72feffd700 1 -- 172.21.15.6:0/2109604107 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19594 (secure 0 0 0) 0x7f7304056340 con 0x7f72f4004d90 2023-11-28T13:01:08.470 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:08.470 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":25,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:00:25.865312+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:08.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 -- 172.21.15.6:0/2109604107 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72ec0682f0 msgr2=0x7f72ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72ec0682f0 0x7f72ec06a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f73080c9770 tx=0x7f72f8008040 comp rx=0 tx=0).stop 2023-11-28T13:01:08.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 -- 172.21.15.6:0/2109604107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 msgr2=0x7f73080b5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 0x7f73080b5310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f73040119a0 tx=0x7f7304011d60 comp rx=0 tx=0).stop 2023-11-28T13:01:08.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 -- 172.21.15.6:0/2109604107 shutdown_connections 2023-11-28T13:01:08.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72ec0682f0 0x7f72ec06a7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73080b32f0 0x7f73080b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f72f4004d90 0x7f73080b5310 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 --2- 172.21.15.6:0/2109604107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7308155890 0x7f7308155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 -- 172.21.15.6:0/2109604107 >> 172.21.15.6:0/2109604107 conn(0x7f73080b1650 msgr2=0x7f7308158800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 -- 172.21.15.6:0/2109604107 shutdown_connections 2023-11-28T13:01:08.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.470+0000 7f73103c4700 1 -- 172.21.15.6:0/2109604107 wait complete. 2023-11-28T13:01:08.484 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:08.485 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:08.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.612+0000 7f8e938f1700 1 Processor -- start 2023-11-28T13:01:08.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.612+0000 7f8e938f1700 1 -- start start 2023-11-28T13:01:08.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e938f1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8e8c0b1630 0x7f8e8c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e938f1700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8e8c06afc0 0x7f8e8c06b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e938f1700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8c0ce550 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e938f1700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8c0ce830 con 0x7f8e8c06afc0 2023-11-28T13:01:08.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e938f1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8c0ceb10 con 0x7f8e8c0b1630 2023-11-28T13:01:08.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e91e8e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8e8c06afc0 0x7f8e8c06b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e90e8c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f8e8c0b1a00 0x7f8e8c0caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46470/0 (socket says 172.21.15.6:46470) 2023-11-28T13:01:08.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e91e8e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8e8c06afc0 0x7f8e8c06b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59154/0 (socket says 172.21.15.6:59154) 2023-11-28T13:01:08.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e90e8c700 1 -- 172.21.15.6:0/2261397924 learned_addr learned my addr 172.21.15.6:0/2261397924 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:08.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e9168d700 1 --2- 172.21.15.6:0/2261397924 >> v2:172.21.15.61:3300/0 conn(0x7f8e8c0b1630 0x7f8e8c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e91e8e700 1 -- 172.21.15.6:0/2261397924 >> v2:172.21.15.61:3300/0 conn(0x7f8e8c0b1630 msgr2=0x7f8e8c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e91e8e700 1 --2- 172.21.15.6:0/2261397924 >> v2:172.21.15.61:3300/0 conn(0x7f8e8c0b1630 0x7f8e8c0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e91e8e700 1 -- 172.21.15.6:0/2261397924 >> v1:172.21.15.6:6789/0 conn(0x7f8e8c0b1a00 legacy=0x7f8e8c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e91e8e700 1 -- 172.21.15.6:0/2261397924 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c0cedf0 con 0x7f8e8c06afc0 2023-11-28T13:01:08.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.613+0000 7f8e83fff700 1 -- 172.21.15.6:0/2261397924 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567429348 0 0) 0x7f8e8c0ce550 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e9168d700 1 --2- 172.21.15.6:0/2261397924 >> v2:172.21.15.61:3300/0 conn(0x7f8e8c0b1630 0x7f8e8c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:08.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e91e8e700 1 --2- 172.21.15.6:0/2261397924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c06afc0 0x7f8e8c06b420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009d90 tx=0x7f8e8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130f61ee6190330 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e83fff700 1 -- 172.21.15.6:0/2261397924 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8400e070 con 0x7f8e8c06afc0 2023-11-28T13:01:08.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e83fff700 1 -- 172.21.15.6:0/2261397924 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8400bb90 con 0x7f8e8c06afc0 2023-11-28T13:01:08.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e83fff700 1 -- 172.21.15.6:0/2261397924 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84012430 con 0x7f8e8c06afc0 2023-11-28T13:01:08.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e938f1700 1 -- 172.21.15.6:0/2261397924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c06afc0 msgr2=0x7f8e8c06b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e938f1700 1 --2- 172.21.15.6:0/2261397924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c06afc0 0x7f8e8c06b420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8e84009d90 tx=0x7f8e8400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:08.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e938f1700 1 -- 172.21.15.6:0/2261397924 shutdown_connections 2023-11-28T13:01:08.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e938f1700 1 --2- 172.21.15.6:0/2261397924 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c06afc0 0x7f8e8c06b420 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e938f1700 1 --2- 172.21.15.6:0/2261397924 >> v2:172.21.15.61:3300/0 conn(0x7f8e8c0b1630 0x7f8e8c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e938f1700 1 -- 172.21.15.6:0/2261397924 >> 172.21.15.6:0/2261397924 conn(0x7f8e8c0cc630 msgr2=0x7f8e8c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e938f1700 1 -- 172.21.15.6:0/2261397924 shutdown_connections 2023-11-28T13:01:08.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.614+0000 7f8e938f1700 1 -- 172.21.15.6:0/2261397924 wait complete. 2023-11-28T13:01:08.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.616+0000 7f8e938f1700 1 Processor -- start 2023-11-28T13:01:08.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.616+0000 7f8e938f1700 1 -- start start 2023-11-28T13:01:08.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.616+0000 7f8e938f1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8e8c06afc0 0x7f8e8c0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.616+0000 7f8e938f1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8e8c0b1630 0x7f8e8c15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.616+0000 7f8e938f1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c0b1a00 0x7f8e8c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.616+0000 7f8e938f1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c15c5b0 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.616+0000 7f8e938f1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0b30e0 con 0x7f8e8c06afc0 2023-11-28T13:01:08.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.616+0000 7f8e938f1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c0b33c0 con 0x7f8e8c0b1630 2023-11-28T13:01:08.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.616+0000 7f8e91e8e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c0b1a00 0x7f8e8c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e91e8e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c0b1a00 0x7f8e8c0b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59160/0 (socket says 172.21.15.6:59160) 2023-11-28T13:01:08.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e91e8e700 1 -- 172.21.15.6:0/3524277147 learned_addr learned my addr 172.21.15.6:0/3524277147 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:08.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e9168d700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8e8c06afc0 0x7f8e8c0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e90e8c700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8e8c0b1630 0x7f8e8c15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e91e8e700 1 -- 172.21.15.6:0/3524277147 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8e8c0b1630 msgr2=0x7f8e8c15b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e91e8e700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8e8c0b1630 0x7f8e8c15b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e91e8e700 1 -- 172.21.15.6:0/3524277147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8e8c06afc0 msgr2=0x7f8e8c0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e91e8e700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8e8c06afc0 0x7f8e8c0b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e91e8e700 1 -- 172.21.15.6:0/3524277147 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e84009a70 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e9168d700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8e8c06afc0 0x7f8e8c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:08.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e91e8e700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c0b1a00 0x7f8e8c0b4f30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8e84014040 tx=0x7f8e84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e81ffb700 1 -- 172.21.15.6:0/3524277147 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84012430 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e81ffb700 1 -- 172.21.15.6:0/3524277147 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e84020070 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e938f1700 1 -- 172.21.15.6:0/3524277147 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8c0b1ea0 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e81ffb700 1 -- 172.21.15.6:0/3524277147 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84004280 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.617+0000 7f8e938f1700 1 -- 172.21.15.6:0/3524277147 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8c0cf0d0 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.618+0000 7f8e938f1700 1 -- 172.21.15.6:0/3524277147 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8c0cb5b0 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.619+0000 7f8e81ffb700 1 -- 172.21.15.6:0/3524277147 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8e840043e0 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.619+0000 7f8e81ffb700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8e700682f0 0x7f8e7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.620+0000 7f8e81ffb700 1 -- 172.21.15.6:0/3524277147 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8e84022070 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.620+0000 7f8e9168d700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8e700682f0 0x7f8e7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.620+0000 7f8e9168d700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8e700682f0 0x7f8e7006a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009a00 tx=0x7f8e7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.623+0000 7f8e81ffb700 1 -- 172.21.15.6:0/3524277147 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8e84054c60 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.816+0000 7f8e938f1700 1 -- 172.21.15.6:0/3524277147 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8e8c0739f0 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.817+0000 7f8e81ffb700 1 -- 172.21.15.6:0/3524277147 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8e84058280 con 0x7f8e8c0b1a00 2023-11-28T13:01:08.818 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:08.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 -- 172.21.15.6:0/3524277147 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8e700682f0 msgr2=0x7f8e7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8e700682f0 0x7f8e7006a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009a00 tx=0x7f8e7c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 -- 172.21.15.6:0/3524277147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c0b1a00 msgr2=0x7f8e8c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c0b1a00 0x7f8e8c0b4f30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8e84014040 tx=0x7f8e84004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 -- 172.21.15.6:0/3524277147 shutdown_connections 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8e8c0b1a00 0x7f8e8c0b4f30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8e8c06afc0 0x7f8e8c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8e700682f0 0x7f8e7006a7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 --2- 172.21.15.6:0/3524277147 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8e8c0b1630 0x7f8e8c15b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 -- 172.21.15.6:0/3524277147 >> 172.21.15.6:0/3524277147 conn(0x7f8e8c0cc630 msgr2=0x7f8e8c06b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 -- 172.21.15.6:0/3524277147 shutdown_connections 2023-11-28T13:01:08.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.819+0000 7f8e938f1700 1 -- 172.21.15.6:0/3524277147 wait complete. 2023-11-28T13:01:08.833 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:08.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.959+0000 7f2d108c7700 1 Processor -- start 2023-11-28T13:01:08.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.959+0000 7f2d108c7700 1 -- start start 2023-11-28T13:01:08.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d108c7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c0b1630 0x7f2d0c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d108c7700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c0ce550 con 0x7f2d0c06afc0 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d108c7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c0ce830 con 0x7f2d0c0b1a00 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d108c7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0ceb10 con 0x7f2d0c0b1630 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d0ad9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f2d0c06afc0 0x7f2d0c06b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46488/0 (socket says 172.21.15.6:46488) 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d0ad9d700 1 -- 172.21.15.6:0/383793696 learned_addr learned my addr 172.21.15.6:0/383793696 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d0a59c700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c0b1630 0x7f2d0c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538273751 0 0) 0x7f2d0c0ce550 con 0x7f2d0c06afc0 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf40035b0 con 0x7f2d0c06afc0 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4195820463 0 0) 0x7f2cf40035b0 con 0x7f2d0c06afc0 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c0b1630 msgr2=0x7f2d0c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d0959a700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c0b1630 0x7f2d0c0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 >> v1:172.21.15.61:6789/0 conn(0x7f2d0c0b1a00 legacy=0x7f2d0c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.960+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0cedf0 con 0x7f2d0c06afc0 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f2d040019f0 con 0x7f2d0c06afc0 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 >> v1:172.21.15.6:6789/0 conn(0x7f2d0c06afc0 legacy=0x7f2d0c06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 0x7f2cf4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c0b1630 0x7f2cf4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2cf4004d90 0x7f2cf4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0ceb10 con 0x7f2cf4003a30 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf400ba10 con 0x7f2cf4004d90 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf400bba0 con 0x7f2d0c0b1630 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0a59c700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 0x7f2cf4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d09d9b700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c0b1630 0x7f2cf4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0ad9d700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2cf4004d90 0x7f2cf4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0a59c700 1 -- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c0b1630 msgr2=0x7f2cf4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0a59c700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c0b1630 0x7f2cf4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0a59c700 1 -- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2cf4004d90 msgr2=0x7f2cf4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0a59c700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2cf4004d90 0x7f2cf4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0a59c700 1 -- 172.21.15.6:0/383793696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40035b0 con 0x7f2cf4003a30 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d09d9b700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c0b1630 0x7f2cf4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0ad9d700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2cf4004d90 0x7f2cf4005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0a59c700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 0x7f2cf4003e00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009b60 tx=0x7f2cf4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ee9509973e4dca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00e6c0 con 0x7f2cf4003a30 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc00e820 con 0x7f2cf4003a30 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.961+0000 7f2d0959a700 1 -- 172.21.15.6:0/383793696 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00eb90 con 0x7f2cf4003a30 2023-11-28T13:01:08.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.962+0000 7f2d108c7700 1 -- 172.21.15.6:0/383793696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 msgr2=0x7f2cf4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.962+0000 7f2d108c7700 1 --2- 172.21.15.6:0/383793696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 0x7f2cf4003e00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009b60 tx=0x7f2cf4005880 comp rx=0 tx=0).stop 2023-11-28T13:01:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.962+0000 7f2d0ad9d700 1 -- 172.21.15.6:0/383793696 reap_dead start 2023-11-28T13:01:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.962+0000 7f2d108c7700 1 -- 172.21.15.6:0/383793696 shutdown_connections 2023-11-28T13:01:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.962+0000 7f2d108c7700 1 -- 172.21.15.6:0/383793696 >> 172.21.15.6:0/383793696 conn(0x7f2d0c0cc630 msgr2=0x7f2d0c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.962+0000 7f2d108c7700 1 -- 172.21.15.6:0/383793696 shutdown_connections 2023-11-28T13:01:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.962+0000 7f2d108c7700 1 -- 172.21.15.6:0/383793696 wait complete. 2023-11-28T13:01:08.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.964+0000 7f2d108c7700 1 Processor -- start 2023-11-28T13:01:08.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.964+0000 7f2d108c7700 1 -- start start 2023-11-28T13:01:08.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.964+0000 7f2d108c7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 0x7f2d0c148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.964+0000 7f2d108c7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2d0c149cf0 0x7f2d0c155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.964+0000 7f2d108c7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c147050 0x7f2d0c1490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d108c7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c148640 con 0x7f2cf4003a30 2023-11-28T13:01:08.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d108c7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1479c0 con 0x7f2d0c149cf0 2023-11-28T13:01:08.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d108c7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c147ca0 con 0x7f2d0c147050 2023-11-28T13:01:08.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 0x7f2d0c148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 0x7f2d0c148a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59176/0 (socket says 172.21.15.6:59176) 2023-11-28T13:01:08.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0a59c700 1 -- 172.21.15.6:0/3793421235 learned_addr learned my addr 172.21.15.6:0/3793421235 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0ad9d700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c147050 0x7f2d0c1490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d09d9b700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2d0c149cf0 0x7f2d0c155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0a59c700 1 -- 172.21.15.6:0/3793421235 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c147050 msgr2=0x7f2d0c1490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0a59c700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c147050 0x7f2d0c1490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0a59c700 1 -- 172.21.15.6:0/3793421235 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2d0c149cf0 msgr2=0x7f2d0c155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0a59c700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2d0c149cf0 0x7f2d0c155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0a59c700 1 -- 172.21.15.6:0/3793421235 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c0b1e70 con 0x7f2cf4003a30 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0ad9d700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c147050 0x7f2d0c1490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d0a59c700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 0x7f2d0c148a60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0090b0 tx=0x7f2cfc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2cfaffd700 1 -- 172.21.15.6:0/3793421235 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc021070 con 0x7f2cf4003a30 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2cfaffd700 1 -- 172.21.15.6:0/3793421235 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cfc0036a0 con 0x7f2cf4003a30 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2cfaffd700 1 -- 172.21.15.6:0/3793421235 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00e6c0 con 0x7f2cf4003a30 2023-11-28T13:01:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d108c7700 1 -- 172.21.15.6:0/3793421235 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c0cf0d0 con 0x7f2cf4003a30 2023-11-28T13:01:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.965+0000 7f2d108c7700 1 -- 172.21.15.6:0/3793421235 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c0cb5b0 con 0x7f2cf4003a30 2023-11-28T13:01:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.966+0000 7f2d108c7700 1 -- 172.21.15.6:0/3793421235 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c068570 con 0x7f2cf4003a30 2023-11-28T13:01:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.967+0000 7f2cfaffd700 1 -- 172.21.15.6:0/3793421235 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2cfc019030 con 0x7f2cf4003a30 2023-11-28T13:01:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.967+0000 7f2cfaffd700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2cec0682f0 0x7f2cec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.968+0000 7f2d09d9b700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2cec0682f0 0x7f2cec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.968+0000 7f2cfaffd700 1 -- 172.21.15.6:0/3793421235 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2cfc086f90 con 0x7f2cf4003a30 2023-11-28T13:01:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.968+0000 7f2d09d9b700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2cec0682f0 0x7f2cec06a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c14a0c0 tx=0x7f2d00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:08.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:08.972+0000 7f2cfaffd700 1 -- 172.21.15.6:0/3793421235 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2cfc053500 con 0x7f2cf4003a30 2023-11-28T13:01:09.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.170+0000 7f2d108c7700 1 -- 172.21.15.6:0/3793421235 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d0c0739f0 con 0x7f2cf4003a30 2023-11-28T13:01:09.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.170+0000 7f2cfaffd700 1 -- 172.21.15.6:0/3793421235 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cfc0170a0 con 0x7f2cf4003a30 2023-11-28T13:01:09.215 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 -- 172.21.15.6:0/3793421235 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2cec0682f0 msgr2=0x7f2cec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2cec0682f0 0x7f2cec06a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c14a0c0 tx=0x7f2d00009040 comp rx=0 tx=0).stop 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 -- 172.21.15.6:0/3793421235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 msgr2=0x7f2d0c148a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 0x7f2d0c148a60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0090b0 tx=0x7f2cfc0047c0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 -- 172.21.15.6:0/3793421235 shutdown_connections 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2cec0682f0 0x7f2cec06a7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2d0c147050 0x7f2d0c1490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cf4003a30 0x7f2d0c148a60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 --2- 172.21.15.6:0/3793421235 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2d0c149cf0 0x7f2d0c155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 -- 172.21.15.6:0/3793421235 >> 172.21.15.6:0/3793421235 conn(0x7f2d0c0cc630 msgr2=0x7f2d0c15c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 -- 172.21.15.6:0/3793421235 shutdown_connections 2023-11-28T13:01:09.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.172+0000 7f2d108c7700 1 -- 172.21.15.6:0/3793421235 wait complete. 2023-11-28T13:01:09.218 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:09.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.344+0000 7fcb460a1700 1 Processor -- start 2023-11-28T13:01:09.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.344+0000 7fcb460a1700 1 -- start start 2023-11-28T13:01:09.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.345+0000 7fcb460a1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcb40159500 0x7fcb400cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.345+0000 7fcb460a1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b1d40 0x7fcb400ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.345+0000 7fcb460a1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4015e040 con 0x7fcb4015dbf0 2023-11-28T13:01:09.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.345+0000 7fcb460a1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb400cede0 con 0x7fcb40159500 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.345+0000 7fcb460a1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400ceff0 con 0x7fcb400b1d40 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b1d40 0x7fcb400ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3f7fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcb40159500 0x7fcb400cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3ffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b1d40 0x7fcb400ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:48896/0 (socket says 172.21.15.6:48896) 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3f7fe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcb40159500 0x7fcb400cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:34674/0 (socket says 172.21.15.6:34674) 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3ffff700 1 -- 172.21.15.6:0/1703409584 learned_addr learned my addr 172.21.15.6:0/1703409584 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3effd700 1 --1- 172.21.15.6:0/1703409584 >> v1:172.21.15.61:6789/0 conn(0x7fcb4015dbf0 0x7fcb400b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:36538/0 (socket says 172.21.15.6:36538) 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3ffff700 1 -- 172.21.15.6:0/1703409584 >> v2:172.21.15.61:3300/0 conn(0x7fcb40159500 msgr2=0x7fcb400cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3ffff700 1 --2- 172.21.15.6:0/1703409584 >> v2:172.21.15.61:3300/0 conn(0x7fcb40159500 0x7fcb400cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3ffff700 1 -- 172.21.15.6:0/1703409584 >> v1:172.21.15.61:6789/0 conn(0x7fcb4015dbf0 legacy=0x7fcb400b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3ffff700 1 -- 172.21.15.6:0/1703409584 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb400cf2d0 con 0x7fcb400b1d40 2023-11-28T13:01:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3f7fe700 1 --2- 172.21.15.6:0/1703409584 >> v2:172.21.15.61:3300/0 conn(0x7fcb40159500 0x7fcb400cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3ffff700 1 --2- 172.21.15.6:0/1703409584 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b1d40 0x7fcb400ce8a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009e50 tx=0x7fcb3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa0ca4559fe6255 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3e7fc700 1 -- 172.21.15.6:0/1703409584 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800e070 con 0x7fcb400b1d40 2023-11-28T13:01:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3e7fc700 1 -- 172.21.15.6:0/1703409584 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3800b910 con 0x7fcb400b1d40 2023-11-28T13:01:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.346+0000 7fcb3e7fc700 1 -- 172.21.15.6:0/1703409584 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800bcc0 con 0x7fcb400b1d40 2023-11-28T13:01:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.347+0000 7fcb460a1700 1 -- 172.21.15.6:0/1703409584 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b1d40 msgr2=0x7fcb400ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.347+0000 7fcb460a1700 1 --2- 172.21.15.6:0/1703409584 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b1d40 0x7fcb400ce8a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009e50 tx=0x7fcb3800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.347+0000 7fcb460a1700 1 -- 172.21.15.6:0/1703409584 shutdown_connections 2023-11-28T13:01:09.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.347+0000 7fcb460a1700 1 --2- 172.21.15.6:0/1703409584 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb400b1d40 0x7fcb400ce8a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.347+0000 7fcb460a1700 1 --2- 172.21.15.6:0/1703409584 >> v2:172.21.15.61:3300/0 conn(0x7fcb40159500 0x7fcb400cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.347+0000 7fcb460a1700 1 -- 172.21.15.6:0/1703409584 >> 172.21.15.6:0/1703409584 conn(0x7fcb400cc630 msgr2=0x7fcb400cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.347+0000 7fcb460a1700 1 -- 172.21.15.6:0/1703409584 shutdown_connections 2023-11-28T13:01:09.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.347+0000 7fcb460a1700 1 -- 172.21.15.6:0/1703409584 wait complete. 2023-11-28T13:01:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.348+0000 7fcb460a1700 1 Processor -- start 2023-11-28T13:01:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.348+0000 7fcb460a1700 1 -- start start 2023-11-28T13:01:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb460a1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b1d40 0x7fcb40153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb460a1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb40159500 0x7fcb400c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb460a1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb4015dbf0 0x7fcb401f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb460a1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb40153ca0 con 0x7fcb4015dbf0 2023-11-28T13:01:09.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb460a1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400c47d0 con 0x7fcb400b1d40 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb460a1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400c4ab0 con 0x7fcb40159500 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb3ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb4015dbf0 0x7fcb401f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb3ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb4015dbf0 0x7fcb401f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59198/0 (socket says 172.21.15.6:59198) 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb3ffff700 1 -- 172.21.15.6:0/834625536 learned_addr learned my addr 172.21.15.6:0/834625536 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb3effd700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb40159500 0x7fcb400c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb3f7fe700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b1d40 0x7fcb40153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb3ffff700 1 -- 172.21.15.6:0/834625536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb40159500 msgr2=0x7fcb400c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb3ffff700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb40159500 0x7fcb400c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb3ffff700 1 -- 172.21.15.6:0/834625536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b1d40 msgr2=0x7fcb40153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb3ffff700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b1d40 0x7fcb40153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.349+0000 7fcb3ffff700 1 -- 172.21.15.6:0/834625536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb300094a0 con 0x7fcb4015dbf0 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.350+0000 7fcb3ffff700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb4015dbf0 0x7fcb401f02a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcb4015e1d0 tx=0x7fcb38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.350+0000 7fcb27fff700 1 -- 172.21.15.6:0/834625536 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38004960 con 0x7fcb4015dbf0 2023-11-28T13:01:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.350+0000 7fcb27fff700 1 -- 172.21.15.6:0/834625536 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb380099b0 con 0x7fcb4015dbf0 2023-11-28T13:01:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.350+0000 7fcb460a1700 1 -- 172.21.15.6:0/834625536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb38009b30 con 0x7fcb4015dbf0 2023-11-28T13:01:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.350+0000 7fcb27fff700 1 -- 172.21.15.6:0/834625536 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3801b670 con 0x7fcb4015dbf0 2023-11-28T13:01:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.350+0000 7fcb460a1700 1 -- 172.21.15.6:0/834625536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4015d3f0 con 0x7fcb4015dbf0 2023-11-28T13:01:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.351+0000 7fcb460a1700 1 -- 172.21.15.6:0/834625536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4015d110 con 0x7fcb4015dbf0 2023-11-28T13:01:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.351+0000 7fcb27fff700 1 -- 172.21.15.6:0/834625536 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcb3802b070 con 0x7fcb4015dbf0 2023-11-28T13:01:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.351+0000 7fcb27fff700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb2c0682f0 0x7fcb2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.352+0000 7fcb27fff700 1 -- 172.21.15.6:0/834625536 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcb38087850 con 0x7fcb4015dbf0 2023-11-28T13:01:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.352+0000 7fcb3f7fe700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb2c0682f0 0x7fcb2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.352+0000 7fcb3f7fe700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb2c0682f0 0x7fcb2c06a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb400c6f10 tx=0x7fcb30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.357 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.356+0000 7fcb27fff700 1 -- 172.21.15.6:0/834625536 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcb38053d80 con 0x7fcb4015dbf0 2023-11-28T13:01:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.551+0000 7fcb460a1700 1 -- 172.21.15.6:0/834625536 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb4015d6d0 con 0x7fcb4015dbf0 2023-11-28T13:01:09.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.552+0000 7fcb27fff700 1 -- 172.21.15.6:0/834625536 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb380573a0 con 0x7fcb4015dbf0 2023-11-28T13:01:09.553 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 -- 172.21.15.6:0/834625536 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb2c0682f0 msgr2=0x7fcb2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb2c0682f0 0x7fcb2c06a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb400c6f10 tx=0x7fcb30008040 comp rx=0 tx=0).stop 2023-11-28T13:01:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 -- 172.21.15.6:0/834625536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb4015dbf0 msgr2=0x7fcb401f02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb4015dbf0 0x7fcb401f02a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcb4015e1d0 tx=0x7fcb38004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 -- 172.21.15.6:0/834625536 shutdown_connections 2023-11-28T13:01:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb4015dbf0 0x7fcb401f02a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb400b1d40 0x7fcb40153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb2c0682f0 0x7fcb2c06a7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 --2- 172.21.15.6:0/834625536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb40159500 0x7fcb400c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 -- 172.21.15.6:0/834625536 >> 172.21.15.6:0/834625536 conn(0x7fcb400cc630 msgr2=0x7fcb400cafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 -- 172.21.15.6:0/834625536 shutdown_connections 2023-11-28T13:01:09.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.554+0000 7fcb460a1700 1 -- 172.21.15.6:0/834625536 wait complete. 2023-11-28T13:01:09.568 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:09.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.694+0000 7f5c7b12b700 1 Processor -- start 2023-11-28T13:01:09.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.695+0000 7f5c7b12b700 1 -- start start 2023-11-28T13:01:09.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.695+0000 7f5c7b12b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5c740cf5f0 0x7f5c740caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.695+0000 7f5c7b12b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5c740cb3e0 0x7f5c7406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.695+0000 7f5c7b12b700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c740ce550 con 0x7f5c740b1630 2023-11-28T13:01:09.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.695+0000 7f5c7b12b700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c740ce830 con 0x7f5c740cf5f0 2023-11-28T13:01:09.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.695+0000 7f5c7b12b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c740ceb10 con 0x7f5c740cb3e0 2023-11-28T13:01:09.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c73fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5c740cf5f0 0x7f5c740caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c73fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5c740cf5f0 0x7f5c740caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59204/0 (socket says 172.21.15.6:59204) 2023-11-28T13:01:09.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c73fff700 1 -- 172.21.15.6:0/2523963845 learned_addr learned my addr 172.21.15.6:0/2523963845 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:09.701 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c78ec7700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f5c740b1630 0x7f5c740b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46522/0 (socket says 172.21.15.6:46522) 2023-11-28T13:01:09.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c796c8700 1 --2- 172.21.15.6:0/2523963845 >> v2:172.21.15.61:3300/0 conn(0x7f5c740cb3e0 0x7f5c7406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c73fff700 1 -- 172.21.15.6:0/2523963845 >> v2:172.21.15.61:3300/0 conn(0x7f5c740cb3e0 msgr2=0x7f5c7406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:09.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c73fff700 1 --2- 172.21.15.6:0/2523963845 >> v2:172.21.15.61:3300/0 conn(0x7f5c740cb3e0 0x7f5c7406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c73fff700 1 -- 172.21.15.6:0/2523963845 >> v1:172.21.15.6:6789/0 conn(0x7f5c740b1630 legacy=0x7f5c740b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c73fff700 1 -- 172.21.15.6:0/2523963845 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c740cedf0 con 0x7f5c740cf5f0 2023-11-28T13:01:09.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c737fe700 1 -- 172.21.15.6:0/2523963845 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125456942 0 0) 0x7f5c740ce550 con 0x7f5c740b1630 2023-11-28T13:01:09.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c796c8700 1 --2- 172.21.15.6:0/2523963845 >> v2:172.21.15.61:3300/0 conn(0x7f5c740cb3e0 0x7f5c7406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:09.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c73fff700 1 --2- 172.21.15.6:0/2523963845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740cf5f0 0x7f5c740caea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5c600098f0 tx=0x7f5c60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f6296eb85c85d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c737fe700 1 -- 172.21.15.6:0/2523963845 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000b860 con 0x7f5c740cf5f0 2023-11-28T13:01:09.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c737fe700 1 -- 172.21.15.6:0/2523963845 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6000b9c0 con 0x7f5c740cf5f0 2023-11-28T13:01:09.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.696+0000 7f5c737fe700 1 -- 172.21.15.6:0/2523963845 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000bd50 con 0x7f5c740cf5f0 2023-11-28T13:01:09.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.697+0000 7f5c7b12b700 1 -- 172.21.15.6:0/2523963845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740cf5f0 msgr2=0x7f5c740caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.697+0000 7f5c7b12b700 1 --2- 172.21.15.6:0/2523963845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740cf5f0 0x7f5c740caea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5c600098f0 tx=0x7f5c60009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:09.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.697+0000 7f5c7b12b700 1 -- 172.21.15.6:0/2523963845 shutdown_connections 2023-11-28T13:01:09.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.697+0000 7f5c7b12b700 1 --2- 172.21.15.6:0/2523963845 >> v2:172.21.15.61:3300/0 conn(0x7f5c740cb3e0 0x7f5c7406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.697+0000 7f5c7b12b700 1 --2- 172.21.15.6:0/2523963845 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740cf5f0 0x7f5c740caea0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.697+0000 7f5c7b12b700 1 -- 172.21.15.6:0/2523963845 >> 172.21.15.6:0/2523963845 conn(0x7f5c740cc630 msgr2=0x7f5c740cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:09.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.697+0000 7f5c7b12b700 1 -- 172.21.15.6:0/2523963845 shutdown_connections 2023-11-28T13:01:09.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.697+0000 7f5c7b12b700 1 -- 172.21.15.6:0/2523963845 wait complete. 2023-11-28T13:01:09.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.698+0000 7f5c7b12b700 1 Processor -- start 2023-11-28T13:01:09.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.698+0000 7f5c7b12b700 1 -- start start 2023-11-28T13:01:09.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c7b12b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740b1630 0x7f5c740c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c7b12b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c740cb3e0 0x7f5c7414bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c7b12b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c740cf5f0 0x7f5c740ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c7b12b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7414c530 con 0x7f5c740b1630 2023-11-28T13:01:09.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c7b12b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740c7f00 con 0x7f5c740cb3e0 2023-11-28T13:01:09.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c7b12b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740c81e0 con 0x7f5c740cf5f0 2023-11-28T13:01:09.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c78ec7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740b1630 0x7f5c740c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c78ec7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740b1630 0x7f5c740c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59210/0 (socket says 172.21.15.6:59210) 2023-11-28T13:01:09.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c78ec7700 1 -- 172.21.15.6:0/1715043290 learned_addr learned my addr 172.21.15.6:0/1715043290 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:09.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c73fff700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c740cb3e0 0x7f5c7414bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c796c8700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c740cf5f0 0x7f5c740ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c78ec7700 1 -- 172.21.15.6:0/1715043290 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c740cf5f0 msgr2=0x7f5c740ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c78ec7700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c740cf5f0 0x7f5c740ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c78ec7700 1 -- 172.21.15.6:0/1715043290 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c740cb3e0 msgr2=0x7f5c7414bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c78ec7700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c740cb3e0 0x7f5c7414bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c78ec7700 1 -- 172.21.15.6:0/1715043290 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c60009550 con 0x7f5c740b1630 2023-11-28T13:01:09.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c73fff700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c740cb3e0 0x7f5c7414bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:09.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c796c8700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c740cf5f0 0x7f5c740ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:09.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.699+0000 7f5c78ec7700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740b1630 0x7f5c740c78b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5c680011e0 tx=0x7f5c6800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.700+0000 7f5c717fa700 1 -- 172.21.15.6:0/1715043290 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68004c80 con 0x7f5c740b1630 2023-11-28T13:01:09.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.700+0000 7f5c717fa700 1 -- 172.21.15.6:0/1715043290 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c68004e00 con 0x7f5c740b1630 2023-11-28T13:01:09.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.700+0000 7f5c7b12b700 1 -- 172.21.15.6:0/1715043290 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c740ce140 con 0x7f5c740b1630 2023-11-28T13:01:09.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.700+0000 7f5c717fa700 1 -- 172.21.15.6:0/1715043290 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68007310 con 0x7f5c740b1630 2023-11-28T13:01:09.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.700+0000 7f5c7b12b700 1 -- 172.21.15.6:0/1715043290 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c740cf0d0 con 0x7f5c740b1630 2023-11-28T13:01:09.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.701+0000 7f5c7b12b700 1 -- 172.21.15.6:0/1715043290 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c740ce2d0 con 0x7f5c740b1630 2023-11-28T13:01:09.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.701+0000 7f5c717fa700 1 -- 172.21.15.6:0/1715043290 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c6801a020 con 0x7f5c740b1630 2023-11-28T13:01:09.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.701+0000 7f5c717fa700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c580682f0 0x7f5c5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:09.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.701+0000 7f5c717fa700 1 -- 172.21.15.6:0/1715043290 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5c68085c40 con 0x7f5c740b1630 2023-11-28T13:01:09.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.702+0000 7f5c73fff700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c580682f0 0x7f5c5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:09.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.702+0000 7f5c73fff700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c580682f0 0x7f5c5806a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c740d09e0 tx=0x7f5c60008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:09.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.706+0000 7f5c717fa700 1 -- 172.21.15.6:0/1715043290 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c68052170 con 0x7f5c740b1630 2023-11-28T13:01:09.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.902+0000 7f5c7b12b700 1 -- 172.21.15.6:0/1715043290 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c740739f0 con 0x7f5c740b1630 2023-11-28T13:01:09.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.902+0000 7f5c717fa700 1 -- 172.21.15.6:0/1715043290 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c68055790 con 0x7f5c740b1630 2023-11-28T13:01:09.904 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:09.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.904+0000 7f5c7b12b700 1 -- 172.21.15.6:0/1715043290 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c580682f0 msgr2=0x7f5c5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.904+0000 7f5c7b12b700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c580682f0 0x7f5c5806a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c740d09e0 tx=0x7f5c60008030 comp rx=0 tx=0).stop 2023-11-28T13:01:09.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.904+0000 7f5c7b12b700 1 -- 172.21.15.6:0/1715043290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740b1630 msgr2=0x7f5c740c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:09.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.904+0000 7f5c7b12b700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740b1630 0x7f5c740c78b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5c680011e0 tx=0x7f5c6800cae0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.905+0000 7f5c7b12b700 1 -- 172.21.15.6:0/1715043290 shutdown_connections 2023-11-28T13:01:09.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.905+0000 7f5c7b12b700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c580682f0 0x7f5c5806a7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.905+0000 7f5c7b12b700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5c740cf5f0 0x7f5c740ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.905+0000 7f5c7b12b700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c740b1630 0x7f5c740c78b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.905+0000 7f5c7b12b700 1 --2- 172.21.15.6:0/1715043290 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c740cb3e0 0x7f5c7414bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:09.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.905+0000 7f5c7b12b700 1 -- 172.21.15.6:0/1715043290 >> 172.21.15.6:0/1715043290 conn(0x7f5c740cc630 msgr2=0x7f5c7406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:09.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.905+0000 7f5c7b12b700 1 -- 172.21.15.6:0/1715043290 shutdown_connections 2023-11-28T13:01:09.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:09.905+0000 7f5c7b12b700 1 -- 172.21.15.6:0/1715043290 wait complete. 2023-11-28T13:01:09.918 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:10.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.045+0000 7fd636f32700 1 Processor -- start 2023-11-28T13:01:10.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.045+0000 7fd636f32700 1 -- start start 2023-11-28T13:01:10.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.046+0000 7fd636f32700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 0x7fd6300cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.046+0000 7fd636f32700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd63006a1e0 0x7fd63006a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.046+0000 7fd636f32700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63006bb90 con 0x7fd63006ab90 2023-11-28T13:01:10.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.046+0000 7fd636f32700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6300cd920 con 0x7fd63006a1e0 2023-11-28T13:01:10.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.046+0000 7fd636f32700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6300cfa50 con 0x7fd63015fa10 2023-11-28T13:01:10.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.046+0000 7fd62ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd63006a1e0 0x7fd63006a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.046+0000 7fd634cce700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 0x7fd6300cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.046+0000 7fd6354cf700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fd63006ab90 0x7fd6301600c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:36568/0 (socket says 172.21.15.6:36568) 2023-11-28T13:01:10.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.046+0000 7fd6354cf700 1 -- 172.21.15.6:0/2487653348 learned_addr learned my addr 172.21.15.6:0/2487653348 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:10.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.046+0000 7fd62ffff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd63006a1e0 0x7fd63006a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:34702/0 (socket says 172.21.15.6:34702) 2023-11-28T13:01:10.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.047+0000 7fd634cce700 1 -- 172.21.15.6:0/2487653348 >> v2:172.21.15.61:3300/0 conn(0x7fd63006a1e0 msgr2=0x7fd63006a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.047+0000 7fd634cce700 1 --2- 172.21.15.6:0/2487653348 >> v2:172.21.15.61:3300/0 conn(0x7fd63006a1e0 0x7fd63006a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.047+0000 7fd634cce700 1 -- 172.21.15.6:0/2487653348 >> v1:172.21.15.61:6789/0 conn(0x7fd63006ab90 legacy=0x7fd6301600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.047+0000 7fd634cce700 1 -- 172.21.15.6:0/2487653348 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63006b010 con 0x7fd63015fa10 2023-11-28T13:01:10.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.047+0000 7fd62ffff700 1 --2- 172.21.15.6:0/2487653348 >> v2:172.21.15.61:3300/0 conn(0x7fd63006a1e0 0x7fd63006a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:10.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.047+0000 7fd634cce700 1 --2- 172.21.15.6:0/2487653348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 0x7fd6300cd3e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd624009840 tx=0x7fd624009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923ec039e283a052 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.047+0000 7fd62f7fe700 1 -- 172.21.15.6:0/2487653348 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624011830 con 0x7fd63015fa10 2023-11-28T13:01:10.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.047+0000 7fd62f7fe700 1 -- 172.21.15.6:0/2487653348 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd624011990 con 0x7fd63015fa10 2023-11-28T13:01:10.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.047+0000 7fd62f7fe700 1 -- 172.21.15.6:0/2487653348 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624011d20 con 0x7fd63015fa10 2023-11-28T13:01:10.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.048+0000 7fd636f32700 1 -- 172.21.15.6:0/2487653348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 msgr2=0x7fd6300cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.048+0000 7fd636f32700 1 --2- 172.21.15.6:0/2487653348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 0x7fd6300cd3e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd624009840 tx=0x7fd624009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:10.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.048+0000 7fd636f32700 1 -- 172.21.15.6:0/2487653348 shutdown_connections 2023-11-28T13:01:10.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.048+0000 7fd636f32700 1 --2- 172.21.15.6:0/2487653348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 0x7fd6300cd3e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.048+0000 7fd636f32700 1 --2- 172.21.15.6:0/2487653348 >> v2:172.21.15.61:3300/0 conn(0x7fd63006a1e0 0x7fd63006a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.048+0000 7fd636f32700 1 -- 172.21.15.6:0/2487653348 >> 172.21.15.6:0/2487653348 conn(0x7fd63006ce40 msgr2=0x7fd6300b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.048+0000 7fd636f32700 1 -- 172.21.15.6:0/2487653348 shutdown_connections 2023-11-28T13:01:10.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.048+0000 7fd636f32700 1 -- 172.21.15.6:0/2487653348 wait complete. 2023-11-28T13:01:10.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.049+0000 7fd636f32700 1 Processor -- start 2023-11-28T13:01:10.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.049+0000 7fd636f32700 1 -- start start 2023-11-28T13:01:10.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd636f32700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63006a1e0 0x7fd63014db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd636f32700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd63006ab90 0x7fd6300b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd636f32700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 0x7fd630159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd636f32700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63015a110 con 0x7fd63006ab90 2023-11-28T13:01:10.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd636f32700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63015b930 con 0x7fd63006a1e0 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd636f32700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63015a340 con 0x7fd63015fa10 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd62ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd63006ab90 0x7fd6300b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd62ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd63006ab90 0x7fd6300b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59222/0 (socket says 172.21.15.6:59222) 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd62ffff700 1 -- 172.21.15.6:0/2080997329 learned_addr learned my addr 172.21.15.6:0/2080997329 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd6354cf700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 0x7fd630159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd634cce700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63006a1e0 0x7fd63014db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd62ffff700 1 -- 172.21.15.6:0/2080997329 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 msgr2=0x7fd630159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd62ffff700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 0x7fd630159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd62ffff700 1 -- 172.21.15.6:0/2080997329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63006a1e0 msgr2=0x7fd63014db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd62ffff700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63006a1e0 0x7fd63014db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd62ffff700 1 -- 172.21.15.6:0/2080997329 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c0094a0 con 0x7fd63006ab90 2023-11-28T13:01:10.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.050+0000 7fd62ffff700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd63006ab90 0x7fd6300b4e70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00ebe0 tx=0x7fd61c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.051+0000 7fd62d7fa700 1 -- 172.21.15.6:0/2080997329 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c009760 con 0x7fd63006ab90 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.051+0000 7fd62d7fa700 1 -- 172.21.15.6:0/2080997329 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c00c850 con 0x7fd63006ab90 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.051+0000 7fd636f32700 1 -- 172.21.15.6:0/2080997329 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6240094a0 con 0x7fd63006ab90 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.051+0000 7fd62d7fa700 1 -- 172.21.15.6:0/2080997329 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00cbe0 con 0x7fd63006ab90 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.051+0000 7fd636f32700 1 -- 172.21.15.6:0/2080997329 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6300cfca0 con 0x7fd63006ab90 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.052+0000 7fd636f32700 1 -- 172.21.15.6:0/2080997329 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6300d0ae0 con 0x7fd63006ab90 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.052+0000 7fd62d7fa700 1 -- 172.21.15.6:0/2080997329 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd61c019030 con 0x7fd63006ab90 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.052+0000 7fd62d7fa700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6140682f0 0x7fd61406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.052+0000 7fd62d7fa700 1 -- 172.21.15.6:0/2080997329 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd61c085f60 con 0x7fd63006ab90 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.052+0000 7fd634cce700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6140682f0 0x7fd61406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.053+0000 7fd634cce700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6140682f0 0x7fd61406a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd624000c00 tx=0x7fd62401d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.061+0000 7fd62d7fa700 1 -- 172.21.15.6:0/2080997329 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd61c052380 con 0x7fd63006ab90 2023-11-28T13:01:10.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.258+0000 7fd636f32700 1 -- 172.21.15.6:0/2080997329 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd6300cff80 con 0x7fd63006ab90 2023-11-28T13:01:10.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.258+0000 7fd62d7fa700 1 -- 172.21.15.6:0/2080997329 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd61c0210a0 con 0x7fd63006ab90 2023-11-28T13:01:10.259 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:10.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 -- 172.21.15.6:0/2080997329 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6140682f0 msgr2=0x7fd61406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6140682f0 0x7fd61406a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd624000c00 tx=0x7fd62401d040 comp rx=0 tx=0).stop 2023-11-28T13:01:10.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 -- 172.21.15.6:0/2080997329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd63006ab90 msgr2=0x7fd6300b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd63006ab90 0x7fd6300b4e70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00ebe0 tx=0x7fd61c00eef0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 -- 172.21.15.6:0/2080997329 shutdown_connections 2023-11-28T13:01:10.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6140682f0 0x7fd61406a7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd63015fa10 0x7fd630159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd63006a1e0 0x7fd63014db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 --2- 172.21.15.6:0/2080997329 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd63006ab90 0x7fd6300b4e70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 -- 172.21.15.6:0/2080997329 >> 172.21.15.6:0/2080997329 conn(0x7fd63006ce40 msgr2=0x7fd6300d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 -- 172.21.15.6:0/2080997329 shutdown_connections 2023-11-28T13:01:10.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.260+0000 7fd636f32700 1 -- 172.21.15.6:0/2080997329 wait complete. 2023-11-28T13:01:10.274 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:10.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.398+0000 7f875549e700 1 Processor -- start 2023-11-28T13:01:10.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.398+0000 7f875549e700 1 -- start start 2023-11-28T13:01:10.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.399+0000 7f875549e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f875015c640 0x7f87500cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.399+0000 7f875549e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87500d2820 0x7f87500b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.399+0000 7f875549e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87500d1470 con 0x7f875006bf30 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.399+0000 7f875549e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875015ca10 con 0x7f875015c640 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.399+0000 7f875549e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87500cd4e0 con 0x7f87500d2820 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874effd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f875015c640 0x7f87500cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874f7fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f875006bf30 0x7f875006c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:36580/0 (socket says 172.21.15.6:36580) 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874f7fe700 1 -- 172.21.15.6:0/366872962 learned_addr learned my addr 172.21.15.6:0/366872962 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874e7fc700 1 --2- 172.21.15.6:0/366872962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87500d2820 0x7f87500b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874effd700 1 -- 172.21.15.6:0/366872962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87500d2820 msgr2=0x7f87500b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874effd700 1 --2- 172.21.15.6:0/366872962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87500d2820 0x7f87500b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874effd700 1 -- 172.21.15.6:0/366872962 >> v1:172.21.15.61:6789/0 conn(0x7f875006bf30 legacy=0x7f875006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874effd700 1 -- 172.21.15.6:0/366872962 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87500cd7c0 con 0x7f875015c640 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874effd700 1 --2- 172.21.15.6:0/366872962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875015c640 0x7f87500cf040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8738009840 tx=0x7f8738009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a5e010e19cc908 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874dffb700 1 -- 172.21.15.6:0/366872962 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738011830 con 0x7f875015c640 2023-11-28T13:01:10.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874dffb700 1 -- 172.21.15.6:0/366872962 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8738011990 con 0x7f875015c640 2023-11-28T13:01:10.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.400+0000 7f874dffb700 1 -- 172.21.15.6:0/366872962 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738011d20 con 0x7f875015c640 2023-11-28T13:01:10.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.401+0000 7f875549e700 1 -- 172.21.15.6:0/366872962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875015c640 msgr2=0x7f87500cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.401+0000 7f875549e700 1 --2- 172.21.15.6:0/366872962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875015c640 0x7f87500cf040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8738009840 tx=0x7f8738009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:10.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.401+0000 7f875549e700 1 -- 172.21.15.6:0/366872962 shutdown_connections 2023-11-28T13:01:10.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.401+0000 7f875549e700 1 --2- 172.21.15.6:0/366872962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875015c640 0x7f87500cf040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.401+0000 7f875549e700 1 --2- 172.21.15.6:0/366872962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87500d2820 0x7f87500b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.401+0000 7f875549e700 1 -- 172.21.15.6:0/366872962 >> 172.21.15.6:0/366872962 conn(0x7f87500d2df0 msgr2=0x7f87500d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.401+0000 7f875549e700 1 -- 172.21.15.6:0/366872962 shutdown_connections 2023-11-28T13:01:10.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.401+0000 7f875549e700 1 -- 172.21.15.6:0/366872962 wait complete. 2023-11-28T13:01:10.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.402+0000 7f875549e700 1 Processor -- start 2023-11-28T13:01:10.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.402+0000 7f875549e700 1 -- start start 2023-11-28T13:01:10.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f875549e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875006bf30 0x7f87500c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f875549e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87500d2820 0x7f875014f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f875549e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f875015c640 0x7f87500c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f875549e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87501f5f90 con 0x7f87500d2820 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f875549e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87500c96e0 con 0x7f875006bf30 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f875549e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87501f5660 con 0x7f875015c640 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f874e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87500d2820 0x7f875014f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f874e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87500d2820 0x7f875014f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59234/0 (socket says 172.21.15.6:59234) 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f874e7fc700 1 -- 172.21.15.6:0/1934992000 learned_addr learned my addr 172.21.15.6:0/1934992000 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f874effd700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875006bf30 0x7f87500c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.403+0000 7f874f7fe700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f875015c640 0x7f87500c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f874e7fc700 1 -- 172.21.15.6:0/1934992000 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f875015c640 msgr2=0x7f87500c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f874e7fc700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f875015c640 0x7f87500c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f874e7fc700 1 -- 172.21.15.6:0/1934992000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875006bf30 msgr2=0x7f87500c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f874e7fc700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875006bf30 0x7f87500c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f874e7fc700 1 -- 172.21.15.6:0/1934992000 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87380094a0 con 0x7f87500d2820 2023-11-28T13:01:10.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f874e7fc700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87500d2820 0x7f875014f4e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f874000e9b0 tx=0x7f874000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f87477fe700 1 -- 172.21.15.6:0/1934992000 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740009710 con 0x7f87500d2820 2023-11-28T13:01:10.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f87477fe700 1 -- 172.21.15.6:0/1934992000 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874000ee50 con 0x7f87500d2820 2023-11-28T13:01:10.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f87477fe700 1 -- 172.21.15.6:0/1934992000 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874000ca50 con 0x7f87500d2820 2023-11-28T13:01:10.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f875549e700 1 -- 172.21.15.6:0/1934992000 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87500cf580 con 0x7f87500d2820 2023-11-28T13:01:10.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.404+0000 7f875549e700 1 -- 172.21.15.6:0/1934992000 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87500d1710 con 0x7f87500d2820 2023-11-28T13:01:10.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.405+0000 7f875549e700 1 -- 172.21.15.6:0/1934992000 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87500cdaa0 con 0x7f87500d2820 2023-11-28T13:01:10.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.405+0000 7f87477fe700 1 -- 172.21.15.6:0/1934992000 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8740019030 con 0x7f87500d2820 2023-11-28T13:01:10.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.406+0000 7f87477fe700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87300682f0 0x7f873006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.406+0000 7f87477fe700 1 -- 172.21.15.6:0/1934992000 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8740085e40 con 0x7f87500d2820 2023-11-28T13:01:10.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.406+0000 7f874effd700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87300682f0 0x7f873006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.406+0000 7f874effd700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87300682f0 0x7f873006a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8738000c00 tx=0x7f873801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.410+0000 7f87477fe700 1 -- 172.21.15.6:0/1934992000 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8740052370 con 0x7f87500d2820 2023-11-28T13:01:10.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.622+0000 7f875549e700 1 -- 172.21.15.6:0/1934992000 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8750068570 con 0x7f87500d2820 2023-11-28T13:01:10.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.623+0000 7f87477fe700 1 -- 172.21.15.6:0/1934992000 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8740055990 con 0x7f87500d2820 2023-11-28T13:01:10.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 -- 172.21.15.6:0/1934992000 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87300682f0 msgr2=0x7f873006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87300682f0 0x7f873006a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8738000c00 tx=0x7f873801c040 comp rx=0 tx=0).stop 2023-11-28T13:01:10.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 -- 172.21.15.6:0/1934992000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87500d2820 msgr2=0x7f875014f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87500d2820 0x7f875014f4e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f874000e9b0 tx=0x7f874000ecc0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 -- 172.21.15.6:0/1934992000 shutdown_connections 2023-11-28T13:01:10.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87300682f0 0x7f873006a7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f875015c640 0x7f87500c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f875006bf30 0x7f87500c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 --2- 172.21.15.6:0/1934992000 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f87500d2820 0x7f875014f4e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 -- 172.21.15.6:0/1934992000 >> 172.21.15.6:0/1934992000 conn(0x7f87500d2df0 msgr2=0x7f87500b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 -- 172.21.15.6:0/1934992000 shutdown_connections 2023-11-28T13:01:10.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.625+0000 7f875549e700 1 -- 172.21.15.6:0/1934992000 wait complete. 2023-11-28T13:01:10.628 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:01:10.639 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2023-11-28T13:01:10.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:10.639 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:10.677 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:10.678 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:01:10.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:10.678 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:01:10.735 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:01:10.736 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:10.757 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:01:10.758 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:10.819 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:10.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.943+0000 7f92eb7e5700 1 Processor -- start 2023-11-28T13:01:10.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.943+0000 7f92eb7e5700 1 -- start start 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92eb7e5700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f92e4159590 0x7f92e406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92eb7e5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e415db70 0x7f92e40ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92eb7e5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f92e40ce890 0x7f92e40ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92eb7e5700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e40cf250 con 0x7f92e4159590 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92eb7e5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e415d120 con 0x7f92e40ce890 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92eb7e5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e415d6e0 con 0x7f92e415db70 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92e9581700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f92e4159590 0x7f92e406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92e9581700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f92e4159590 0x7f92e406e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59240/0 (socket says 172.21.15.6:59240) 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92e9581700 1 -- 172.21.15.6:0/3947446014 learned_addr learned my addr 172.21.15.6:0/3947446014 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92e9581700 1 -- 172.21.15.6:0/3947446014 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e415db70 msgr2=0x7f92e40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92e9581700 1 --2- 172.21.15.6:0/3947446014 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e415db70 0x7f92e40ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92e9581700 1 -- 172.21.15.6:0/3947446014 >> v2:172.21.15.61:3300/0 conn(0x7f92e40ce890 msgr2=0x7f92e40ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:10.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92e9581700 1 --2- 172.21.15.6:0/3947446014 >> v2:172.21.15.61:3300/0 conn(0x7f92e40ce890 0x7f92e40ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92e9581700 1 -- 172.21.15.6:0/3947446014 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40aef60 con 0x7f92e4159590 2023-11-28T13:01:10.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.944+0000 7f92e9581700 1 --2- 172.21.15.6:0/3947446014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 0x7f92e406e400 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f92e40b15e0 tx=0x7f92e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaaa842f96f1407b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92dbfff700 1 -- 172.21.15.6:0/3947446014 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e000f8a0 con 0x7f92e4159590 2023-11-28T13:01:10.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92dbfff700 1 -- 172.21.15.6:0/3947446014 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e000fa00 con 0x7f92e4159590 2023-11-28T13:01:10.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92dbfff700 1 -- 172.21.15.6:0/3947446014 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e000fd90 con 0x7f92e4159590 2023-11-28T13:01:10.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92eb7e5700 1 -- 172.21.15.6:0/3947446014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 msgr2=0x7f92e406e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92eb7e5700 1 --2- 172.21.15.6:0/3947446014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 0x7f92e406e400 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f92e40b15e0 tx=0x7f92e0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92eb7e5700 1 -- 172.21.15.6:0/3947446014 shutdown_connections 2023-11-28T13:01:10.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92eb7e5700 1 --2- 172.21.15.6:0/3947446014 >> v2:172.21.15.61:3300/0 conn(0x7f92e40ce890 0x7f92e40ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92eb7e5700 1 --2- 172.21.15.6:0/3947446014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 0x7f92e406e400 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92eb7e5700 1 --2- 172.21.15.6:0/3947446014 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e415db70 0x7f92e40ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92eb7e5700 1 -- 172.21.15.6:0/3947446014 >> 172.21.15.6:0/3947446014 conn(0x7f92e40b1650 msgr2=0x7f92e40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:10.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92eb7e5700 1 -- 172.21.15.6:0/3947446014 shutdown_connections 2023-11-28T13:01:10.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.945+0000 7f92eb7e5700 1 -- 172.21.15.6:0/3947446014 wait complete. 2023-11-28T13:01:10.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.946+0000 7f92eb7e5700 1 Processor -- start 2023-11-28T13:01:10.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.946+0000 7f92eb7e5700 1 -- start start 2023-11-28T13:01:10.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92eb7e5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e40ce890 0x7f92e415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92eb7e5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 0x7f92e40b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92eb7e5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92e415db70 0x7f92e4151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92eb7e5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41f0990 con 0x7f92e4159590 2023-11-28T13:01:10.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92eb7e5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41efd10 con 0x7f92e415db70 2023-11-28T13:01:10.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92eb7e5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41efff0 con 0x7f92e40ce890 2023-11-28T13:01:10.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92e8d80700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 0x7f92e40b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92e8d80700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 0x7f92e40b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59248/0 (socket says 172.21.15.6:59248) 2023-11-28T13:01:10.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92e8d80700 1 -- 172.21.15.6:0/2649024926 learned_addr learned my addr 172.21.15.6:0/2649024926 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:10.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92e9d82700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92e415db70 0x7f92e4151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.947+0000 7f92e9581700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e40ce890 0x7f92e415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92e8d80700 1 -- 172.21.15.6:0/2649024926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e40ce890 msgr2=0x7f92e415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92e8d80700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e40ce890 0x7f92e415c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92e8d80700 1 -- 172.21.15.6:0/2649024926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92e415db70 msgr2=0x7f92e4151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:10.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92e8d80700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92e415db70 0x7f92e4151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:10.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92e8d80700 1 -- 172.21.15.6:0/2649024926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e0009550 con 0x7f92e4159590 2023-11-28T13:01:10.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92e8d80700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 0x7f92e40b2950 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f92d400b4a0 tx=0x7f92d400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92d9ffb700 1 -- 172.21.15.6:0/2649024926 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4011830 con 0x7f92e4159590 2023-11-28T13:01:10.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92d9ffb700 1 -- 172.21.15.6:0/2649024926 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4011990 con 0x7f92e4159590 2023-11-28T13:01:10.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92eb7e5700 1 -- 172.21.15.6:0/2649024926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e415d370 con 0x7f92e4159590 2023-11-28T13:01:10.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92d9ffb700 1 -- 172.21.15.6:0/2649024926 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4011d40 con 0x7f92e4159590 2023-11-28T13:01:10.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.948+0000 7f92eb7e5700 1 -- 172.21.15.6:0/2649024926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e40af240 con 0x7f92e4159590 2023-11-28T13:01:10.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.949+0000 7f92eb7e5700 1 -- 172.21.15.6:0/2649024926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e40aec70 con 0x7f92e4159590 2023-11-28T13:01:10.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.950+0000 7f92d9ffb700 1 -- 172.21.15.6:0/2649024926 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f92d4021070 con 0x7f92e4159590 2023-11-28T13:01:10.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.950+0000 7f92d9ffb700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c80682f0 0x7f92c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:10.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.950+0000 7f92d9ffb700 1 -- 172.21.15.6:0/2649024926 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f92d4085eb0 con 0x7f92e4159590 2023-11-28T13:01:10.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.950+0000 7f92e9581700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c80682f0 0x7f92c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:10.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.950+0000 7f92e9581700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c80682f0 0x7f92c806a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f92e0000c00 tx=0x7f92e0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:10.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:10.954+0000 7f92d9ffb700 1 -- 172.21.15.6:0/2649024926 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f92d4052360 con 0x7f92e4159590 2023-11-28T13:01:11.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.151+0000 7f92eb7e5700 1 -- 172.21.15.6:0/2649024926 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92e41f0ad0 con 0x7f92e4159590 2023-11-28T13:01:11.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.152+0000 7f92d9ffb700 1 -- 172.21.15.6:0/2649024926 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19690 (secure 0 0 0) 0x7f92e41f0ad0 con 0x7f92e4159590 2023-11-28T13:01:11.154 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:11.154 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:10.053012+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:11.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 -- 172.21.15.6:0/2649024926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c80682f0 msgr2=0x7f92c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c80682f0 0x7f92c806a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f92e0000c00 tx=0x7f92e0008030 comp rx=0 tx=0).stop 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 -- 172.21.15.6:0/2649024926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 msgr2=0x7f92e40b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 0x7f92e40b2950 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f92d400b4a0 tx=0x7f92d400b860 comp rx=0 tx=0).stop 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 -- 172.21.15.6:0/2649024926 shutdown_connections 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92e415db70 0x7f92e4151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c80682f0 0x7f92c806a7a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e40ce890 0x7f92e415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 --2- 172.21.15.6:0/2649024926 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e4159590 0x7f92e40b2950 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 -- 172.21.15.6:0/2649024926 >> 172.21.15.6:0/2649024926 conn(0x7f92e40b1650 msgr2=0x7f92e406d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 -- 172.21.15.6:0/2649024926 shutdown_connections 2023-11-28T13:01:11.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.155+0000 7f92eb7e5700 1 -- 172.21.15.6:0/2649024926 wait complete. 2023-11-28T13:01:11.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:11.169 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:11.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.296+0000 7f480153f700 1 Processor -- start 2023-11-28T13:01:11.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.296+0000 7f480153f700 1 -- start start 2023-11-28T13:01:11.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.297+0000 7f480153f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc15c640 0x7f47fc0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.297+0000 7f480153f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f47fc0d2820 0x7f47fc0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.297+0000 7f480153f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47fc0d1470 con 0x7f47fc06bf30 2023-11-28T13:01:11.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.297+0000 7f480153f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47fc15ca10 con 0x7f47fc0d2820 2023-11-28T13:01:11.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.297+0000 7f480153f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0cd4e0 con 0x7f47fc15c640 2023-11-28T13:01:11.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.297+0000 7f47fa7fc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f47fc0d2820 0x7f47fc0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.297+0000 7f47faffd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc15c640 0x7f47fc0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.297+0000 7f47fb7fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f47fc06bf30 0x7f47fc06c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:36608/0 (socket says 172.21.15.6:36608) 2023-11-28T13:01:11.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.297+0000 7f47fb7fe700 1 -- 172.21.15.6:0/407295852 learned_addr learned my addr 172.21.15.6:0/407295852 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:11.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.298+0000 7f47fa7fc700 1 -- 172.21.15.6:0/407295852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc15c640 msgr2=0x7f47fc0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:11.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.298+0000 7f47fa7fc700 1 --2- 172.21.15.6:0/407295852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc15c640 0x7f47fc0cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.298+0000 7f47fa7fc700 1 -- 172.21.15.6:0/407295852 >> v1:172.21.15.61:6789/0 conn(0x7f47fc06bf30 legacy=0x7f47fc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.298+0000 7f47f9ffb700 1 -- 172.21.15.6:0/407295852 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897572230 0 0) 0x7f47fc0d1470 con 0x7f47fc06bf30 2023-11-28T13:01:11.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.298+0000 7f47fa7fc700 1 -- 172.21.15.6:0/407295852 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc0cd7c0 con 0x7f47fc0d2820 2023-11-28T13:01:11.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.298+0000 7f47faffd700 1 --2- 172.21.15.6:0/407295852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc15c640 0x7f47fc0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:11.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.298+0000 7f47fa7fc700 1 --2- 172.21.15.6:0/407295852 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47fc0d2820 0x7f47fc0b5c40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009840 tx=0x7f47ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6d1238fdda9c057 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.298+0000 7f47f9ffb700 1 -- 172.21.15.6:0/407295852 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec011830 con 0x7f47fc0d2820 2023-11-28T13:01:11.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.298+0000 7f47f9ffb700 1 -- 172.21.15.6:0/407295852 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec011990 con 0x7f47fc0d2820 2023-11-28T13:01:11.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.298+0000 7f47f9ffb700 1 -- 172.21.15.6:0/407295852 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec011d20 con 0x7f47fc0d2820 2023-11-28T13:01:11.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.299+0000 7f480153f700 1 -- 172.21.15.6:0/407295852 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47fc0d2820 msgr2=0x7f47fc0b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.299+0000 7f480153f700 1 --2- 172.21.15.6:0/407295852 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47fc0d2820 0x7f47fc0b5c40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009840 tx=0x7f47ec009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:11.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.299+0000 7f480153f700 1 -- 172.21.15.6:0/407295852 shutdown_connections 2023-11-28T13:01:11.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.299+0000 7f480153f700 1 --2- 172.21.15.6:0/407295852 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc15c640 0x7f47fc0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.299+0000 7f480153f700 1 --2- 172.21.15.6:0/407295852 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47fc0d2820 0x7f47fc0b5c40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.299+0000 7f480153f700 1 -- 172.21.15.6:0/407295852 >> 172.21.15.6:0/407295852 conn(0x7f47fc0d2df0 msgr2=0x7f47fc0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:11.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.299+0000 7f480153f700 1 -- 172.21.15.6:0/407295852 shutdown_connections 2023-11-28T13:01:11.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.299+0000 7f480153f700 1 -- 172.21.15.6:0/407295852 wait complete. 2023-11-28T13:01:11.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.300+0000 7f480153f700 1 Processor -- start 2023-11-28T13:01:11.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.300+0000 7f480153f700 1 -- start start 2023-11-28T13:01:11.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f480153f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47fc06bf30 0x7f47fc0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f480153f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc0d2820 0x7f47fc157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f480153f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47fc15c640 0x7f47fc0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f480153f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0cb460 con 0x7f47fc15c640 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f480153f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0cafd0 con 0x7f47fc06bf30 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f480153f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0ca3c0 con 0x7f47fc0d2820 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47fb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47fc15c640 0x7f47fc0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47fb7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47fc15c640 0x7f47fc0c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59264/0 (socket says 172.21.15.6:59264) 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47fb7fe700 1 -- 172.21.15.6:0/4185451883 learned_addr learned my addr 172.21.15.6:0/4185451883 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47faffd700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47fc06bf30 0x7f47fc0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47fa7fc700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc0d2820 0x7f47fc157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47fb7fe700 1 -- 172.21.15.6:0/4185451883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc0d2820 msgr2=0x7f47fc157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47fb7fe700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc0d2820 0x7f47fc157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47fb7fe700 1 -- 172.21.15.6:0/4185451883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47fc06bf30 msgr2=0x7f47fc0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47fb7fe700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47fc06bf30 0x7f47fc0cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47fb7fe700 1 -- 172.21.15.6:0/4185451883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f00094a0 con 0x7f47fc15c640 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47faffd700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47fc06bf30 0x7f47fc0cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.301+0000 7f47fb7fe700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47fc15c640 0x7f47fc0c8900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f47f400cc70 tx=0x7f47f400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.302+0000 7f47e37fe700 1 -- 172.21.15.6:0/4185451883 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f400ada0 con 0x7f47fc15c640 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.302+0000 7f47e37fe700 1 -- 172.21.15.6:0/4185451883 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f400ce70 con 0x7f47fc15c640 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.302+0000 7f480153f700 1 -- 172.21.15.6:0/4185451883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ec0094a0 con 0x7f47fc15c640 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.302+0000 7f47e37fe700 1 -- 172.21.15.6:0/4185451883 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f40072f0 con 0x7f47fc15c640 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.302+0000 7f480153f700 1 -- 172.21.15.6:0/4185451883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc0cf580 con 0x7f47fc15c640 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.303+0000 7f480153f700 1 -- 172.21.15.6:0/4185451883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47fc0d1710 con 0x7f47fc15c640 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.303+0000 7f47e37fe700 1 -- 172.21.15.6:0/4185451883 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f47f401a030 con 0x7f47fc15c640 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.303+0000 7f47e37fe700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f47e80682f0 0x7f47e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.303+0000 7f47e37fe700 1 -- 172.21.15.6:0/4185451883 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f47f4085d80 con 0x7f47fc15c640 2023-11-28T13:01:11.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.303+0000 7f47faffd700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f47e80682f0 0x7f47e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.304+0000 7f47faffd700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f47e80682f0 0x7f47e806a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f47fc1589d0 tx=0x7f47f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.308+0000 7f47e37fe700 1 -- 172.21.15.6:0/4185451883 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f47f4052250 con 0x7f47fc15c640 2023-11-28T13:01:11.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.500+0000 7f480153f700 1 -- 172.21.15.6:0/4185451883 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f47fc0cdaa0 con 0x7f47fc15c640 2023-11-28T13:01:11.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.501+0000 7f47e37fe700 1 -- 172.21.15.6:0/4185451883 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f47f40180a0 con 0x7f47fc15c640 2023-11-28T13:01:11.502 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:11.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 -- 172.21.15.6:0/4185451883 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f47e80682f0 msgr2=0x7f47e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f47e80682f0 0x7f47e806a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f47fc1589d0 tx=0x7f47f0008040 comp rx=0 tx=0).stop 2023-11-28T13:01:11.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 -- 172.21.15.6:0/4185451883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47fc15c640 msgr2=0x7f47fc0c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47fc15c640 0x7f47fc0c8900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f47f400cc70 tx=0x7f47f400a340 comp rx=0 tx=0).stop 2023-11-28T13:01:11.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 -- 172.21.15.6:0/4185451883 shutdown_connections 2023-11-28T13:01:11.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47fc15c640 0x7f47fc0c8900 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f47fc06bf30 0x7f47fc0cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f47e80682f0 0x7f47e806a7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 --2- 172.21.15.6:0/4185451883 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f47fc0d2820 0x7f47fc157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 -- 172.21.15.6:0/4185451883 >> 172.21.15.6:0/4185451883 conn(0x7f47fc0d2df0 msgr2=0x7f47fc06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:11.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 -- 172.21.15.6:0/4185451883 shutdown_connections 2023-11-28T13:01:11.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.503+0000 7f480153f700 1 -- 172.21.15.6:0/4185451883 wait complete. 2023-11-28T13:01:11.516 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:11.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.642+0000 7fbff4abd700 1 Processor -- start 2023-11-28T13:01:11.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.643+0000 7fbff4abd700 1 -- start start 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.643+0000 7fbff4abd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbff00cf5f0 0x7fbff00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.643+0000 7fbff4abd700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff00ce550 con 0x7fbff00b1630 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.643+0000 7fbff4abd700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff00ce830 con 0x7fbff00cb3e0 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.643+0000 7fbff4abd700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff00ceb10 con 0x7fbff00cf5f0 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfee59c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fbff00b1630 0x7fbff00b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46588/0 (socket says 172.21.15.6:46588) 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfee59c700 1 -- 172.21.15.6:0/777007757 learned_addr learned my addr 172.21.15.6:0/777007757 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfed59a700 1 -- 172.21.15.6:0/777007757 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413662350 0 0) 0x7fbff00ce550 con 0x7fbff00b1630 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfedd9b700 1 --2- 172.21.15.6:0/777007757 >> v2:172.21.15.61:3300/0 conn(0x7fbff00cf5f0 0x7fbff00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfed59a700 1 -- 172.21.15.6:0/777007757 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfd80035b0 con 0x7fbff00b1630 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfed59a700 1 -- 172.21.15.6:0/777007757 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022597258 0 0) 0x7fbff00ce830 con 0x7fbff00cb3e0 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfed59a700 1 -- 172.21.15.6:0/777007757 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff00ce550 con 0x7fbff00cb3e0 2023-11-28T13:01:11.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfed59a700 1 -- 172.21.15.6:0/777007757 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2247644208 0 0) 0x7fbfd80035b0 con 0x7fbff00b1630 2023-11-28T13:01:11.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfedd9b700 1 -- 172.21.15.6:0/777007757 >> v1:172.21.15.61:6789/0 conn(0x7fbff00cb3e0 legacy=0x7fbff006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfedd9b700 1 -- 172.21.15.6:0/777007757 >> v1:172.21.15.6:6789/0 conn(0x7fbff00b1630 legacy=0x7fbff00b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfedd9b700 1 -- 172.21.15.6:0/777007757 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00cedf0 con 0x7fbff00cf5f0 2023-11-28T13:01:11.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.644+0000 7fbfedd9b700 1 --2- 172.21.15.6:0/777007757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbff00cf5f0 0x7fbff00caea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009870 tx=0x7fbfe0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f38997bfce83a1d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.645+0000 7fbfed59a700 1 -- 172.21.15.6:0/777007757 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4001670 con 0x7fbff00cf5f0 2023-11-28T13:01:11.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.645+0000 7fbfed59a700 1 -- 172.21.15.6:0/777007757 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe0004030 con 0x7fbff00cf5f0 2023-11-28T13:01:11.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.645+0000 7fbfed59a700 1 -- 172.21.15.6:0/777007757 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe00043c0 con 0x7fbff00cf5f0 2023-11-28T13:01:11.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.645+0000 7fbff4abd700 1 -- 172.21.15.6:0/777007757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbff00cf5f0 msgr2=0x7fbff00caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.645+0000 7fbff4abd700 1 --2- 172.21.15.6:0/777007757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbff00cf5f0 0x7fbff00caea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009870 tx=0x7fbfe0009b80 comp rx=0 tx=0).stop 2023-11-28T13:01:11.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.645+0000 7fbff4abd700 1 -- 172.21.15.6:0/777007757 shutdown_connections 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.645+0000 7fbff4abd700 1 --2- 172.21.15.6:0/777007757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbff00cf5f0 0x7fbff00caea0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.645+0000 7fbff4abd700 1 -- 172.21.15.6:0/777007757 >> 172.21.15.6:0/777007757 conn(0x7fbff00cc630 msgr2=0x7fbff00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.645+0000 7fbff4abd700 1 -- 172.21.15.6:0/777007757 shutdown_connections 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.645+0000 7fbff4abd700 1 -- 172.21.15.6:0/777007757 wait complete. 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.647+0000 7fbff4abd700 1 Processor -- start 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.647+0000 7fbff4abd700 1 -- start start 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.647+0000 7fbff4abd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbff00b1630 0x7fbff00b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.647+0000 7fbff4abd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbff00cb3e0 0x7fbff00c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.647+0000 7fbff4abd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbff00cf5f0 0x7fbff0156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.647+0000 7fbff4abd700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00c98e0 con 0x7fbff00cf5f0 2023-11-28T13:01:11.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.647+0000 7fbff4abd700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00c9bc0 con 0x7fbff00cb3e0 2023-11-28T13:01:11.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.647+0000 7fbff4abd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff00c9ea0 con 0x7fbff00b1630 2023-11-28T13:01:11.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.647+0000 7fbfeed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbff00cf5f0 0x7fbff0156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfeed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbff00cf5f0 0x7fbff0156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59278/0 (socket says 172.21.15.6:59278) 2023-11-28T13:01:11.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfeed9d700 1 -- 172.21.15.6:0/3764873628 learned_addr learned my addr 172.21.15.6:0/3764873628 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:11.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfee59c700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbff00b1630 0x7fbff00b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfedd9b700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbff00cb3e0 0x7fbff00c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfeed9d700 1 -- 172.21.15.6:0/3764873628 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbff00b1630 msgr2=0x7fbff00b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfeed9d700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbff00b1630 0x7fbff00b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfeed9d700 1 -- 172.21.15.6:0/3764873628 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbff00cb3e0 msgr2=0x7fbff00c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfeed9d700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbff00cb3e0 0x7fbff00c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfeed9d700 1 -- 172.21.15.6:0/3764873628 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe00094a0 con 0x7fbff00cf5f0 2023-11-28T13:01:11.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfee59c700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbff00b1630 0x7fbff00b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:11.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfeed9d700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbff00cf5f0 0x7fbff0156e90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800cc20 tx=0x7fbfe800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfdeffd700 1 -- 172.21.15.6:0/3764873628 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe800ac20 con 0x7fbff00cf5f0 2023-11-28T13:01:11.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfdeffd700 1 -- 172.21.15.6:0/3764873628 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe800ad80 con 0x7fbff00cf5f0 2023-11-28T13:01:11.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbff4abd700 1 -- 172.21.15.6:0/3764873628 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff00ce140 con 0x7fbff00cf5f0 2023-11-28T13:01:11.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbfdeffd700 1 -- 172.21.15.6:0/3764873628 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8003c20 con 0x7fbff00cf5f0 2023-11-28T13:01:11.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.648+0000 7fbff4abd700 1 -- 172.21.15.6:0/3764873628 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff00cf0d0 con 0x7fbff00cf5f0 2023-11-28T13:01:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.649+0000 7fbff4abd700 1 -- 172.21.15.6:0/3764873628 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff00ce2d0 con 0x7fbff00cf5f0 2023-11-28T13:01:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.650+0000 7fbfdeffd700 1 -- 172.21.15.6:0/3764873628 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbfe8010050 con 0x7fbff00cf5f0 2023-11-28T13:01:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.650+0000 7fbfdeffd700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfd00682f0 0x7fbfd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.650+0000 7fbfdeffd700 1 -- 172.21.15.6:0/3764873628 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbfe8086030 con 0x7fbff00cf5f0 2023-11-28T13:01:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.650+0000 7fbfee59c700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfd00682f0 0x7fbfd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.650+0000 7fbfee59c700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfd00682f0 0x7fbfd006a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40079b0 tx=0x7fbfe4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.654+0000 7fbfdeffd700 1 -- 172.21.15.6:0/3764873628 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbfe8052560 con 0x7fbff00cf5f0 2023-11-28T13:01:11.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.845+0000 7fbff4abd700 1 -- 172.21.15.6:0/3764873628 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbff00739f0 con 0x7fbff00cf5f0 2023-11-28T13:01:11.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.845+0000 7fbfdeffd700 1 -- 172.21.15.6:0/3764873628 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbfe8055b80 con 0x7fbff00cf5f0 2023-11-28T13:01:11.847 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:11.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.847+0000 7fbff4abd700 1 -- 172.21.15.6:0/3764873628 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfd00682f0 msgr2=0x7fbfd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.847+0000 7fbff4abd700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfd00682f0 0x7fbfd006a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40079b0 tx=0x7fbfe4009040 comp rx=0 tx=0).stop 2023-11-28T13:01:11.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.847+0000 7fbff4abd700 1 -- 172.21.15.6:0/3764873628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbff00cf5f0 msgr2=0x7fbff0156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.847+0000 7fbff4abd700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbff00cf5f0 0x7fbff0156e90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800cc20 tx=0x7fbfe800a060 comp rx=0 tx=0).stop 2023-11-28T13:01:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.847+0000 7fbff4abd700 1 -- 172.21.15.6:0/3764873628 shutdown_connections 2023-11-28T13:01:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.848+0000 7fbff4abd700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbff00cf5f0 0x7fbff0156e90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.848+0000 7fbff4abd700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfd00682f0 0x7fbfd006a7a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.848+0000 7fbff4abd700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbff00b1630 0x7fbff00b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.848+0000 7fbff4abd700 1 --2- 172.21.15.6:0/3764873628 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbff00cb3e0 0x7fbff00c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.848+0000 7fbff4abd700 1 -- 172.21.15.6:0/3764873628 >> 172.21.15.6:0/3764873628 conn(0x7fbff00cc630 msgr2=0x7fbff00b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.848+0000 7fbff4abd700 1 -- 172.21.15.6:0/3764873628 shutdown_connections 2023-11-28T13:01:11.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.848+0000 7fbff4abd700 1 -- 172.21.15.6:0/3764873628 wait complete. 2023-11-28T13:01:11.862 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:11.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.988+0000 7f52d2c82700 1 Processor -- start 2023-11-28T13:01:11.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.988+0000 7f52d2c82700 1 -- start start 2023-11-28T13:01:11.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.989+0000 7f52d2c82700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f52cc06b900 0x7f52cc0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:11.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.989+0000 7f52d2c82700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc0ae730 con 0x7f52cc0cd140 2023-11-28T13:01:11.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.989+0000 7f52d2c82700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc0c3e10 con 0x7f52cc15a640 2023-11-28T13:01:11.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.989+0000 7f52d2c82700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0cb210 con 0x7f52cc06b900 2023-11-28T13:01:11.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.989+0000 7f52d121f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f52cc06b900 0x7f52cc0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:11.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.989+0000 7f52cbfff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f52cc0cd140 0x7f52cc06b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46604/0 (socket says 172.21.15.6:46604) 2023-11-28T13:01:11.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.989+0000 7f52cbfff700 1 -- 172.21.15.6:0/656589573 learned_addr learned my addr 172.21.15.6:0/656589573 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:11.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d121f700 1 -- 172.21.15.6:0/656589573 >> v1:172.21.15.61:6789/0 conn(0x7f52cc15a640 legacy=0x7f52cc0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:11.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d121f700 1 -- 172.21.15.6:0/656589573 >> v1:172.21.15.6:6789/0 conn(0x7f52cc0cd140 legacy=0x7f52cc06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d121f700 1 -- 172.21.15.6:0/656589573 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc06e9e0 con 0x7f52cc06b900 2023-11-28T13:01:11.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52cb7fe700 1 -- 172.21.15.6:0/656589573 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869849595 0 0) 0x7f52cc0ae730 con 0x7f52cc0cd140 2023-11-28T13:01:11.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d121f700 1 --2- 172.21.15.6:0/656589573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc06b900 0x7f52cc0cb630 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009d90 tx=0x7f52c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b6a52d3353c79e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:11.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52cb7fe700 1 -- 172.21.15.6:0/656589573 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c400e070 con 0x7f52cc06b900 2023-11-28T13:01:11.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52cb7fe700 1 -- 172.21.15.6:0/656589573 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c400bb90 con 0x7f52cc06b900 2023-11-28T13:01:11.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52cb7fe700 1 -- 172.21.15.6:0/656589573 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4003a20 con 0x7f52cc06b900 2023-11-28T13:01:11.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d2c82700 1 -- 172.21.15.6:0/656589573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc06b900 msgr2=0x7f52cc0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:11.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d2c82700 1 --2- 172.21.15.6:0/656589573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc06b900 0x7f52cc0cb630 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009d90 tx=0x7f52c400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:11.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d2c82700 1 -- 172.21.15.6:0/656589573 shutdown_connections 2023-11-28T13:01:11.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d2c82700 1 --2- 172.21.15.6:0/656589573 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc06b900 0x7f52cc0cb630 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:11.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d2c82700 1 -- 172.21.15.6:0/656589573 >> 172.21.15.6:0/656589573 conn(0x7f52cc0cc630 msgr2=0x7f52cc0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:11.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d2c82700 1 -- 172.21.15.6:0/656589573 shutdown_connections 2023-11-28T13:01:11.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.990+0000 7f52d2c82700 1 -- 172.21.15.6:0/656589573 wait complete. 2023-11-28T13:01:11.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.992+0000 7f52d2c82700 1 Processor -- start 2023-11-28T13:01:12.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.992+0000 7f52d2c82700 1 -- start start 2023-11-28T13:01:12.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d2c82700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52cc06b900 0x7f52cc1f0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d2c82700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52cc0cd140 0x7f52cc1f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d2c82700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc15a640 0x7f52cc1ef770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d2c82700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0b48e0 con 0x7f52cc15a640 2023-11-28T13:01:12.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d2c82700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1f3b30 con 0x7f52cc06b900 2023-11-28T13:01:12.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d2c82700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1f3e00 con 0x7f52cc0cd140 2023-11-28T13:01:12.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d121f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc15a640 0x7f52cc1ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d121f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc15a640 0x7f52cc1ef770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59290/0 (socket says 172.21.15.6:59290) 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d121f700 1 -- 172.21.15.6:0/2136470780 learned_addr learned my addr 172.21.15.6:0/2136470780 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d0a1e700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52cc06b900 0x7f52cc1f0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52cbfff700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52cc0cd140 0x7f52cc1f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d121f700 1 -- 172.21.15.6:0/2136470780 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52cc0cd140 msgr2=0x7f52cc1f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d121f700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52cc0cd140 0x7f52cc1f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d121f700 1 -- 172.21.15.6:0/2136470780 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52cc06b900 msgr2=0x7f52cc1f0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d121f700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52cc06b900 0x7f52cc1f0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d121f700 1 -- 172.21.15.6:0/2136470780 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009a70 con 0x7f52cc15a640 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52cbfff700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52cc0cd140 0x7f52cc1f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d0a1e700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52cc06b900 0x7f52cc1f0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.993+0000 7f52d121f700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc15a640 0x7f52cc1ef770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f52c4004000 tx=0x7f52c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.994+0000 7f52c97fa700 1 -- 172.21.15.6:0/2136470780 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4012630 con 0x7f52cc15a640 2023-11-28T13:01:12.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.994+0000 7f52c97fa700 1 -- 172.21.15.6:0/2136470780 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c40127b0 con 0x7f52cc15a640 2023-11-28T13:01:12.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.994+0000 7f52d2c82700 1 -- 172.21.15.6:0/2136470780 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc0aec60 con 0x7f52cc15a640 2023-11-28T13:01:12.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.994+0000 7f52c97fa700 1 -- 172.21.15.6:0/2136470780 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4012b60 con 0x7f52cc15a640 2023-11-28T13:01:12.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.994+0000 7f52d2c82700 1 -- 172.21.15.6:0/2136470780 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc0ae980 con 0x7f52cc15a640 2023-11-28T13:01:12.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.995+0000 7f52d2c82700 1 -- 172.21.15.6:0/2136470780 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc0aef40 con 0x7f52cc15a640 2023-11-28T13:01:12.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.996+0000 7f52c97fa700 1 -- 172.21.15.6:0/2136470780 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f52c402c070 con 0x7f52cc15a640 2023-11-28T13:01:12.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.996+0000 7f52c97fa700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52b00682f0 0x7f52b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.996+0000 7f52c97fa700 1 -- 172.21.15.6:0/2136470780 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f52c4088910 con 0x7f52cc15a640 2023-11-28T13:01:12.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.996+0000 7f52d0a1e700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52b00682f0 0x7f52b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.996+0000 7f52d0a1e700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52b00682f0 0x7f52b006a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0b1e30 tx=0x7f52c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.004 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:11.999+0000 7f52c97fa700 1 -- 172.21.15.6:0/2136470780 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52c4058910 con 0x7f52cc15a640 2023-11-28T13:01:12.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.199+0000 7f52d2c82700 1 -- 172.21.15.6:0/2136470780 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52cc073a50 con 0x7f52cc15a640 2023-11-28T13:01:12.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.200+0000 7f52c97fa700 1 -- 172.21.15.6:0/2136470780 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52c4058500 con 0x7f52cc15a640 2023-11-28T13:01:12.202 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:12.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 -- 172.21.15.6:0/2136470780 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52b00682f0 msgr2=0x7f52b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52b00682f0 0x7f52b006a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0b1e30 tx=0x7f52c0008040 comp rx=0 tx=0).stop 2023-11-28T13:01:12.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 -- 172.21.15.6:0/2136470780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc15a640 msgr2=0x7f52cc1ef770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc15a640 0x7f52cc1ef770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f52c4004000 tx=0x7f52c4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 -- 172.21.15.6:0/2136470780 shutdown_connections 2023-11-28T13:01:12.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52cc15a640 0x7f52cc1ef770 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52cc06b900 0x7f52cc1f0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52b00682f0 0x7f52b006a7a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 --2- 172.21.15.6:0/2136470780 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52cc0cd140 0x7f52cc1f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 -- 172.21.15.6:0/2136470780 >> 172.21.15.6:0/2136470780 conn(0x7f52cc0cc630 msgr2=0x7f52cc0cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:12.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 -- 172.21.15.6:0/2136470780 shutdown_connections 2023-11-28T13:01:12.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.202+0000 7f52d2c82700 1 -- 172.21.15.6:0/2136470780 wait complete. 2023-11-28T13:01:12.215 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:12.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.342+0000 7f8d3b04d700 1 Processor -- start 2023-11-28T13:01:12.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.342+0000 7f8d3b04d700 1 -- start start 2023-11-28T13:01:12.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d3b04d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8d34158d90 0x7f8d340ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d3b04d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8d3406e900 0x7f8d3406ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d3b04d700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3406e470 con 0x7f8d341589c0 2023-11-28T13:01:12.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d3b04d700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3406de60 con 0x7f8d3406e900 2023-11-28T13:01:12.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d3b04d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d340ceb40 con 0x7f8d34158d90 2023-11-28T13:01:12.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d395ea700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8d3406e900 0x7f8d3406ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d38de9700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f8d341589c0 0x7f8d3406aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46614/0 (socket says 172.21.15.6:46614) 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d38de9700 1 -- 172.21.15.6:0/1215753263 learned_addr learned my addr 172.21.15.6:0/1215753263 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d33fff700 1 --2- 172.21.15.6:0/1215753263 >> v2:172.21.15.61:3300/0 conn(0x7f8d34158d90 0x7f8d340ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d395ea700 1 -- 172.21.15.6:0/1215753263 >> v2:172.21.15.61:3300/0 conn(0x7f8d34158d90 msgr2=0x7f8d340ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d395ea700 1 --2- 172.21.15.6:0/1215753263 >> v2:172.21.15.61:3300/0 conn(0x7f8d34158d90 0x7f8d340ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d395ea700 1 -- 172.21.15.6:0/1215753263 >> v1:172.21.15.6:6789/0 conn(0x7f8d341589c0 legacy=0x7f8d3406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d395ea700 1 -- 172.21.15.6:0/1215753263 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340cee20 con 0x7f8d3406e900 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.343+0000 7f8d337fe700 1 -- 172.21.15.6:0/1215753263 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481309538 0 0) 0x7f8d3406e470 con 0x7f8d341589c0 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d395ea700 1 --2- 172.21.15.6:0/1215753263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d3406e900 0x7f8d3406ed60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8d34071010 tx=0x7f8d2c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934265c58b47de2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d337fe700 1 -- 172.21.15.6:0/1215753263 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00c9d0 con 0x7f8d3406e900 2023-11-28T13:01:12.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d337fe700 1 -- 172.21.15.6:0/1215753263 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c00cb30 con 0x7f8d3406e900 2023-11-28T13:01:12.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d337fe700 1 -- 172.21.15.6:0/1215753263 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c011400 con 0x7f8d3406e900 2023-11-28T13:01:12.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d3b04d700 1 -- 172.21.15.6:0/1215753263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d3406e900 msgr2=0x7f8d3406ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d3b04d700 1 --2- 172.21.15.6:0/1215753263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d3406e900 0x7f8d3406ed60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8d34071010 tx=0x7f8d2c009f60 comp rx=0 tx=0).stop 2023-11-28T13:01:12.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d3b04d700 1 -- 172.21.15.6:0/1215753263 shutdown_connections 2023-11-28T13:01:12.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d3b04d700 1 --2- 172.21.15.6:0/1215753263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d3406e900 0x7f8d3406ed60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d3b04d700 1 --2- 172.21.15.6:0/1215753263 >> v2:172.21.15.61:3300/0 conn(0x7f8d34158d90 0x7f8d340ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d3b04d700 1 -- 172.21.15.6:0/1215753263 >> 172.21.15.6:0/1215753263 conn(0x7f8d340b0df0 msgr2=0x7f8d340b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:12.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.344+0000 7f8d3b04d700 1 -- 172.21.15.6:0/1215753263 shutdown_connections 2023-11-28T13:01:12.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.345+0000 7f8d3b04d700 1 -- 172.21.15.6:0/1215753263 wait complete. 2023-11-28T13:01:12.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.346+0000 7f8d3b04d700 1 Processor -- start 2023-11-28T13:01:12.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.346+0000 7f8d3b04d700 1 -- start start 2023-11-28T13:01:12.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.347+0000 7f8d3b04d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d3406e900 0x7f8d34156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.347+0000 7f8d3b04d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d341589c0 0x7f8d340b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.347+0000 7f8d3b04d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d34158d90 0x7f8d340c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.347+0000 7f8d3b04d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340ac260 con 0x7f8d341589c0 2023-11-28T13:01:12.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.347+0000 7f8d3b04d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340b3440 con 0x7f8d3406e900 2023-11-28T13:01:12.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.347+0000 7f8d3b04d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340ab8c0 con 0x7f8d34158d90 2023-11-28T13:01:12.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.347+0000 7f8d33fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d341589c0 0x7f8d340b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.347+0000 7f8d33fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d341589c0 0x7f8d340b2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59302/0 (socket says 172.21.15.6:59302) 2023-11-28T13:01:12.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.347+0000 7f8d33fff700 1 -- 172.21.15.6:0/998893514 learned_addr learned my addr 172.21.15.6:0/998893514 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:12.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d395ea700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d34158d90 0x7f8d340c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d38de9700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d3406e900 0x7f8d34156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d33fff700 1 -- 172.21.15.6:0/998893514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d34158d90 msgr2=0x7f8d340c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d33fff700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d34158d90 0x7f8d340c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d33fff700 1 -- 172.21.15.6:0/998893514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d3406e900 msgr2=0x7f8d34156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d33fff700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d3406e900 0x7f8d34156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d33fff700 1 -- 172.21.15.6:0/998893514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c00d040 con 0x7f8d341589c0 2023-11-28T13:01:12.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d33fff700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d341589c0 0x7f8d340b2b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000e840 tx=0x7f8d2000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d317fa700 1 -- 172.21.15.6:0/998893514 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d20009590 con 0x7f8d341589c0 2023-11-28T13:01:12.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d317fa700 1 -- 172.21.15.6:0/998893514 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d20013680 con 0x7f8d341589c0 2023-11-28T13:01:12.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d317fa700 1 -- 172.21.15.6:0/998893514 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d20013a30 con 0x7f8d341589c0 2023-11-28T13:01:12.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d3b04d700 1 -- 172.21.15.6:0/998893514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d340c9d90 con 0x7f8d341589c0 2023-11-28T13:01:12.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.348+0000 7f8d3b04d700 1 -- 172.21.15.6:0/998893514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3406e100 con 0x7f8d341589c0 2023-11-28T13:01:12.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.349+0000 7f8d3b04d700 1 -- 172.21.15.6:0/998893514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3406f2a0 con 0x7f8d341589c0 2023-11-28T13:01:12.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.349+0000 7f8d317fa700 1 -- 172.21.15.6:0/998893514 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8d20023070 con 0x7f8d341589c0 2023-11-28T13:01:12.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.350+0000 7f8d317fa700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d18068320 0x7f8d1806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.350+0000 7f8d317fa700 1 -- 172.21.15.6:0/998893514 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8d20086120 con 0x7f8d341589c0 2023-11-28T13:01:12.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.350+0000 7f8d38de9700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d18068320 0x7f8d1806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.350+0000 7f8d38de9700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d18068320 0x7f8d1806a7d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d28000f60 tx=0x7f8d28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.354+0000 7f8d317fa700 1 -- 172.21.15.6:0/998893514 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8d20052650 con 0x7f8d341589c0 2023-11-28T13:01:12.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.545+0000 7f8d3b04d700 1 -- 172.21.15.6:0/998893514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d340739f0 con 0x7f8d341589c0 2023-11-28T13:01:12.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.546+0000 7f8d317fa700 1 -- 172.21.15.6:0/998893514 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d20055c70 con 0x7f8d341589c0 2023-11-28T13:01:12.547 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.547+0000 7f8d3b04d700 1 -- 172.21.15.6:0/998893514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d18068320 msgr2=0x7f8d1806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.547+0000 7f8d3b04d700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d18068320 0x7f8d1806a7d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d28000f60 tx=0x7f8d28009040 comp rx=0 tx=0).stop 2023-11-28T13:01:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.547+0000 7f8d3b04d700 1 -- 172.21.15.6:0/998893514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d341589c0 msgr2=0x7f8d340b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.547+0000 7f8d3b04d700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d341589c0 0x7f8d340b2b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000e840 tx=0x7f8d2000ec00 comp rx=0 tx=0).stop 2023-11-28T13:01:12.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.548+0000 7f8d3b04d700 1 -- 172.21.15.6:0/998893514 shutdown_connections 2023-11-28T13:01:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.548+0000 7f8d3b04d700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d18068320 0x7f8d1806a7d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.548+0000 7f8d3b04d700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d34158d90 0x7f8d340c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.548+0000 7f8d3b04d700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d3406e900 0x7f8d34156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.548+0000 7f8d3b04d700 1 --2- 172.21.15.6:0/998893514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d341589c0 0x7f8d340b2b40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.548+0000 7f8d3b04d700 1 -- 172.21.15.6:0/998893514 >> 172.21.15.6:0/998893514 conn(0x7f8d340b0df0 msgr2=0x7f8d34157840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.548+0000 7f8d3b04d700 1 -- 172.21.15.6:0/998893514 shutdown_connections 2023-11-28T13:01:12.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.548+0000 7f8d3b04d700 1 -- 172.21.15.6:0/998893514 wait complete. 2023-11-28T13:01:12.561 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.687+0000 7fca8934f700 1 Processor -- start 2023-11-28T13:01:12.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.687+0000 7fca8934f700 1 -- start start 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca8934f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fca8415c640 0x7fca840cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca8934f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fca840d2820 0x7fca840b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca8934f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca8406bf30 0x7fca8406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca8934f700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8415ca10 con 0x7fca8415c640 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca8934f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8406c870 con 0x7fca840d2820 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca8934f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca840d1470 con 0x7fca8406bf30 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca82ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fca8415c640 0x7fca840cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca82ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fca8415c640 0x7fca840cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59308/0 (socket says 172.21.15.6:59308) 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca82ffd700 1 -- 172.21.15.6:0/938776133 learned_addr learned my addr 172.21.15.6:0/938776133 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca837fe700 1 --2- 172.21.15.6:0/938776133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca8406bf30 0x7fca8406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca827fc700 1 --2- 172.21.15.6:0/938776133 >> v2:172.21.15.61:3300/0 conn(0x7fca840d2820 0x7fca840b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca82ffd700 1 -- 172.21.15.6:0/938776133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca8406bf30 msgr2=0x7fca8406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca82ffd700 1 --2- 172.21.15.6:0/938776133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca8406bf30 0x7fca8406c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca82ffd700 1 -- 172.21.15.6:0/938776133 >> v2:172.21.15.61:3300/0 conn(0x7fca840d2820 msgr2=0x7fca840b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca82ffd700 1 --2- 172.21.15.6:0/938776133 >> v2:172.21.15.61:3300/0 conn(0x7fca840d2820 0x7fca840b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.688+0000 7fca82ffd700 1 -- 172.21.15.6:0/938776133 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca840cd4e0 con 0x7fca8415c640 2023-11-28T13:01:12.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca82ffd700 1 --2- 172.21.15.6:0/938776133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8415c640 0x7fca840cf040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fca840c9e50 tx=0x7fca78009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3329357560454c3e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca81ffb700 1 -- 172.21.15.6:0/938776133 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7800b870 con 0x7fca8415c640 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca81ffb700 1 -- 172.21.15.6:0/938776133 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7800b9d0 con 0x7fca8415c640 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca81ffb700 1 -- 172.21.15.6:0/938776133 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7800bd60 con 0x7fca8415c640 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca8934f700 1 -- 172.21.15.6:0/938776133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8415c640 msgr2=0x7fca840cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca8934f700 1 --2- 172.21.15.6:0/938776133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8415c640 0x7fca840cf040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fca840c9e50 tx=0x7fca78009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca8934f700 1 -- 172.21.15.6:0/938776133 shutdown_connections 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca8934f700 1 --2- 172.21.15.6:0/938776133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca8406bf30 0x7fca8406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca8934f700 1 --2- 172.21.15.6:0/938776133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8415c640 0x7fca840cf040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca8934f700 1 --2- 172.21.15.6:0/938776133 >> v2:172.21.15.61:3300/0 conn(0x7fca840d2820 0x7fca840b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca8934f700 1 -- 172.21.15.6:0/938776133 >> 172.21.15.6:0/938776133 conn(0x7fca840d2df0 msgr2=0x7fca840d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca8934f700 1 -- 172.21.15.6:0/938776133 shutdown_connections 2023-11-28T13:01:12.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.689+0000 7fca8934f700 1 -- 172.21.15.6:0/938776133 wait complete. 2023-11-28T13:01:12.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.691+0000 7fca8934f700 1 Processor -- start 2023-11-28T13:01:12.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.691+0000 7fca8934f700 1 -- start start 2023-11-28T13:01:12.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca8934f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8406bf30 0x7fca84158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca8934f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca840d2820 0x7fca8414f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca8934f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca8415c640 0x7fca8415da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca8934f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca840a9740 con 0x7fca8406bf30 2023-11-28T13:01:12.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca8934f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca840a9a20 con 0x7fca840d2820 2023-11-28T13:01:12.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca8934f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca840a9d00 con 0x7fca8415c640 2023-11-28T13:01:12.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca82ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8406bf30 0x7fca84158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca82ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8406bf30 0x7fca84158610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59314/0 (socket says 172.21.15.6:59314) 2023-11-28T13:01:12.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca82ffd700 1 -- 172.21.15.6:0/381514220 learned_addr learned my addr 172.21.15.6:0/381514220 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:12.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca82ffd700 1 -- 172.21.15.6:0/381514220 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca8415c640 msgr2=0x7fca8415da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca82ffd700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca8415c640 0x7fca8415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca82ffd700 1 -- 172.21.15.6:0/381514220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca840d2820 msgr2=0x7fca8414f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca82ffd700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca840d2820 0x7fca8414f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca82ffd700 1 -- 172.21.15.6:0/381514220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca78009550 con 0x7fca8406bf30 2023-11-28T13:01:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca82ffd700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8406bf30 0x7fca84158610 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fca8408a670 tx=0x7fca78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca6b7fe700 1 -- 172.21.15.6:0/381514220 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78004310 con 0x7fca8406bf30 2023-11-28T13:01:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca6b7fe700 1 -- 172.21.15.6:0/381514220 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7801f070 con 0x7fca8406bf30 2023-11-28T13:01:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca8934f700 1 -- 172.21.15.6:0/381514220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca840d1710 con 0x7fca8406bf30 2023-11-28T13:01:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca6b7fe700 1 -- 172.21.15.6:0/381514220 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780036a0 con 0x7fca8406bf30 2023-11-28T13:01:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.692+0000 7fca8934f700 1 -- 172.21.15.6:0/381514220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca840cd7c0 con 0x7fca8406bf30 2023-11-28T13:01:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.693+0000 7fca8934f700 1 -- 172.21.15.6:0/381514220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca840cf580 con 0x7fca8406bf30 2023-11-28T13:01:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.694+0000 7fca6b7fe700 1 -- 172.21.15.6:0/381514220 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fca7802a020 con 0x7fca8406bf30 2023-11-28T13:01:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.694+0000 7fca6b7fe700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca700682f0 0x7fca7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:12.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.694+0000 7fca6b7fe700 1 -- 172.21.15.6:0/381514220 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fca780875d0 con 0x7fca8406bf30 2023-11-28T13:01:12.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.694+0000 7fca827fc700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca700682f0 0x7fca7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:12.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.695+0000 7fca827fc700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca700682f0 0x7fca7006a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca840ccbd0 tx=0x7fca74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:12.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.699+0000 7fca6b7fe700 1 -- 172.21.15.6:0/381514220 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fca78053b00 con 0x7fca8406bf30 2023-11-28T13:01:12.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.892+0000 7fca8934f700 1 -- 172.21.15.6:0/381514220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fca84074280 con 0x7fca8406bf30 2023-11-28T13:01:12.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.892+0000 7fca6b7fe700 1 -- 172.21.15.6:0/381514220 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fca78057120 con 0x7fca8406bf30 2023-11-28T13:01:12.894 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 -- 172.21.15.6:0/381514220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca700682f0 msgr2=0x7fca7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca700682f0 0x7fca7006a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fca840ccbd0 tx=0x7fca74008040 comp rx=0 tx=0).stop 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 -- 172.21.15.6:0/381514220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8406bf30 msgr2=0x7fca84158610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8406bf30 0x7fca84158610 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fca8408a670 tx=0x7fca78004750 comp rx=0 tx=0).stop 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 -- 172.21.15.6:0/381514220 shutdown_connections 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca700682f0 0x7fca7006a7a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca8415c640 0x7fca8415da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8406bf30 0x7fca84158610 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 --2- 172.21.15.6:0/381514220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca840d2820 0x7fca8414f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 -- 172.21.15.6:0/381514220 >> 172.21.15.6:0/381514220 conn(0x7fca840d2df0 msgr2=0x7fca840b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:12.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 -- 172.21.15.6:0/381514220 shutdown_connections 2023-11-28T13:01:12.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:12.894+0000 7fca8934f700 1 -- 172.21.15.6:0/381514220 wait complete. 2023-11-28T13:01:12.908 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:13.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.033+0000 7fbfe139f700 1 Processor -- start 2023-11-28T13:01:13.035 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.033+0000 7fbfe139f700 1 -- start start 2023-11-28T13:01:13.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.034+0000 7fbfe139f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbfdc06a560 0x7fbfdc06aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.034+0000 7fbfe139f700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfdc0cdec0 con 0x7fbfdc0cceb0 2023-11-28T13:01:13.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.034+0000 7fbfe139f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfdc0c9cd0 con 0x7fbfdc15d1e0 2023-11-28T13:01:13.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.034+0000 7fbfe139f700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfdc06af40 con 0x7fbfdc06a560 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.034+0000 7fbfda7fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbfdc06a560 0x7fbfdc06aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.034+0000 7fbfdb7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fbfdc0cceb0 0x7fbfdc0cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46646/0 (socket says 172.21.15.6:46646) 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.034+0000 7fbfdb7fe700 1 -- 172.21.15.6:0/4071483578 learned_addr learned my addr 172.21.15.6:0/4071483578 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.035+0000 7fbfda7fc700 1 -- 172.21.15.6:0/4071483578 >> v1:172.21.15.61:6789/0 conn(0x7fbfdc15d1e0 legacy=0x7fbfdc15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.035+0000 7fbfda7fc700 1 -- 172.21.15.6:0/4071483578 >> v1:172.21.15.6:6789/0 conn(0x7fbfdc0cceb0 legacy=0x7fbfdc0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.035+0000 7fbfd9ffb700 1 -- 172.21.15.6:0/4071483578 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761592626 0 0) 0x7fbfdc0cdec0 con 0x7fbfdc0cceb0 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.035+0000 7fbfda7fc700 1 -- 172.21.15.6:0/4071483578 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc0c9fb0 con 0x7fbfdc06a560 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.035+0000 7fbfda7fc700 1 --2- 172.21.15.6:0/4071483578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 0x7fbfdc06aa00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009840 tx=0x7fbfcc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce68e40d573d4de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.035+0000 7fbfd9ffb700 1 -- 172.21.15.6:0/4071483578 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc011830 con 0x7fbfdc06a560 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.035+0000 7fbfd9ffb700 1 -- 172.21.15.6:0/4071483578 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc011990 con 0x7fbfdc06a560 2023-11-28T13:01:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.035+0000 7fbfd9ffb700 1 -- 172.21.15.6:0/4071483578 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc011d20 con 0x7fbfdc06a560 2023-11-28T13:01:13.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.035+0000 7fbfe139f700 1 -- 172.21.15.6:0/4071483578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 msgr2=0x7fbfdc06aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.035+0000 7fbfe139f700 1 --2- 172.21.15.6:0/4071483578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 0x7fbfdc06aa00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009840 tx=0x7fbfcc009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:13.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.036+0000 7fbfe139f700 1 -- 172.21.15.6:0/4071483578 shutdown_connections 2023-11-28T13:01:13.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.036+0000 7fbfe139f700 1 --2- 172.21.15.6:0/4071483578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 0x7fbfdc06aa00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.036+0000 7fbfe139f700 1 -- 172.21.15.6:0/4071483578 >> 172.21.15.6:0/4071483578 conn(0x7fbfdc0b0dd0 msgr2=0x7fbfdc0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.036+0000 7fbfe139f700 1 -- 172.21.15.6:0/4071483578 shutdown_connections 2023-11-28T13:01:13.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.036+0000 7fbfe139f700 1 -- 172.21.15.6:0/4071483578 wait complete. 2023-11-28T13:01:13.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.037+0000 7fbfe139f700 1 Processor -- start 2023-11-28T13:01:13.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.037+0000 7fbfe139f700 1 -- start start 2023-11-28T13:01:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfe139f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 0x7fbfdc153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfe139f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbfdc0cceb0 0x7fbfdc1f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfe139f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbfdc15d1e0 0x7fbfdc1f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfe139f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1541a0 con 0x7fbfdc06a560 2023-11-28T13:01:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfe139f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1f3b60 con 0x7fbfdc15d1e0 2023-11-28T13:01:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfe139f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1f3e40 con 0x7fbfdc0cceb0 2023-11-28T13:01:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfdaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 0x7fbfdc153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfdaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 0x7fbfdc153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59328/0 (socket says 172.21.15.6:59328) 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfdaffd700 1 -- 172.21.15.6:0/1024863900 learned_addr learned my addr 172.21.15.6:0/1024863900 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfdaffd700 1 -- 172.21.15.6:0/1024863900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbfdc0cceb0 msgr2=0x7fbfdc1f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfdaffd700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbfdc0cceb0 0x7fbfdc1f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfdaffd700 1 -- 172.21.15.6:0/1024863900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbfdc15d1e0 msgr2=0x7fbfdc1f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfdaffd700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbfdc15d1e0 0x7fbfdc1f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfdaffd700 1 -- 172.21.15.6:0/1024863900 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc0094a0 con 0x7fbfdc06a560 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfdaffd700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 0x7fbfdc153c60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000c4e0 tx=0x7fbfd000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfc37fe700 1 -- 172.21.15.6:0/1024863900 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00052d0 con 0x7fbfdc06a560 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfc37fe700 1 -- 172.21.15.6:0/1024863900 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd0005430 con 0x7fbfdc06a560 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfe139f700 1 -- 172.21.15.6:0/1024863900 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc0ca570 con 0x7fbfdc06a560 2023-11-28T13:01:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfc37fe700 1 -- 172.21.15.6:0/1024863900 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd00038c0 con 0x7fbfdc06a560 2023-11-28T13:01:13.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.038+0000 7fbfe139f700 1 -- 172.21.15.6:0/1024863900 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc0ca290 con 0x7fbfdc06a560 2023-11-28T13:01:13.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.039+0000 7fbfe139f700 1 -- 172.21.15.6:0/1024863900 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfdc0ca850 con 0x7fbfdc06a560 2023-11-28T13:01:13.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.040+0000 7fbfc37fe700 1 -- 172.21.15.6:0/1024863900 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbfd0019030 con 0x7fbfdc06a560 2023-11-28T13:01:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.040+0000 7fbfc37fe700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfc80682f0 0x7fbfc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.040+0000 7fbfc37fe700 1 -- 172.21.15.6:0/1024863900 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbfd0085370 con 0x7fbfdc06a560 2023-11-28T13:01:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.040+0000 7fbfda7fc700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfc80682f0 0x7fbfc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.041+0000 7fbfda7fc700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfc80682f0 0x7fbfc806a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009b90 tx=0x7fbfcc010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.045+0000 7fbfc37fe700 1 -- 172.21.15.6:0/1024863900 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbfd00518a0 con 0x7fbfdc06a560 2023-11-28T13:01:13.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.253+0000 7fbfe139f700 1 -- 172.21.15.6:0/1024863900 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbfdc0675a0 con 0x7fbfdc06a560 2023-11-28T13:01:13.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.254+0000 7fbfc37fe700 1 -- 172.21.15.6:0/1024863900 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbfd0054ec0 con 0x7fbfdc06a560 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 -- 172.21.15.6:0/1024863900 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfc80682f0 msgr2=0x7fbfc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfc80682f0 0x7fbfc806a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009b90 tx=0x7fbfcc010000 comp rx=0 tx=0).stop 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 -- 172.21.15.6:0/1024863900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 msgr2=0x7fbfdc153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 0x7fbfdc153c60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000c4e0 tx=0x7fbfd000c8a0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 -- 172.21.15.6:0/1024863900 shutdown_connections 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbfdc15d1e0 0x7fbfdc1f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbfdc06a560 0x7fbfdc153c60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbfc80682f0 0x7fbfc806a7a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 --2- 172.21.15.6:0/1024863900 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbfdc0cceb0 0x7fbfdc1f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 -- 172.21.15.6:0/1024863900 >> 172.21.15.6:0/1024863900 conn(0x7fbfdc0b0dd0 msgr2=0x7fbfdc06ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 -- 172.21.15.6:0/1024863900 shutdown_connections 2023-11-28T13:01:13.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.256+0000 7fbfe139f700 1 -- 172.21.15.6:0/1024863900 wait complete. 2023-11-28T13:01:13.260 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:01:13.269 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:01:13.269 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:13.291 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:13.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.414+0000 7fbb26a82700 1 Processor -- start 2023-11-28T13:01:13.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.414+0000 7fbb26a82700 1 -- start start 2023-11-28T13:01:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb26a82700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbb20159590 0x7fbb2006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb26a82700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb2015db70 0x7fbb200ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb26a82700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbb200ce890 0x7fbb200ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb26a82700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb200cf250 con 0x7fbb20159590 2023-11-28T13:01:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb26a82700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb2015d120 con 0x7fbb200ce890 2023-11-28T13:01:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb26a82700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2015d6e0 con 0x7fbb2015db70 2023-11-28T13:01:13.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb2481e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbb20159590 0x7fbb2006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb2481e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbb20159590 0x7fbb2006e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59336/0 (socket says 172.21.15.6:59336) 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb2481e700 1 -- 172.21.15.6:0/279647693 learned_addr learned my addr 172.21.15.6:0/279647693 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb2501f700 1 --2- 172.21.15.6:0/279647693 >> v2:172.21.15.61:3300/0 conn(0x7fbb200ce890 0x7fbb200ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.415+0000 7fbb1ffff700 1 --2- 172.21.15.6:0/279647693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb2015db70 0x7fbb200ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb2481e700 1 -- 172.21.15.6:0/279647693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb2015db70 msgr2=0x7fbb200ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb2481e700 1 --2- 172.21.15.6:0/279647693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb2015db70 0x7fbb200ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb2481e700 1 -- 172.21.15.6:0/279647693 >> v2:172.21.15.61:3300/0 conn(0x7fbb200ce890 msgr2=0x7fbb200ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb2481e700 1 --2- 172.21.15.6:0/279647693 >> v2:172.21.15.61:3300/0 conn(0x7fbb200ce890 0x7fbb200ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb2481e700 1 -- 172.21.15.6:0/279647693 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb200aef60 con 0x7fbb20159590 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb2501f700 1 --2- 172.21.15.6:0/279647693 >> v2:172.21.15.61:3300/0 conn(0x7fbb200ce890 0x7fbb200ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb2481e700 1 --2- 172.21.15.6:0/279647693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb20159590 0x7fbb2006e400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbb200b15e0 tx=0x7fbb14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225e3b11a777ae1c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb1f7fe700 1 -- 172.21.15.6:0/279647693 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400b870 con 0x7fbb20159590 2023-11-28T13:01:13.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb1f7fe700 1 -- 172.21.15.6:0/279647693 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1400b9d0 con 0x7fbb20159590 2023-11-28T13:01:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb1f7fe700 1 -- 172.21.15.6:0/279647693 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400bd60 con 0x7fbb20159590 2023-11-28T13:01:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb26a82700 1 -- 172.21.15.6:0/279647693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb20159590 msgr2=0x7fbb2006e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.416+0000 7fbb26a82700 1 --2- 172.21.15.6:0/279647693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb20159590 0x7fbb2006e400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbb200b15e0 tx=0x7fbb14009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.417+0000 7fbb26a82700 1 -- 172.21.15.6:0/279647693 shutdown_connections 2023-11-28T13:01:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.417+0000 7fbb26a82700 1 --2- 172.21.15.6:0/279647693 >> v2:172.21.15.61:3300/0 conn(0x7fbb200ce890 0x7fbb200ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.417+0000 7fbb26a82700 1 --2- 172.21.15.6:0/279647693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb20159590 0x7fbb2006e400 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.417+0000 7fbb26a82700 1 --2- 172.21.15.6:0/279647693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb2015db70 0x7fbb200ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.417+0000 7fbb26a82700 1 -- 172.21.15.6:0/279647693 >> 172.21.15.6:0/279647693 conn(0x7fbb200b1650 msgr2=0x7fbb200b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.417+0000 7fbb26a82700 1 -- 172.21.15.6:0/279647693 shutdown_connections 2023-11-28T13:01:13.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.417+0000 7fbb26a82700 1 -- 172.21.15.6:0/279647693 wait complete. 2023-11-28T13:01:13.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.418+0000 7fbb26a82700 1 Processor -- start 2023-11-28T13:01:13.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.418+0000 7fbb26a82700 1 -- start start 2023-11-28T13:01:13.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb26a82700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb200ce890 0x7fbb2015c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb26a82700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb20159590 0x7fbb200c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb26a82700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb2015db70 0x7fbb2015abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb26a82700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200c8ba0 con 0x7fbb200ce890 2023-11-28T13:01:13.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb26a82700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb200c6170 con 0x7fbb2015db70 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb26a82700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2015b120 con 0x7fbb20159590 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb2481e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb200ce890 0x7fbb2015c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb2481e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb200ce890 0x7fbb2015c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59342/0 (socket says 172.21.15.6:59342) 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb2481e700 1 -- 172.21.15.6:0/1399500050 learned_addr learned my addr 172.21.15.6:0/1399500050 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb2501f700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb2015db70 0x7fbb2015abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb1ffff700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb20159590 0x7fbb200c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb2481e700 1 -- 172.21.15.6:0/1399500050 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb20159590 msgr2=0x7fbb200c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb2481e700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb20159590 0x7fbb200c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb2481e700 1 -- 172.21.15.6:0/1399500050 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb2015db70 msgr2=0x7fbb2015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb2481e700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb2015db70 0x7fbb2015abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb2481e700 1 -- 172.21.15.6:0/1399500050 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb14009550 con 0x7fbb200ce890 2023-11-28T13:01:13.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb2481e700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb200ce890 0x7fbb2015c480 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbb14000c00 tx=0x7fbb14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb1d7fa700 1 -- 172.21.15.6:0/1399500050 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14004540 con 0x7fbb200ce890 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb1d7fa700 1 -- 172.21.15.6:0/1399500050 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb140036a0 con 0x7fbb200ce890 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb26a82700 1 -- 172.21.15.6:0/1399500050 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2015d370 con 0x7fbb200ce890 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.419+0000 7fbb1d7fa700 1 -- 172.21.15.6:0/1399500050 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14003a50 con 0x7fbb200ce890 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.420+0000 7fbb26a82700 1 -- 172.21.15.6:0/1399500050 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb200af240 con 0x7fbb200ce890 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.420+0000 7fbb26a82700 1 -- 172.21.15.6:0/1399500050 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb200aec70 con 0x7fbb200ce890 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.421+0000 7fbb1d7fa700 1 -- 172.21.15.6:0/1399500050 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb1400f050 con 0x7fbb200ce890 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.422+0000 7fbb1d7fa700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb040682f0 0x7fbb0406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.422+0000 7fbb1d7fa700 1 -- 172.21.15.6:0/1399500050 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbb14087df0 con 0x7fbb200ce890 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.422+0000 7fbb1ffff700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb040682f0 0x7fbb0406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.422+0000 7fbb1ffff700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb040682f0 0x7fbb0406a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbb2014d720 tx=0x7fbb0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.425+0000 7fbb1d7fa700 1 -- 172.21.15.6:0/1399500050 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbb140542b0 con 0x7fbb200ce890 2023-11-28T13:01:13.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.624+0000 7fbb26a82700 1 -- 172.21.15.6:0/1399500050 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb200cf660 con 0x7fbb200ce890 2023-11-28T13:01:13.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.626+0000 7fbb1d7fa700 1 -- 172.21.15.6:0/1399500050 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19690 (secure 0 0 0) 0x7fbb140578d0 con 0x7fbb200ce890 2023-11-28T13:01:13.627 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:13.627 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:10.053012+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:13.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 -- 172.21.15.6:0/1399500050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb040682f0 msgr2=0x7fbb0406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb040682f0 0x7fbb0406a7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbb2014d720 tx=0x7fbb0c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:13.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 -- 172.21.15.6:0/1399500050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb200ce890 msgr2=0x7fbb2015c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb200ce890 0x7fbb2015c480 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbb14000c00 tx=0x7fbb14004750 comp rx=0 tx=0).stop 2023-11-28T13:01:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 -- 172.21.15.6:0/1399500050 shutdown_connections 2023-11-28T13:01:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb2015db70 0x7fbb2015abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb200ce890 0x7fbb2015c480 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb040682f0 0x7fbb0406a7a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 --2- 172.21.15.6:0/1399500050 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb20159590 0x7fbb200c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 -- 172.21.15.6:0/1399500050 >> 172.21.15.6:0/1399500050 conn(0x7fbb200b1650 msgr2=0x7fbb201549c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 -- 172.21.15.6:0/1399500050 shutdown_connections 2023-11-28T13:01:13.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.628+0000 7fbb26a82700 1 -- 172.21.15.6:0/1399500050 wait complete. 2023-11-28T13:01:13.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:13.642 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.768+0000 7f3217dea700 1 Processor -- start 2023-11-28T13:01:13.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.768+0000 7f3217dea700 1 -- start start 2023-11-28T13:01:13.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3217dea700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3210159500 0x7f32100cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3217dea700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f32100b1d40 0x7f32100ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3217dea700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321015e040 con 0x7f321015dbf0 2023-11-28T13:01:13.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3217dea700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32100cede0 con 0x7f32100b1d40 2023-11-28T13:01:13.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3217dea700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32100ceff0 con 0x7f3210159500 2023-11-28T13:01:13.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3216387700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f32100b1d40 0x7f32100ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3215385700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f321015dbf0 0x7f32100b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46670/0 (socket says 172.21.15.6:46670) 2023-11-28T13:01:13.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3215385700 1 -- 172.21.15.6:0/738588239 learned_addr learned my addr 172.21.15.6:0/738588239 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:13.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3216387700 1 -- 172.21.15.6:0/738588239 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3210159500 msgr2=0x7f32100cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3215b86700 1 --2- 172.21.15.6:0/738588239 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3210159500 0x7f32100cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3216387700 1 --2- 172.21.15.6:0/738588239 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3210159500 0x7f32100cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3216387700 1 -- 172.21.15.6:0/738588239 >> v1:172.21.15.6:6789/0 conn(0x7f321015dbf0 legacy=0x7f32100b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3214b84700 1 -- 172.21.15.6:0/738588239 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146307951 0 0) 0x7f321015e040 con 0x7f321015dbf0 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3216387700 1 -- 172.21.15.6:0/738588239 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100cf2d0 con 0x7f32100b1d40 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.769+0000 7f3215b86700 1 --2- 172.21.15.6:0/738588239 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3210159500 0x7f32100cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3216387700 1 --2- 172.21.15.6:0/738588239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32100b1d40 0x7f32100ce8a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3208009d90 tx=0x7f320800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62ee0c0e69326fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3214b84700 1 -- 172.21.15.6:0/738588239 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320800e070 con 0x7f32100b1d40 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3214b84700 1 -- 172.21.15.6:0/738588239 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320800bbd0 con 0x7f32100b1d40 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3214b84700 1 -- 172.21.15.6:0/738588239 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208012430 con 0x7f32100b1d40 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3217dea700 1 -- 172.21.15.6:0/738588239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32100b1d40 msgr2=0x7f32100ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3217dea700 1 --2- 172.21.15.6:0/738588239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32100b1d40 0x7f32100ce8a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3208009d90 tx=0x7f320800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:13.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3217dea700 1 -- 172.21.15.6:0/738588239 shutdown_connections 2023-11-28T13:01:13.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3217dea700 1 --2- 172.21.15.6:0/738588239 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f32100b1d40 0x7f32100ce8a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3217dea700 1 --2- 172.21.15.6:0/738588239 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3210159500 0x7f32100cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3217dea700 1 -- 172.21.15.6:0/738588239 >> 172.21.15.6:0/738588239 conn(0x7f32100cc630 msgr2=0x7f32100cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3217dea700 1 -- 172.21.15.6:0/738588239 shutdown_connections 2023-11-28T13:01:13.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.770+0000 7f3217dea700 1 -- 172.21.15.6:0/738588239 wait complete. 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.772+0000 7f3217dea700 1 Processor -- start 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.772+0000 7f3217dea700 1 -- start start 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.772+0000 7f3217dea700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32100b1d40 0x7f321015b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.772+0000 7f3217dea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3210159500 0x7f3210152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.772+0000 7f3217dea700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f321015dbf0 0x7f32100b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.772+0000 7f3217dea700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100b3c90 con 0x7f3210159500 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.772+0000 7f3217dea700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321015be20 con 0x7f321015dbf0 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.772+0000 7f3217dea700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32100b33f0 con 0x7f32100b1d40 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.772+0000 7f3215385700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3210159500 0x7f3210152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3215385700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3210159500 0x7f3210152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59358/0 (socket says 172.21.15.6:59358) 2023-11-28T13:01:13.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3215385700 1 -- 172.21.15.6:0/4267906138 learned_addr learned my addr 172.21.15.6:0/4267906138 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3215b86700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32100b1d40 0x7f321015b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3216387700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f321015dbf0 0x7f32100b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3215385700 1 -- 172.21.15.6:0/4267906138 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32100b1d40 msgr2=0x7f321015b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3215385700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32100b1d40 0x7f321015b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3215385700 1 -- 172.21.15.6:0/4267906138 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f321015dbf0 msgr2=0x7f32100b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3215385700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f321015dbf0 0x7f32100b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3215385700 1 -- 172.21.15.6:0/4267906138 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3208009a70 con 0x7f3210159500 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3215385700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3210159500 0x7f3210152f60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f320c0011e0 tx=0x7f320c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f32067fc700 1 -- 172.21.15.6:0/4267906138 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c00ee10 con 0x7f3210159500 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f32067fc700 1 -- 172.21.15.6:0/4267906138 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c004a80 con 0x7f3210159500 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3217dea700 1 -- 172.21.15.6:0/4267906138 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321015d3f0 con 0x7f3210159500 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f32067fc700 1 -- 172.21.15.6:0/4267906138 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c004e30 con 0x7f3210159500 2023-11-28T13:01:13.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.773+0000 7f3217dea700 1 -- 172.21.15.6:0/4267906138 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321015d110 con 0x7f3210159500 2023-11-28T13:01:13.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.774+0000 7f3217dea700 1 -- 172.21.15.6:0/4267906138 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321015d6d0 con 0x7f3210159500 2023-11-28T13:01:13.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.774+0000 7f32067fc700 1 -- 172.21.15.6:0/4267906138 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f320c0070d0 con 0x7f3210159500 2023-11-28T13:01:13.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.775+0000 7f32067fc700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31f40682f0 0x7f31f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:13.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.775+0000 7f32067fc700 1 -- 172.21.15.6:0/4267906138 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f320c088510 con 0x7f3210159500 2023-11-28T13:01:13.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.775+0000 7f3215b86700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31f40682f0 0x7f31f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:13.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.775+0000 7f3215b86700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31f40682f0 0x7f31f406a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3200006fd0 tx=0x7f3200008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:13.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.779+0000 7f32067fc700 1 -- 172.21.15.6:0/4267906138 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f320c054a40 con 0x7f3210159500 2023-11-28T13:01:13.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.970+0000 7f3217dea700 1 -- 172.21.15.6:0/4267906138 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f32100b3ad0 con 0x7f3210159500 2023-11-28T13:01:13.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.970+0000 7f32067fc700 1 -- 172.21.15.6:0/4267906138 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f320c015020 con 0x7f3210159500 2023-11-28T13:01:13.972 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:13.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 -- 172.21.15.6:0/4267906138 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31f40682f0 msgr2=0x7f31f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31f40682f0 0x7f31f406a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3200006fd0 tx=0x7f3200008040 comp rx=0 tx=0).stop 2023-11-28T13:01:13.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 -- 172.21.15.6:0/4267906138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3210159500 msgr2=0x7f3210152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:13.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3210159500 0x7f3210152f60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f320c0011e0 tx=0x7f320c00ecf0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 -- 172.21.15.6:0/4267906138 shutdown_connections 2023-11-28T13:01:13.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f321015dbf0 0x7f32100b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31f40682f0 0x7f31f406a7a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f32100b1d40 0x7f321015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 --2- 172.21.15.6:0/4267906138 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3210159500 0x7f3210152f60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:13.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 -- 172.21.15.6:0/4267906138 >> 172.21.15.6:0/4267906138 conn(0x7f32100cc630 msgr2=0x7f32100cbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:13.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 -- 172.21.15.6:0/4267906138 shutdown_connections 2023-11-28T13:01:13.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:13.972+0000 7f3217dea700 1 -- 172.21.15.6:0/4267906138 wait complete. 2023-11-28T13:01:13.986 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:14.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.113+0000 7f398cde6700 1 Processor -- start 2023-11-28T13:01:14.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.113+0000 7f398cde6700 1 -- start start 2023-11-28T13:01:14.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.114+0000 7f398cde6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f398815a490 0x7f398806f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.114+0000 7f398cde6700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398815a860 con 0x7f398806dd40 2023-11-28T13:01:14.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.114+0000 7f398cde6700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398806b510 con 0x7f39880b2150 2023-11-28T13:01:14.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.114+0000 7f398cde6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39880c9a80 con 0x7f398815a490 2023-11-28T13:01:14.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.114+0000 7f3986d9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f398806dd40 0x7f39880ca520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46688/0 (socket says 172.21.15.6:46688) 2023-11-28T13:01:14.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.114+0000 7f3986d9d700 1 -- 172.21.15.6:0/948918671 learned_addr learned my addr 172.21.15.6:0/948918671 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:14.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.114+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994779698 0 0) 0x7f398815a860 con 0x7f398806dd40 2023-11-28T13:01:14.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.114+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39700035b0 con 0x7f398806dd40 2023-11-28T13:01:14.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.114+0000 7f398659c700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f398815a490 0x7f398806f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1438850016 0 0) 0x7f39700035b0 con 0x7f398806dd40 2023-11-28T13:01:14.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f398815a490 msgr2=0x7f398806f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:14.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f398815a490 0x7f398806f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 >> v1:172.21.15.61:6789/0 conn(0x7f39880b2150 legacy=0x7f398806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398806b230 con 0x7f398806dd40 2023-11-28T13:01:14.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f39800019f0 con 0x7f398806dd40 2023-11-28T13:01:14.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 >> v1:172.21.15.6:6789/0 conn(0x7f398806dd40 legacy=0x7f39880ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3970003a30 0x7f3970003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f398815a490 0x7f3970004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3970004d90 0x7f3970005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39880c9a80 con 0x7f3970004d90 2023-11-28T13:01:14.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397000bbd0 con 0x7f3970003a30 2023-11-28T13:01:14.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397000bd90 con 0x7f398815a490 2023-11-28T13:01:14.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f3986d9d700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3970004d90 0x7f3970005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398659c700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3970003a30 0x7f3970003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f3985d9b700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f398815a490 0x7f3970004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f3986d9d700 1 -- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f398815a490 msgr2=0x7f3970004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:14.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f3986d9d700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f398815a490 0x7f3970004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f3986d9d700 1 -- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3970003a30 msgr2=0x7f3970003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:14.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f3986d9d700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3970003a30 0x7f3970003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f3986d9d700 1 -- 172.21.15.6:0/948918671 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700035b0 con 0x7f3970004d90 2023-11-28T13:01:14.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398659c700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3970003a30 0x7f3970003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:14.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f3985d9b700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f398815a490 0x7f3970004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:14.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f3986d9d700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3970004d90 0x7f3970005200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f39800013d0 tx=0x7f3970005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace12bdeb28c4856 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:14.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980012600 con 0x7f3970004d90 2023-11-28T13:01:14.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39800127a0 con 0x7f3970004d90 2023-11-28T13:01:14.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.115+0000 7f398559a700 1 -- 172.21.15.6:0/948918671 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3980012b50 con 0x7f3970004d90 2023-11-28T13:01:14.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.116+0000 7f398cde6700 1 -- 172.21.15.6:0/948918671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3970004d90 msgr2=0x7f3970005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.116+0000 7f398cde6700 1 --2- 172.21.15.6:0/948918671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3970004d90 0x7f3970005200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f39800013d0 tx=0x7f3970005880 comp rx=0 tx=0).stop 2023-11-28T13:01:14.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.116+0000 7f3986d9d700 1 -- 172.21.15.6:0/948918671 reap_dead start 2023-11-28T13:01:14.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.116+0000 7f398cde6700 1 -- 172.21.15.6:0/948918671 shutdown_connections 2023-11-28T13:01:14.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.116+0000 7f398cde6700 1 -- 172.21.15.6:0/948918671 >> 172.21.15.6:0/948918671 conn(0x7f39880b0df0 msgr2=0x7f39880b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:14.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.116+0000 7f398cde6700 1 -- 172.21.15.6:0/948918671 shutdown_connections 2023-11-28T13:01:14.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.116+0000 7f398cde6700 1 -- 172.21.15.6:0/948918671 wait complete. 2023-11-28T13:01:14.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.118+0000 7f398cde6700 1 Processor -- start 2023-11-28T13:01:14.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.118+0000 7f398cde6700 1 -- start start 2023-11-28T13:01:14.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f398cde6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3970004d90 0x7f3988150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f398cde6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398814f680 0x7f398814faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f398cde6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3988156760 0x7f3988156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f398cde6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3988150b50 con 0x7f398814f680 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f398cde6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3988157700 con 0x7f3988156760 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f398cde6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3988151100 con 0x7f3970004d90 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f3985d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398814f680 0x7f398814faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f3985d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398814f680 0x7f398814faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59378/0 (socket says 172.21.15.6:59378) 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f3985d9b700 1 -- 172.21.15.6:0/2872022798 learned_addr learned my addr 172.21.15.6:0/2872022798 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f398659c700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3970004d90 0x7f3988150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f3986d9d700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3988156760 0x7f3988156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f3985d9b700 1 -- 172.21.15.6:0/2872022798 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3970004d90 msgr2=0x7f3988150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f3985d9b700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3970004d90 0x7f3988150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f3985d9b700 1 -- 172.21.15.6:0/2872022798 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3988156760 msgr2=0x7f3988156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f3985d9b700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3988156760 0x7f3988156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f3985d9b700 1 -- 172.21.15.6:0/2872022798 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39880ca000 con 0x7f398814f680 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.119+0000 7f3985d9b700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398814f680 0x7f398814faf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f397c00e7e0 tx=0x7f397c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.120+0000 7f3976ffd700 1 -- 172.21.15.6:0/2872022798 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c017070 con 0x7f398814f680 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.120+0000 7f3976ffd700 1 -- 172.21.15.6:0/2872022798 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c0052e0 con 0x7f398814f680 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.120+0000 7f398cde6700 1 -- 172.21.15.6:0/2872022798 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39880c9d20 con 0x7f398814f680 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.120+0000 7f3976ffd700 1 -- 172.21.15.6:0/2872022798 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c005450 con 0x7f398814f680 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.120+0000 7f398cde6700 1 -- 172.21.15.6:0/2872022798 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398806ae90 con 0x7f398814f680 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.121+0000 7f398cde6700 1 -- 172.21.15.6:0/2872022798 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39880739f0 con 0x7f398814f680 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.121+0000 7f3976ffd700 1 -- 172.21.15.6:0/2872022798 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f397c006860 con 0x7f398814f680 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.121+0000 7f3976ffd700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39680682f0 0x7f396806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.122+0000 7f3976ffd700 1 -- 172.21.15.6:0/2872022798 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f397c085790 con 0x7f398814f680 2023-11-28T13:01:14.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.122+0000 7f398659c700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39680682f0 0x7f396806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.122+0000 7f398659c700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39680682f0 0x7f396806a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f397800c780 tx=0x7f3978008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:14.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.126+0000 7f3976ffd700 1 -- 172.21.15.6:0/2872022798 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f397c051cc0 con 0x7f398814f680 2023-11-28T13:01:14.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.319+0000 7f398cde6700 1 -- 172.21.15.6:0/2872022798 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39880cba50 con 0x7f398814f680 2023-11-28T13:01:14.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.319+0000 7f3976ffd700 1 -- 172.21.15.6:0/2872022798 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f397c0552e0 con 0x7f398814f680 2023-11-28T13:01:14.320 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 -- 172.21.15.6:0/2872022798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39680682f0 msgr2=0x7f396806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39680682f0 0x7f396806a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f397800c780 tx=0x7f3978008040 comp rx=0 tx=0).stop 2023-11-28T13:01:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 -- 172.21.15.6:0/2872022798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398814f680 msgr2=0x7f398814faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398814f680 0x7f398814faf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f397c00e7e0 tx=0x7f397c00eba0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 -- 172.21.15.6:0/2872022798 shutdown_connections 2023-11-28T13:01:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3988156760 0x7f3988156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39680682f0 0x7f396806a7a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3970004d90 0x7f3988150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 --2- 172.21.15.6:0/2872022798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398814f680 0x7f398814faf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 -- 172.21.15.6:0/2872022798 >> 172.21.15.6:0/2872022798 conn(0x7f39880b0df0 msgr2=0x7f398806b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 -- 172.21.15.6:0/2872022798 shutdown_connections 2023-11-28T13:01:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.321+0000 7f398cde6700 1 -- 172.21.15.6:0/2872022798 wait complete. 2023-11-28T13:01:14.335 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.462+0000 7fd6cda69700 1 Processor -- start 2023-11-28T13:01:14.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.462+0000 7fd6cda69700 1 -- start start 2023-11-28T13:01:14.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.462+0000 7fd6cda69700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6c80b1d40 0x7fd6c80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.462+0000 7fd6cda69700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6c815dbf0 0x7fd6c80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.462+0000 7fd6cda69700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80ceff0 con 0x7fd6c8159500 2023-11-28T13:01:14.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.462+0000 7fd6cda69700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c815e090 con 0x7fd6c80b1d40 2023-11-28T13:01:14.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.462+0000 7fd6cda69700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80cede0 con 0x7fd6c815dbf0 2023-11-28T13:01:14.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c6ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6c80b1d40 0x7fd6c80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c77fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd6c8159500 0x7fd6c80b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46704/0 (socket says 172.21.15.6:46704) 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c77fe700 1 -- 172.21.15.6:0/889031276 learned_addr learned my addr 172.21.15.6:0/889031276 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c67fc700 1 -- 172.21.15.6:0/889031276 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435000195 0 0) 0x7fd6c80ceff0 con 0x7fd6c8159500 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c67fc700 1 -- 172.21.15.6:0/889031276 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b40035b0 con 0x7fd6c8159500 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c6ffd700 1 -- 172.21.15.6:0/889031276 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6c815dbf0 msgr2=0x7fd6c80ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c7fff700 1 --2- 172.21.15.6:0/889031276 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6c815dbf0 0x7fd6c80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c6ffd700 1 --2- 172.21.15.6:0/889031276 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6c815dbf0 0x7fd6c80ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c6ffd700 1 -- 172.21.15.6:0/889031276 >> v1:172.21.15.6:6789/0 conn(0x7fd6c8159500 legacy=0x7fd6c80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c6ffd700 1 -- 172.21.15.6:0/889031276 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80cf2d0 con 0x7fd6c80b1d40 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c6ffd700 1 --2- 172.21.15.6:0/889031276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 0x7fd6c80cd140 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009840 tx=0x7fd6b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f41a101fe5de879 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:14.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c67fc700 1 -- 172.21.15.6:0/889031276 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0011830 con 0x7fd6c80b1d40 2023-11-28T13:01:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c67fc700 1 -- 172.21.15.6:0/889031276 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b0011990 con 0x7fd6c80b1d40 2023-11-28T13:01:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.463+0000 7fd6c67fc700 1 -- 172.21.15.6:0/889031276 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0011d20 con 0x7fd6c80b1d40 2023-11-28T13:01:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.464+0000 7fd6cda69700 1 -- 172.21.15.6:0/889031276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 msgr2=0x7fd6c80cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.464+0000 7fd6cda69700 1 --2- 172.21.15.6:0/889031276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 0x7fd6c80cd140 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009840 tx=0x7fd6b0009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.464+0000 7fd6cda69700 1 -- 172.21.15.6:0/889031276 shutdown_connections 2023-11-28T13:01:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.464+0000 7fd6cda69700 1 --2- 172.21.15.6:0/889031276 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6c815dbf0 0x7fd6c80ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.464+0000 7fd6cda69700 1 --2- 172.21.15.6:0/889031276 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 0x7fd6c80cd140 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.464+0000 7fd6cda69700 1 -- 172.21.15.6:0/889031276 >> 172.21.15.6:0/889031276 conn(0x7fd6c80cc630 msgr2=0x7fd6c80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.464+0000 7fd6cda69700 1 -- 172.21.15.6:0/889031276 shutdown_connections 2023-11-28T13:01:14.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.464+0000 7fd6cda69700 1 -- 172.21.15.6:0/889031276 wait complete. 2023-11-28T13:01:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.465+0000 7fd6cda69700 1 Processor -- start 2023-11-28T13:01:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.465+0000 7fd6cda69700 1 -- start start 2023-11-28T13:01:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6cda69700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 0x7fd6c80c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6cda69700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6c8159500 0x7fd6c80b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6cda69700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6c815dbf0 0x7fd6c8145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6cda69700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80c6520 con 0x7fd6c80b1d40 2023-11-28T13:01:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6cda69700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c815a170 con 0x7fd6c815dbf0 2023-11-28T13:01:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6cda69700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c81598d0 con 0x7fd6c8159500 2023-11-28T13:01:14.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6c77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 0x7fd6c80c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6c77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 0x7fd6c80c5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59390/0 (socket says 172.21.15.6:59390) 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6c77fe700 1 -- 172.21.15.6:0/1765234282 learned_addr learned my addr 172.21.15.6:0/1765234282 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6c7fff700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6c815dbf0 0x7fd6c8145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6c77fe700 1 -- 172.21.15.6:0/1765234282 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6c8159500 msgr2=0x7fd6c80b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6c6ffd700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6c8159500 0x7fd6c80b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6c77fe700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6c8159500 0x7fd6c80b4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6c77fe700 1 -- 172.21.15.6:0/1765234282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6c815dbf0 msgr2=0x7fd6c8145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6c77fe700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6c815dbf0 0x7fd6c8145bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.466+0000 7fd6c77fe700 1 -- 172.21.15.6:0/1765234282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b00094a0 con 0x7fd6c80b1d40 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.467+0000 7fd6c77fe700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 0x7fd6c80c5fe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800cbf0 tx=0x7fd6b800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:14.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.467+0000 7fd6bffff700 1 -- 172.21.15.6:0/1765234282 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800abe0 con 0x7fd6c80b1d40 2023-11-28T13:01:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.467+0000 7fd6bffff700 1 -- 172.21.15.6:0/1765234282 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b800ad40 con 0x7fd6c80b1d40 2023-11-28T13:01:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.467+0000 7fd6cda69700 1 -- 172.21.15.6:0/1765234282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c815d3f0 con 0x7fd6c80b1d40 2023-11-28T13:01:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.467+0000 7fd6bffff700 1 -- 172.21.15.6:0/1765234282 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8012620 con 0x7fd6c80b1d40 2023-11-28T13:01:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.467+0000 7fd6cda69700 1 -- 172.21.15.6:0/1765234282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c815d110 con 0x7fd6c80b1d40 2023-11-28T13:01:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.468+0000 7fd6cda69700 1 -- 172.21.15.6:0/1765234282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c815d6d0 con 0x7fd6c80b1d40 2023-11-28T13:01:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.468+0000 7fd6bffff700 1 -- 172.21.15.6:0/1765234282 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd6b8019030 con 0x7fd6c80b1d40 2023-11-28T13:01:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.469+0000 7fd6bffff700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6a80682f0 0x7fd6a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.469+0000 7fd6bffff700 1 -- 172.21.15.6:0/1765234282 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd6b8085d90 con 0x7fd6c80b1d40 2023-11-28T13:01:14.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.469+0000 7fd6c6ffd700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6a80682f0 0x7fd6a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.469+0000 7fd6c6ffd700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6a80682f0 0x7fd6a806a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009840 tx=0x7fd6b0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:14.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.473+0000 7fd6bffff700 1 -- 172.21.15.6:0/1765234282 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6b8052230 con 0x7fd6c80b1d40 2023-11-28T13:01:14.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.668+0000 7fd6cda69700 1 -- 172.21.15.6:0/1765234282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6c80739f0 con 0x7fd6c80b1d40 2023-11-28T13:01:14.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.668+0000 7fd6bffff700 1 -- 172.21.15.6:0/1765234282 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6b80170a0 con 0x7fd6c80b1d40 2023-11-28T13:01:14.670 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:14.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 -- 172.21.15.6:0/1765234282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6a80682f0 msgr2=0x7fd6a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6a80682f0 0x7fd6a806a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009840 tx=0x7fd6b0008030 comp rx=0 tx=0).stop 2023-11-28T13:01:14.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 -- 172.21.15.6:0/1765234282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 msgr2=0x7fd6c80c5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 0x7fd6c80c5fe0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800cbf0 tx=0x7fd6b800a060 comp rx=0 tx=0).stop 2023-11-28T13:01:14.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 -- 172.21.15.6:0/1765234282 shutdown_connections 2023-11-28T13:01:14.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6c815dbf0 0x7fd6c8145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6c80b1d40 0x7fd6c80c5fe0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6a80682f0 0x7fd6a806a7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 --2- 172.21.15.6:0/1765234282 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6c8159500 0x7fd6c80b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 -- 172.21.15.6:0/1765234282 >> 172.21.15.6:0/1765234282 conn(0x7fd6c80cc630 msgr2=0x7fd6c80cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:14.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 -- 172.21.15.6:0/1765234282 shutdown_connections 2023-11-28T13:01:14.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.670+0000 7fd6cda69700 1 -- 172.21.15.6:0/1765234282 wait complete. 2023-11-28T13:01:14.684 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:14.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.810+0000 7feba7717700 1 Processor -- start 2023-11-28T13:01:14.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.810+0000 7feba7717700 1 -- start start 2023-11-28T13:01:14.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba7717700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7feba0159500 0x7feba00cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba7717700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feba015dbf0 0x7feba00ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba7717700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7feba00b1630 0x7feba00b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba7717700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba00b2010 con 0x7feba00b1630 2023-11-28T13:01:14.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba7717700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba00cede0 con 0x7feba0159500 2023-11-28T13:01:14.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba7717700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba015e040 con 0x7feba015dbf0 2023-11-28T13:01:14.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba5cb4700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7feba00b1630 0x7feba00b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba5cb4700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7feba00b1630 0x7feba00b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59402/0 (socket says 172.21.15.6:59402) 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba5cb4700 1 -- 172.21.15.6:0/630905743 learned_addr learned my addr 172.21.15.6:0/630905743 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba4cb2700 1 --2- 172.21.15.6:0/630905743 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feba015dbf0 0x7feba00ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba54b3700 1 --2- 172.21.15.6:0/630905743 >> v2:172.21.15.61:3300/0 conn(0x7feba0159500 0x7feba00cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba5cb4700 1 -- 172.21.15.6:0/630905743 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feba015dbf0 msgr2=0x7feba00ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba5cb4700 1 --2- 172.21.15.6:0/630905743 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feba015dbf0 0x7feba00ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba5cb4700 1 -- 172.21.15.6:0/630905743 >> v2:172.21.15.61:3300/0 conn(0x7feba0159500 msgr2=0x7feba00cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba5cb4700 1 --2- 172.21.15.6:0/630905743 >> v2:172.21.15.61:3300/0 conn(0x7feba0159500 0x7feba00cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba5cb4700 1 -- 172.21.15.6:0/630905743 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba00ceff0 con 0x7feba00b1630 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feba5cb4700 1 --2- 172.21.15.6:0/630905743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 0x7feba00b1ad0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feb9800d150 tx=0x7feb9800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9c9268c4312e13 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feb93fff700 1 -- 172.21.15.6:0/630905743 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb98010070 con 0x7feba00b1630 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feb93fff700 1 -- 172.21.15.6:0/630905743 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9800dd80 con 0x7feba00b1630 2023-11-28T13:01:14.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.811+0000 7feb93fff700 1 -- 172.21.15.6:0/630905743 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9800b310 con 0x7feba00b1630 2023-11-28T13:01:14.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.812+0000 7feba7717700 1 -- 172.21.15.6:0/630905743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 msgr2=0x7feba00b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.812+0000 7feba7717700 1 --2- 172.21.15.6:0/630905743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 0x7feba00b1ad0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7feb9800d150 tx=0x7feb9800d460 comp rx=0 tx=0).stop 2023-11-28T13:01:14.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.812+0000 7feba7717700 1 -- 172.21.15.6:0/630905743 shutdown_connections 2023-11-28T13:01:14.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.812+0000 7feba7717700 1 --2- 172.21.15.6:0/630905743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 0x7feba00b1ad0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.812+0000 7feba7717700 1 --2- 172.21.15.6:0/630905743 >> v2:172.21.15.61:3300/0 conn(0x7feba0159500 0x7feba00cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.812+0000 7feba7717700 1 --2- 172.21.15.6:0/630905743 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feba015dbf0 0x7feba00ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.812+0000 7feba7717700 1 -- 172.21.15.6:0/630905743 >> 172.21.15.6:0/630905743 conn(0x7feba00cc630 msgr2=0x7feba00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:14.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.812+0000 7feba7717700 1 -- 172.21.15.6:0/630905743 shutdown_connections 2023-11-28T13:01:14.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.812+0000 7feba7717700 1 -- 172.21.15.6:0/630905743 wait complete. 2023-11-28T13:01:14.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.813+0000 7feba7717700 1 Processor -- start 2023-11-28T13:01:14.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.813+0000 7feba7717700 1 -- start start 2023-11-28T13:01:14.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba7717700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 0x7feba01513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba7717700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feba0159500 0x7feba00ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba7717700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feba015dbf0 0x7feba00abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba7717700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba015a960 con 0x7feba00b1630 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba7717700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba00ac430 con 0x7feba0159500 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba7717700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba00ac710 con 0x7feba015dbf0 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba54b3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 0x7feba01513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba54b3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 0x7feba01513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59404/0 (socket says 172.21.15.6:59404) 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba54b3700 1 -- 172.21.15.6:0/4156175884 learned_addr learned my addr 172.21.15.6:0/4156175884 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba4cb2700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feba0159500 0x7feba00ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba5cb4700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feba015dbf0 0x7feba00abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba54b3700 1 -- 172.21.15.6:0/4156175884 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feba015dbf0 msgr2=0x7feba00abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba54b3700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feba015dbf0 0x7feba00abe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba54b3700 1 -- 172.21.15.6:0/4156175884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feba0159500 msgr2=0x7feba00ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba54b3700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feba0159500 0x7feba00ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba54b3700 1 -- 172.21.15.6:0/4156175884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9800bd40 con 0x7feba00b1630 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.814+0000 7feba4cb2700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feba0159500 0x7feba00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.815+0000 7feba54b3700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 0x7feba01513f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feb9400e810 tx=0x7feb9400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.815+0000 7feb91ffb700 1 -- 172.21.15.6:0/4156175884 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94013610 con 0x7feba00b1630 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.815+0000 7feb91ffb700 1 -- 172.21.15.6:0/4156175884 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb94013770 con 0x7feba00b1630 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.815+0000 7feba7717700 1 -- 172.21.15.6:0/4156175884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba015d110 con 0x7feba00b1630 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.815+0000 7feb91ffb700 1 -- 172.21.15.6:0/4156175884 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb94013b00 con 0x7feba00b1630 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.815+0000 7feba7717700 1 -- 172.21.15.6:0/4156175884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba00cf2d0 con 0x7feba00b1630 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.816+0000 7feba7717700 1 -- 172.21.15.6:0/4156175884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba015d3f0 con 0x7feba00b1630 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.816+0000 7feb91ffb700 1 -- 172.21.15.6:0/4156175884 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feb94023070 con 0x7feba00b1630 2023-11-28T13:01:14.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.817+0000 7feb91ffb700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb840682f0 0x7feb8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:14.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.817+0000 7feb91ffb700 1 -- 172.21.15.6:0/4156175884 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7feb94085bf0 con 0x7feba00b1630 2023-11-28T13:01:14.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.817+0000 7feba4cb2700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb840682f0 0x7feb8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:14.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.817+0000 7feba4cb2700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb840682f0 0x7feb8406a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feba00ce690 tx=0x7feb9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:14.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:14.821+0000 7feb91ffb700 1 -- 172.21.15.6:0/4156175884 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feb94052120 con 0x7feba00b1630 2023-11-28T13:01:15.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.013+0000 7feba7717700 1 -- 172.21.15.6:0/4156175884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feba015ac40 con 0x7feba00b1630 2023-11-28T13:01:15.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.013+0000 7feb91ffb700 1 -- 172.21.15.6:0/4156175884 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7feb9400f040 con 0x7feba00b1630 2023-11-28T13:01:15.015 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 -- 172.21.15.6:0/4156175884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb840682f0 msgr2=0x7feb8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb840682f0 0x7feb8406a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feba00ce690 tx=0x7feb9c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 -- 172.21.15.6:0/4156175884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 msgr2=0x7feba01513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 0x7feba01513f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feb9400e810 tx=0x7feb9400ebd0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 -- 172.21.15.6:0/4156175884 shutdown_connections 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feb840682f0 0x7feb8406a7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feba015dbf0 0x7feba00abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feba00b1630 0x7feba01513f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 --2- 172.21.15.6:0/4156175884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feba0159500 0x7feba00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 -- 172.21.15.6:0/4156175884 >> 172.21.15.6:0/4156175884 conn(0x7feba00cc630 msgr2=0x7feba00cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:15.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 -- 172.21.15.6:0/4156175884 shutdown_connections 2023-11-28T13:01:15.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.015+0000 7feba7717700 1 -- 172.21.15.6:0/4156175884 wait complete. 2023-11-28T13:01:15.028 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:15.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.156+0000 7f40529f9700 1 Processor -- start 2023-11-28T13:01:15.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.156+0000 7f40529f9700 1 -- start start 2023-11-28T13:01:15.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f40529f9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f404c0b2150 0x7f404c06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f40529f9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f404c06dd40 0x7f404c0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f40529f9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c15a860 con 0x7f404c15a490 2023-11-28T13:01:15.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f40529f9700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404c06b510 con 0x7f404c06dd40 2023-11-28T13:01:15.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f40529f9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404c0c9a80 con 0x7f404c0b2150 2023-11-28T13:01:15.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f4050f96700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f404c06dd40 0x7f404c0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f4050f96700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f404c06dd40 0x7f404c0ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59416/0 (socket says 172.21.15.6:59416) 2023-11-28T13:01:15.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f4050f96700 1 -- 172.21.15.6:0/3592717689 learned_addr learned my addr 172.21.15.6:0/3592717689 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:15.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f404b7fe700 1 --2- 172.21.15.6:0/3592717689 >> v2:172.21.15.61:3300/0 conn(0x7f404c0b2150 0x7f404c06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f4050f96700 1 -- 172.21.15.6:0/3592717689 >> v2:172.21.15.61:3300/0 conn(0x7f404c0b2150 msgr2=0x7f404c06f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f4050f96700 1 --2- 172.21.15.6:0/3592717689 >> v2:172.21.15.61:3300/0 conn(0x7f404c0b2150 0x7f404c06f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f4050f96700 1 -- 172.21.15.6:0/3592717689 >> v1:172.21.15.61:6789/0 conn(0x7f404c15a490 legacy=0x7f404c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f4050f96700 1 -- 172.21.15.6:0/3592717689 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c06b230 con 0x7f404c06dd40 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.157+0000 7f4050f96700 1 --2- 172.21.15.6:0/3592717689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c06dd40 0x7f404c0ca520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f404c0b51b0 tx=0x7f404400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6ef198f3b6f545 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f404affd700 1 -- 172.21.15.6:0/3592717689 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404400e070 con 0x7f404c06dd40 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f404affd700 1 -- 172.21.15.6:0/3592717689 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404400b910 con 0x7f404c06dd40 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f404affd700 1 -- 172.21.15.6:0/3592717689 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404400bcc0 con 0x7f404c06dd40 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f40529f9700 1 -- 172.21.15.6:0/3592717689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c06dd40 msgr2=0x7f404c0ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f40529f9700 1 --2- 172.21.15.6:0/3592717689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c06dd40 0x7f404c0ca520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f404c0b51b0 tx=0x7f404400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f40529f9700 1 -- 172.21.15.6:0/3592717689 shutdown_connections 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f40529f9700 1 --2- 172.21.15.6:0/3592717689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c06dd40 0x7f404c0ca520 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f40529f9700 1 --2- 172.21.15.6:0/3592717689 >> v2:172.21.15.61:3300/0 conn(0x7f404c0b2150 0x7f404c06f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f40529f9700 1 -- 172.21.15.6:0/3592717689 >> 172.21.15.6:0/3592717689 conn(0x7f404c0b0df0 msgr2=0x7f404c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:15.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f40529f9700 1 -- 172.21.15.6:0/3592717689 shutdown_connections 2023-11-28T13:01:15.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.158+0000 7f40529f9700 1 -- 172.21.15.6:0/3592717689 wait complete. 2023-11-28T13:01:15.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.159+0000 7f40529f9700 1 Processor -- start 2023-11-28T13:01:15.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.160+0000 7f40529f9700 1 -- start start 2023-11-28T13:01:15.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.160+0000 7f40529f9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f404c06dd40 0x7f404c0abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.160+0000 7f40529f9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c0b2150 0x7f404c0aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.160+0000 7f40529f9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f404c15a490 0x7f404c158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f40529f9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c0a7f80 con 0x7f404c0b2150 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f40529f9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c15b1d0 con 0x7f404c06dd40 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f40529f9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404c0ac470 con 0x7f404c15a490 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c0b2150 0x7f404c0aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c0b2150 0x7f404c0aa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59420/0 (socket says 172.21.15.6:59420) 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404b7fe700 1 -- 172.21.15.6:0/1901331165 learned_addr learned my addr 172.21.15.6:0/1901331165 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404bfff700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f404c06dd40 0x7f404c0abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f4050f96700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f404c15a490 0x7f404c158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404b7fe700 1 -- 172.21.15.6:0/1901331165 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f404c15a490 msgr2=0x7f404c158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404b7fe700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f404c15a490 0x7f404c158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404b7fe700 1 -- 172.21.15.6:0/1901331165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f404c06dd40 msgr2=0x7f404c0abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404b7fe700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f404c06dd40 0x7f404c0abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404b7fe700 1 -- 172.21.15.6:0/1901331165 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4044009b30 con 0x7f404c0b2150 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404bfff700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f404c06dd40 0x7f404c0abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f404b7fe700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c0b2150 0x7f404c0aa1b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f403400e9e0 tx=0x7f403400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f4048ff9700 1 -- 172.21.15.6:0/1901331165 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034009710 con 0x7f404c0b2150 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f4048ff9700 1 -- 172.21.15.6:0/1901331165 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403400c850 con 0x7f404c0b2150 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f40529f9700 1 -- 172.21.15.6:0/1901331165 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404c0ca000 con 0x7f404c0b2150 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f4048ff9700 1 -- 172.21.15.6:0/1901331165 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403400cbc0 con 0x7f404c0b2150 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.161+0000 7f40529f9700 1 -- 172.21.15.6:0/1901331165 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c0c9d20 con 0x7f404c0b2150 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.162+0000 7f40529f9700 1 -- 172.21.15.6:0/1901331165 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404c06ae90 con 0x7f404c0b2150 2023-11-28T13:01:15.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.163+0000 7f4048ff9700 1 -- 172.21.15.6:0/1901331165 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4034019030 con 0x7f404c0b2150 2023-11-28T13:01:15.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.163+0000 7f4048ff9700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f402c0682f0 0x7f402c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.163+0000 7f4048ff9700 1 -- 172.21.15.6:0/1901331165 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4034085fe0 con 0x7f404c0b2150 2023-11-28T13:01:15.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.163+0000 7f404bfff700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f402c0682f0 0x7f402c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.164+0000 7f404bfff700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f402c0682f0 0x7f402c06a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f403c000f60 tx=0x7f403c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:15.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.167+0000 7f4048ff9700 1 -- 172.21.15.6:0/1901331165 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4034052510 con 0x7f404c0b2150 2023-11-28T13:01:15.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.362+0000 7f40529f9700 1 -- 172.21.15.6:0/1901331165 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f404c0a8130 con 0x7f404c0b2150 2023-11-28T13:01:15.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.362+0000 7f4048ff9700 1 -- 172.21.15.6:0/1901331165 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f404c0a8130 con 0x7f404c0b2150 2023-11-28T13:01:15.364 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.364+0000 7f40529f9700 1 -- 172.21.15.6:0/1901331165 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f402c0682f0 msgr2=0x7f402c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.364+0000 7f40529f9700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f402c0682f0 0x7f402c06a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f403c000f60 tx=0x7f403c009040 comp rx=0 tx=0).stop 2023-11-28T13:01:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.364+0000 7f40529f9700 1 -- 172.21.15.6:0/1901331165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c0b2150 msgr2=0x7f404c0aa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.364+0000 7f40529f9700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c0b2150 0x7f404c0aa1b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f403400e9e0 tx=0x7f403400eda0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.364+0000 7f40529f9700 1 -- 172.21.15.6:0/1901331165 shutdown_connections 2023-11-28T13:01:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.364+0000 7f40529f9700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f402c0682f0 0x7f402c06a7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.364+0000 7f40529f9700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f404c15a490 0x7f404c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.364+0000 7f40529f9700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f404c06dd40 0x7f404c0abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.364+0000 7f40529f9700 1 --2- 172.21.15.6:0/1901331165 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f404c0b2150 0x7f404c0aa1b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.364+0000 7f40529f9700 1 -- 172.21.15.6:0/1901331165 >> 172.21.15.6:0/1901331165 conn(0x7f404c0b0df0 msgr2=0x7f404c0afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:15.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.365+0000 7f40529f9700 1 -- 172.21.15.6:0/1901331165 shutdown_connections 2023-11-28T13:01:15.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.365+0000 7f40529f9700 1 -- 172.21.15.6:0/1901331165 wait complete. 2023-11-28T13:01:15.378 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:15.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.502+0000 7fb9db874700 1 Processor -- start 2023-11-28T13:01:15.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.502+0000 7fb9db874700 1 -- start start 2023-11-28T13:01:15.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.503+0000 7fb9db874700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d4161090 0x7fb9d40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.503+0000 7fb9db874700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb9d40caeb0 0x7fb9d40cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.503+0000 7fb9db874700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d40ce340 con 0x7fb9d415c990 2023-11-28T13:01:15.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.503+0000 7fb9db874700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d40cebe0 con 0x7fb9d40caeb0 2023-11-28T13:01:15.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.503+0000 7fb9db874700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40cf780 con 0x7fb9d4161090 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.503+0000 7fb9d8e0f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d4161090 0x7fb9d40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.503+0000 7fb9d9e11700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb9d40caeb0 0x7fb9d40cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.503+0000 7fb9d8e0f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d4161090 0x7fb9d40ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:49132/0 (socket says 172.21.15.6:49132) 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.503+0000 7fb9d8e0f700 1 -- 172.21.15.6:0/3827540519 learned_addr learned my addr 172.21.15.6:0/3827540519 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.503+0000 7fb9d9e11700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb9d40caeb0 0x7fb9d40cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:34916/0 (socket says 172.21.15.6:34916) 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9d9e11700 1 -- 172.21.15.6:0/3827540519 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d4161090 msgr2=0x7fb9d40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9d9e11700 1 --2- 172.21.15.6:0/3827540519 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d4161090 0x7fb9d40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9d9e11700 1 -- 172.21.15.6:0/3827540519 >> v1:172.21.15.61:6789/0 conn(0x7fb9d415c990 legacy=0x7fb9d40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9d9e11700 1 -- 172.21.15.6:0/3827540519 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d40cb3f0 con 0x7fb9d40caeb0 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9cbfff700 1 -- 172.21.15.6:0/3827540519 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026745008 0 0) 0x7fb9d40ce340 con 0x7fb9d415c990 2023-11-28T13:01:15.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9d8e0f700 1 --2- 172.21.15.6:0/3827540519 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d4161090 0x7fb9d40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9d9e11700 1 --2- 172.21.15.6:0/3827540519 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9d40caeb0 0x7fb9d40cc310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009d90 tx=0x7fb9cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bdb993aace43cc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9cbfff700 1 -- 172.21.15.6:0/3827540519 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc00e070 con 0x7fb9d40caeb0 2023-11-28T13:01:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9cbfff700 1 -- 172.21.15.6:0/3827540519 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc00bb20 con 0x7fb9d40caeb0 2023-11-28T13:01:15.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9cbfff700 1 -- 172.21.15.6:0/3827540519 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc012430 con 0x7fb9d40caeb0 2023-11-28T13:01:15.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9db874700 1 -- 172.21.15.6:0/3827540519 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9d40caeb0 msgr2=0x7fb9d40cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.504+0000 7fb9db874700 1 --2- 172.21.15.6:0/3827540519 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9d40caeb0 0x7fb9d40cc310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009d90 tx=0x7fb9cc00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:15.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.505+0000 7fb9db874700 1 -- 172.21.15.6:0/3827540519 shutdown_connections 2023-11-28T13:01:15.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.505+0000 7fb9db874700 1 --2- 172.21.15.6:0/3827540519 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9d40caeb0 0x7fb9d40cc310 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.505+0000 7fb9db874700 1 --2- 172.21.15.6:0/3827540519 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d4161090 0x7fb9d40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.505+0000 7fb9db874700 1 -- 172.21.15.6:0/3827540519 >> 172.21.15.6:0/3827540519 conn(0x7fb9d40b1650 msgr2=0x7fb9d40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:15.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.505+0000 7fb9db874700 1 -- 172.21.15.6:0/3827540519 shutdown_connections 2023-11-28T13:01:15.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.505+0000 7fb9db874700 1 -- 172.21.15.6:0/3827540519 wait complete. 2023-11-28T13:01:15.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.506+0000 7fb9db874700 1 Processor -- start 2023-11-28T13:01:15.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.506+0000 7fb9db874700 1 -- start start 2023-11-28T13:01:15.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9db874700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d40caeb0 0x7fb9d414dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9db874700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9d415c990 0x7fb9d414e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9db874700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9d4161090 0x7fb9d414e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9db874700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40b4360 con 0x7fb9d415c990 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9db874700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40b4e80 con 0x7fb9d4161090 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9db874700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40b5310 con 0x7fb9d40caeb0 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d8e0f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9d415c990 0x7fb9d414e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d8e0f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9d415c990 0x7fb9d414e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59434/0 (socket says 172.21.15.6:59434) 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d8e0f700 1 -- 172.21.15.6:0/557704617 learned_addr learned my addr 172.21.15.6:0/557704617 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d9e11700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9d4161090 0x7fb9d414e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d9610700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d40caeb0 0x7fb9d414dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d8e0f700 1 -- 172.21.15.6:0/557704617 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d40caeb0 msgr2=0x7fb9d414dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d8e0f700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d40caeb0 0x7fb9d414dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d8e0f700 1 -- 172.21.15.6:0/557704617 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9d4161090 msgr2=0x7fb9d414e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d8e0f700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9d4161090 0x7fb9d414e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d8e0f700 1 -- 172.21.15.6:0/557704617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c4009550 con 0x7fb9d415c990 2023-11-28T13:01:15.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9d8e0f700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9d415c990 0x7fb9d414e1f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400e980 tx=0x7fb9c400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:15.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9c9ffb700 1 -- 172.21.15.6:0/557704617 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400c690 con 0x7fb9d415c990 2023-11-28T13:01:15.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.507+0000 7fb9c9ffb700 1 -- 172.21.15.6:0/557704617 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c400c7f0 con 0x7fb9d415c990 2023-11-28T13:01:15.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.508+0000 7fb9db874700 1 -- 172.21.15.6:0/557704617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc009a70 con 0x7fb9d415c990 2023-11-28T13:01:15.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.508+0000 7fb9c9ffb700 1 -- 172.21.15.6:0/557704617 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400cb60 con 0x7fb9d415c990 2023-11-28T13:01:15.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.508+0000 7fb9db874700 1 -- 172.21.15.6:0/557704617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d40ce870 con 0x7fb9d415c990 2023-11-28T13:01:15.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.509+0000 7fb9db874700 1 -- 172.21.15.6:0/557704617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d40ce590 con 0x7fb9d415c990 2023-11-28T13:01:15.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.509+0000 7fb9c9ffb700 1 -- 172.21.15.6:0/557704617 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb9c4019030 con 0x7fb9d415c990 2023-11-28T13:01:15.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.509+0000 7fb9c9ffb700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9b80682f0 0x7fb9b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:15.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.510+0000 7fb9c9ffb700 1 -- 172.21.15.6:0/557704617 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb9c4085e50 con 0x7fb9d415c990 2023-11-28T13:01:15.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.510+0000 7fb9d9610700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9b80682f0 0x7fb9b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:15.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.510+0000 7fb9d9610700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9b80682f0 0x7fb9b806a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0007fd0 tx=0x7fb9d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:15.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.514+0000 7fb9c9ffb700 1 -- 172.21.15.6:0/557704617 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb9c4052380 con 0x7fb9d415c990 2023-11-28T13:01:15.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.723+0000 7fb9db874700 1 -- 172.21.15.6:0/557704617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb9d40cc850 con 0x7fb9d415c990 2023-11-28T13:01:15.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.723+0000 7fb9c9ffb700 1 -- 172.21.15.6:0/557704617 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb9c40559a0 con 0x7fb9d415c990 2023-11-28T13:01:15.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 -- 172.21.15.6:0/557704617 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9b80682f0 msgr2=0x7fb9b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9b80682f0 0x7fb9b806a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0007fd0 tx=0x7fb9d0009040 comp rx=0 tx=0).stop 2023-11-28T13:01:15.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 -- 172.21.15.6:0/557704617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9d415c990 msgr2=0x7fb9d414e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:15.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9d415c990 0x7fb9d414e1f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400e980 tx=0x7fb9c400ec90 comp rx=0 tx=0).stop 2023-11-28T13:01:15.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 -- 172.21.15.6:0/557704617 shutdown_connections 2023-11-28T13:01:15.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9d4161090 0x7fb9d414e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9b80682f0 0x7fb9b806a7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb9d40caeb0 0x7fb9d414dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 --2- 172.21.15.6:0/557704617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb9d415c990 0x7fb9d414e1f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:15.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 -- 172.21.15.6:0/557704617 >> 172.21.15.6:0/557704617 conn(0x7fb9d40b1650 msgr2=0x7fb9d40c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:15.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 -- 172.21.15.6:0/557704617 shutdown_connections 2023-11-28T13:01:15.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:15.725+0000 7fb9db874700 1 -- 172.21.15.6:0/557704617 wait complete. 2023-11-28T13:01:15.728 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:01:15.738 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi061.front.sepia.ceph.com... 2023-11-28T13:01:15.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:15.739 DEBUG:teuthology.orchestra.run.smithi061:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-28T13:01:15.785 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:15.785 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:01:15.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:15.785 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T13:01:15.811 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:01:15.811 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:15.830 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:01:15.830 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:15.888 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:16.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.051+0000 7fa194b54700 1 Processor -- start 2023-11-28T13:01:16.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.051+0000 7fa194b54700 1 -- start start 2023-11-28T13:01:16.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.052+0000 7fa194b54700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 0x7fa1900cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.052+0000 7fa194b54700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa1900d0020 0x7fa1900ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.052+0000 7fa194b54700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1900cfb90 con 0x7fa1900b2660 2023-11-28T13:01:16.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.052+0000 7fa194b54700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1900cb930 con 0x7fa1900d0020 2023-11-28T13:01:16.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.052+0000 7fa194b54700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1900cf260 con 0x7fa19015a6c0 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.052+0000 7fa18dd9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa1900d0020 0x7fa1900ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.052+0000 7fa18dd9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa1900d0020 0x7fa1900ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59440/0 (socket says 172.21.15.6:59440) 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.052+0000 7fa18dd9b700 1 -- 172.21.15.6:0/2527038396 learned_addr learned my addr 172.21.15.6:0/2527038396 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.053+0000 7fa18e59c700 1 --2- 172.21.15.6:0/2527038396 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 0x7fa1900cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.053+0000 7fa18dd9b700 1 -- 172.21.15.6:0/2527038396 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 msgr2=0x7fa1900cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.053+0000 7fa18dd9b700 1 --2- 172.21.15.6:0/2527038396 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 0x7fa1900cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.053+0000 7fa18dd9b700 1 -- 172.21.15.6:0/2527038396 >> v1:172.21.15.61:6789/0 conn(0x7fa1900b2660 legacy=0x7fa1900b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.053+0000 7fa18dd9b700 1 -- 172.21.15.6:0/2527038396 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1900cf540 con 0x7fa1900d0020 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.053+0000 7fa18e59c700 1 --2- 172.21.15.6:0/2527038396 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 0x7fa1900cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.053+0000 7fa18dd9b700 1 --2- 172.21.15.6:0/2527038396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 0x7fa1900ccea0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1780098f0 tx=0x7fa178009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8b7f77228e6492 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.053+0000 7fa18d59a700 1 -- 172.21.15.6:0/2527038396 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800b8a0 con 0x7fa1900d0020 2023-11-28T13:01:16.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.053+0000 7fa18d59a700 1 -- 172.21.15.6:0/2527038396 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17800ba00 con 0x7fa1900d0020 2023-11-28T13:01:16.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.053+0000 7fa18d59a700 1 -- 172.21.15.6:0/2527038396 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800bd90 con 0x7fa1900d0020 2023-11-28T13:01:16.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.054+0000 7fa194b54700 1 -- 172.21.15.6:0/2527038396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 msgr2=0x7fa1900ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.054+0000 7fa194b54700 1 --2- 172.21.15.6:0/2527038396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 0x7fa1900ccea0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1780098f0 tx=0x7fa178009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:16.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.054+0000 7fa194b54700 1 -- 172.21.15.6:0/2527038396 shutdown_connections 2023-11-28T13:01:16.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.054+0000 7fa194b54700 1 --2- 172.21.15.6:0/2527038396 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 0x7fa1900cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.054+0000 7fa194b54700 1 --2- 172.21.15.6:0/2527038396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 0x7fa1900ccea0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.054+0000 7fa194b54700 1 -- 172.21.15.6:0/2527038396 >> 172.21.15.6:0/2527038396 conn(0x7fa1900cd660 msgr2=0x7fa1900cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.054+0000 7fa194b54700 1 -- 172.21.15.6:0/2527038396 shutdown_connections 2023-11-28T13:01:16.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.054+0000 7fa194b54700 1 -- 172.21.15.6:0/2527038396 wait complete. 2023-11-28T13:01:16.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.055+0000 7fa194b54700 1 Processor -- start 2023-11-28T13:01:16.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.055+0000 7fa194b54700 1 -- start start 2023-11-28T13:01:16.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa194b54700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1900b2660 0x7fa1900c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa194b54700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 0x7fa1900c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa194b54700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 0x7fa1900ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa194b54700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1900c7500 con 0x7fa1900d0020 2023-11-28T13:01:16.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa194b54700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1900ca560 con 0x7fa1900b2660 2023-11-28T13:01:16.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa194b54700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1900c9950 con 0x7fa19015a6c0 2023-11-28T13:01:16.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 0x7fa1900c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 0x7fa1900c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59444/0 (socket says 172.21.15.6:59444) 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18dd9b700 1 -- 172.21.15.6:0/4026892696 learned_addr learned my addr 172.21.15.6:0/4026892696 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18ed9d700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 0x7fa1900ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18e59c700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1900b2660 0x7fa1900c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18dd9b700 1 -- 172.21.15.6:0/4026892696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 msgr2=0x7fa1900ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18dd9b700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 0x7fa1900ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18dd9b700 1 -- 172.21.15.6:0/4026892696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1900b2660 msgr2=0x7fa1900c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18dd9b700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1900b2660 0x7fa1900c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18dd9b700 1 -- 172.21.15.6:0/4026892696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa178009550 con 0x7fa1900d0020 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.056+0000 7fa18dd9b700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 0x7fa1900c9100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa178000c00 tx=0x7fa178004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.057+0000 7fa186ffd700 1 -- 172.21.15.6:0/4026892696 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780044d0 con 0x7fa1900d0020 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.057+0000 7fa186ffd700 1 -- 172.21.15.6:0/4026892696 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17801f070 con 0x7fa1900d0020 2023-11-28T13:01:16.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.057+0000 7fa194b54700 1 -- 172.21.15.6:0/4026892696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1900cd3e0 con 0x7fa1900d0020 2023-11-28T13:01:16.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.057+0000 7fa186ffd700 1 -- 172.21.15.6:0/4026892696 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780038c0 con 0x7fa1900d0020 2023-11-28T13:01:16.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.057+0000 7fa194b54700 1 -- 172.21.15.6:0/4026892696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1900cf820 con 0x7fa1900d0020 2023-11-28T13:01:16.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.058+0000 7fa194b54700 1 -- 172.21.15.6:0/4026892696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1900d0470 con 0x7fa1900d0020 2023-11-28T13:01:16.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.058+0000 7fa186ffd700 1 -- 172.21.15.6:0/4026892696 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa17800f050 con 0x7fa1900d0020 2023-11-28T13:01:16.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.059+0000 7fa186ffd700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1700682f0 0x7fa17006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.059+0000 7fa186ffd700 1 -- 172.21.15.6:0/4026892696 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa178021070 con 0x7fa1900d0020 2023-11-28T13:01:16.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.059+0000 7fa18e59c700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1700682f0 0x7fa17006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.059+0000 7fa18e59c700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1700682f0 0x7fa17006a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa180006fd0 tx=0x7fa180008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.063+0000 7fa186ffd700 1 -- 172.21.15.6:0/4026892696 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa178053dc0 con 0x7fa1900d0020 2023-11-28T13:01:16.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.258+0000 7fa194b54700 1 -- 172.21.15.6:0/4026892696 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1900675a0 con 0x7fa1900d0020 2023-11-28T13:01:16.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.260+0000 7fa186ffd700 1 -- 172.21.15.6:0/4026892696 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19690 (secure 0 0 0) 0x7fa1780573e0 con 0x7fa1900d0020 2023-11-28T13:01:16.261 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:16.261 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:10.053012+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:16.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.261+0000 7fa194b54700 1 -- 172.21.15.6:0/4026892696 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1700682f0 msgr2=0x7fa17006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.261+0000 7fa194b54700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1700682f0 0x7fa17006a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa180006fd0 tx=0x7fa180008040 comp rx=0 tx=0).stop 2023-11-28T13:01:16.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.261+0000 7fa194b54700 1 -- 172.21.15.6:0/4026892696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 msgr2=0x7fa1900c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.261+0000 7fa194b54700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 0x7fa1900c9100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa178000c00 tx=0x7fa178004750 comp rx=0 tx=0).stop 2023-11-28T13:01:16.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.261+0000 7fa194b54700 1 -- 172.21.15.6:0/4026892696 shutdown_connections 2023-11-28T13:01:16.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.261+0000 7fa194b54700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1700682f0 0x7fa17006a7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.261+0000 7fa194b54700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa19015a6c0 0x7fa1900ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.262+0000 7fa194b54700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1900b2660 0x7fa1900c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.262+0000 7fa194b54700 1 --2- 172.21.15.6:0/4026892696 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1900d0020 0x7fa1900c9100 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.262+0000 7fa194b54700 1 -- 172.21.15.6:0/4026892696 >> 172.21.15.6:0/4026892696 conn(0x7fa1900cd660 msgr2=0x7fa1900b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.262+0000 7fa194b54700 1 -- 172.21.15.6:0/4026892696 shutdown_connections 2023-11-28T13:01:16.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.262+0000 7fa194b54700 1 -- 172.21.15.6:0/4026892696 wait complete. 2023-11-28T13:01:16.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:16.276 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:16.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.402+0000 7f8b1de4e700 1 Processor -- start 2023-11-28T13:01:16.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.402+0000 7f8b1de4e700 1 -- start start 2023-11-28T13:01:16.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.403+0000 7f8b1de4e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 0x7f8b180ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.403+0000 7f8b1de4e700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1806e470 con 0x7f8b18158d90 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.403+0000 7f8b1de4e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1806f490 con 0x7f8b1806e900 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.403+0000 7f8b1de4e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1806de60 con 0x7f8b181589c0 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.403+0000 7f8b16ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f8b18158d90 0x7f8b1806aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46772/0 (socket says 172.21.15.6:46772) 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.403+0000 7f8b16ffd700 1 -- 172.21.15.6:0/220063498 learned_addr learned my addr 172.21.15.6:0/220063498 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.403+0000 7f8b177fe700 1 --2- 172.21.15.6:0/220063498 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 0x7f8b180ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b167fc700 1 -- 172.21.15.6:0/220063498 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396934677 0 0) 0x7f8b1806e470 con 0x7f8b18158d90 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b167fc700 1 -- 172.21.15.6:0/220063498 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b040035b0 con 0x7f8b18158d90 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b167fc700 1 -- 172.21.15.6:0/220063498 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308922387 0 0) 0x7f8b1806f490 con 0x7f8b1806e900 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b167fc700 1 -- 172.21.15.6:0/220063498 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b1806e470 con 0x7f8b1806e900 2023-11-28T13:01:16.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b177fe700 1 -- 172.21.15.6:0/220063498 >> v1:172.21.15.61:6789/0 conn(0x7f8b1806e900 legacy=0x7f8b1806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b177fe700 1 -- 172.21.15.6:0/220063498 >> v1:172.21.15.6:6789/0 conn(0x7f8b18158d90 legacy=0x7f8b1806aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b177fe700 1 -- 172.21.15.6:0/220063498 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b180ceb40 con 0x7f8b181589c0 2023-11-28T13:01:16.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b177fe700 1 --2- 172.21.15.6:0/220063498 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 0x7f8b180ce560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009870 tx=0x7f8b08009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5693894705e662ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b167fc700 1 -- 172.21.15.6:0/220063498 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08011830 con 0x7f8b181589c0 2023-11-28T13:01:16.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b167fc700 1 -- 172.21.15.6:0/220063498 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b08011990 con 0x7f8b181589c0 2023-11-28T13:01:16.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.404+0000 7f8b167fc700 1 -- 172.21.15.6:0/220063498 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08011d20 con 0x7f8b181589c0 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.405+0000 7f8b1de4e700 1 -- 172.21.15.6:0/220063498 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 msgr2=0x7f8b180ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.405+0000 7f8b1de4e700 1 --2- 172.21.15.6:0/220063498 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 0x7f8b180ce560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009870 tx=0x7f8b08009b80 comp rx=0 tx=0).stop 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.405+0000 7f8b1de4e700 1 -- 172.21.15.6:0/220063498 shutdown_connections 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.405+0000 7f8b1de4e700 1 --2- 172.21.15.6:0/220063498 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 0x7f8b180ce560 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.405+0000 7f8b1de4e700 1 -- 172.21.15.6:0/220063498 >> 172.21.15.6:0/220063498 conn(0x7f8b180b0df0 msgr2=0x7f8b180b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.405+0000 7f8b1de4e700 1 -- 172.21.15.6:0/220063498 shutdown_connections 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.405+0000 7f8b1de4e700 1 -- 172.21.15.6:0/220063498 wait complete. 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.406+0000 7f8b1de4e700 1 Processor -- start 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.406+0000 7f8b1de4e700 1 -- start start 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b1de4e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b1806e900 0x7f8b18156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b1de4e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 0x7f8b181f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b1de4e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b18158d90 0x7f8b180b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b1de4e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180c5750 con 0x7f8b18158d90 2023-11-28T13:01:16.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b1de4e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180b3670 con 0x7f8b1806e900 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b1de4e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b181f22a0 con 0x7f8b181589c0 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b17fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b18158d90 0x7f8b180b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b17fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b18158d90 0x7f8b180b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59460/0 (socket says 172.21.15.6:59460) 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b17fff700 1 -- 172.21.15.6:0/3289149189 learned_addr learned my addr 172.21.15.6:0/3289149189 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b177fe700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b1806e900 0x7f8b18156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b16ffd700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 0x7f8b181f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b17fff700 1 -- 172.21.15.6:0/3289149189 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 msgr2=0x7f8b181f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b17fff700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 0x7f8b181f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.407+0000 7f8b17fff700 1 -- 172.21.15.6:0/3289149189 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b1806e900 msgr2=0x7f8b18156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.408+0000 7f8b17fff700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b1806e900 0x7f8b18156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.408+0000 7f8b17fff700 1 -- 172.21.15.6:0/3289149189 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b080094a0 con 0x7f8b18158d90 2023-11-28T13:01:16.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.408+0000 7f8b17fff700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b18158d90 0x7f8b180b3130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000cc20 tx=0x7f8b1000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.408+0000 7f8b0ffff700 1 -- 172.21.15.6:0/3289149189 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10003c00 con 0x7f8b18158d90 2023-11-28T13:01:16.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.408+0000 7f8b0ffff700 1 -- 172.21.15.6:0/3289149189 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b10003d60 con 0x7f8b18158d90 2023-11-28T13:01:16.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.408+0000 7f8b1de4e700 1 -- 172.21.15.6:0/3289149189 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1806e100 con 0x7f8b18158d90 2023-11-28T13:01:16.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.408+0000 7f8b0ffff700 1 -- 172.21.15.6:0/3289149189 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000aae0 con 0x7f8b18158d90 2023-11-28T13:01:16.415 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.408+0000 7f8b1de4e700 1 -- 172.21.15.6:0/3289149189 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b180cee20 con 0x7f8b18158d90 2023-11-28T13:01:16.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.409+0000 7f8b1de4e700 1 -- 172.21.15.6:0/3289149189 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b180c9d90 con 0x7f8b18158d90 2023-11-28T13:01:16.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.409+0000 7f8b0ffff700 1 -- 172.21.15.6:0/3289149189 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8b10010050 con 0x7f8b18158d90 2023-11-28T13:01:16.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.410+0000 7f8b0ffff700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8af80682f0 0x7f8af806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.410+0000 7f8b177fe700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8af80682f0 0x7f8af806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.410+0000 7f8b0ffff700 1 -- 172.21.15.6:0/3289149189 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8b10085f60 con 0x7f8b18158d90 2023-11-28T13:01:16.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.410+0000 7f8b177fe700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8af80682f0 0x7f8af806a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8b080097c0 tx=0x7f8b0801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.416 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.414+0000 7f8b0ffff700 1 -- 172.21.15.6:0/3289149189 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8b10052410 con 0x7f8b18158d90 2023-11-28T13:01:16.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.606+0000 7f8b1de4e700 1 -- 172.21.15.6:0/3289149189 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8b180739f0 con 0x7f8b18158d90 2023-11-28T13:01:16.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.606+0000 7f8b0ffff700 1 -- 172.21.15.6:0/3289149189 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8b10055a30 con 0x7f8b18158d90 2023-11-28T13:01:16.608 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:16.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.608+0000 7f8b1de4e700 1 -- 172.21.15.6:0/3289149189 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8af80682f0 msgr2=0x7f8af806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.608+0000 7f8b1de4e700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8af80682f0 0x7f8af806a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8b080097c0 tx=0x7f8b0801c040 comp rx=0 tx=0).stop 2023-11-28T13:01:16.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.608+0000 7f8b1de4e700 1 -- 172.21.15.6:0/3289149189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b18158d90 msgr2=0x7f8b180b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.608+0000 7f8b1de4e700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b18158d90 0x7f8b180b3130 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000cc20 tx=0x7f8b1000cf30 comp rx=0 tx=0).stop 2023-11-28T13:01:16.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.608+0000 7f8b1de4e700 1 -- 172.21.15.6:0/3289149189 shutdown_connections 2023-11-28T13:01:16.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.609+0000 7f8b1de4e700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b18158d90 0x7f8b180b3130 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.609+0000 7f8b1de4e700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b1806e900 0x7f8b18156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.609+0000 7f8b1de4e700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8af80682f0 0x7f8af806a7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.609+0000 7f8b1de4e700 1 --2- 172.21.15.6:0/3289149189 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8b181589c0 0x7f8b181f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.609+0000 7f8b1de4e700 1 -- 172.21.15.6:0/3289149189 >> 172.21.15.6:0/3289149189 conn(0x7f8b180b0df0 msgr2=0x7f8b1806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.609+0000 7f8b1de4e700 1 -- 172.21.15.6:0/3289149189 shutdown_connections 2023-11-28T13:01:16.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.609+0000 7f8b1de4e700 1 -- 172.21.15.6:0/3289149189 wait complete. 2023-11-28T13:01:16.622 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:16.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.749+0000 7fed6066b700 1 Processor -- start 2023-11-28T13:01:16.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.749+0000 7fed6066b700 1 -- start start 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed6066b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fed5815a490 0x7fed5806f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed6066b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fed580b2150 0x7fed5806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed6066b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5815a860 con 0x7fed5806bcd0 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed6066b700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5806b510 con 0x7fed580b2150 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed6066b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed580c9a80 con 0x7fed5815a490 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed5dc06700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fed580b2150 0x7fed5806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed5dc06700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fed580b2150 0x7fed5806d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59464/0 (socket says 172.21.15.6:59464) 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed5dc06700 1 -- 172.21.15.6:0/2651464612 learned_addr learned my addr 172.21.15.6:0/2651464612 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed5e407700 1 --2- 172.21.15.6:0/2651464612 >> v2:172.21.15.61:3300/0 conn(0x7fed5815a490 0x7fed5806f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed5dc06700 1 -- 172.21.15.6:0/2651464612 >> v2:172.21.15.61:3300/0 conn(0x7fed5815a490 msgr2=0x7fed5806f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:16.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed5dc06700 1 --2- 172.21.15.6:0/2651464612 >> v2:172.21.15.61:3300/0 conn(0x7fed5815a490 0x7fed5806f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed5dc06700 1 -- 172.21.15.6:0/2651464612 >> v1:172.21.15.61:6789/0 conn(0x7fed5806bcd0 legacy=0x7fed5815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.750+0000 7fed5dc06700 1 -- 172.21.15.6:0/2651464612 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5806b230 con 0x7fed580b2150 2023-11-28T13:01:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed5dc06700 1 --2- 172.21.15.6:0/2651464612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 0x7fed5806d630 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed580cf1b0 tx=0x7fed54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965c23ea1d05856a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed5d405700 1 -- 172.21.15.6:0/2651464612 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54004230 con 0x7fed580b2150 2023-11-28T13:01:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed5d405700 1 -- 172.21.15.6:0/2651464612 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed54004390 con 0x7fed580b2150 2023-11-28T13:01:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed5d405700 1 -- 172.21.15.6:0/2651464612 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed540036a0 con 0x7fed580b2150 2023-11-28T13:01:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed6066b700 1 -- 172.21.15.6:0/2651464612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 msgr2=0x7fed5806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed6066b700 1 --2- 172.21.15.6:0/2651464612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 0x7fed5806d630 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed580cf1b0 tx=0x7fed54009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed6066b700 1 -- 172.21.15.6:0/2651464612 shutdown_connections 2023-11-28T13:01:16.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed6066b700 1 --2- 172.21.15.6:0/2651464612 >> v2:172.21.15.61:3300/0 conn(0x7fed5815a490 0x7fed5806f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed6066b700 1 --2- 172.21.15.6:0/2651464612 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 0x7fed5806d630 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed6066b700 1 -- 172.21.15.6:0/2651464612 >> 172.21.15.6:0/2651464612 conn(0x7fed580b0df0 msgr2=0x7fed580b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed6066b700 1 -- 172.21.15.6:0/2651464612 shutdown_connections 2023-11-28T13:01:16.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.751+0000 7fed6066b700 1 -- 172.21.15.6:0/2651464612 wait complete. 2023-11-28T13:01:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.753+0000 7fed6066b700 1 Processor -- start 2023-11-28T13:01:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.753+0000 7fed6066b700 1 -- start start 2023-11-28T13:01:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.753+0000 7fed6066b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed5806bcd0 0x7fed580ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.753+0000 7fed6066b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 0x7fed58158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.753+0000 7fed6066b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed5815a490 0x7fed58158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.753+0000 7fed6066b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed581f2cf0 con 0x7fed580b2150 2023-11-28T13:01:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.753+0000 7fed6066b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed581f20e0 con 0x7fed5806bcd0 2023-11-28T13:01:16.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.753+0000 7fed6066b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed581f23c0 con 0x7fed5815a490 2023-11-28T13:01:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.753+0000 7fed5dc06700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 0x7fed58158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed5dc06700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 0x7fed58158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59472/0 (socket says 172.21.15.6:59472) 2023-11-28T13:01:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed5dc06700 1 -- 172.21.15.6:0/1507530280 learned_addr learned my addr 172.21.15.6:0/1507530280 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed5dc06700 1 -- 172.21.15.6:0/1507530280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed5815a490 msgr2=0x7fed58158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed5dc06700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed5815a490 0x7fed58158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed5dc06700 1 -- 172.21.15.6:0/1507530280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed5806bcd0 msgr2=0x7fed580ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed5dc06700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed5806bcd0 0x7fed580ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed5dc06700 1 -- 172.21.15.6:0/1507530280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed54009550 con 0x7fed580b2150 2023-11-28T13:01:16.758 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed5dc06700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 0x7fed58158000 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed54009130 tx=0x7fed54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed4effd700 1 -- 172.21.15.6:0/1507530280 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54011630 con 0x7fed580b2150 2023-11-28T13:01:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed4effd700 1 -- 172.21.15.6:0/1507530280 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed54011790 con 0x7fed580b2150 2023-11-28T13:01:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed6066b700 1 -- 172.21.15.6:0/1507530280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed580ca000 con 0x7fed580b2150 2023-11-28T13:01:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed4effd700 1 -- 172.21.15.6:0/1507530280 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54011b20 con 0x7fed580b2150 2023-11-28T13:01:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.754+0000 7fed6066b700 1 -- 172.21.15.6:0/1507530280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed580c9d20 con 0x7fed580b2150 2023-11-28T13:01:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.755+0000 7fed6066b700 1 -- 172.21.15.6:0/1507530280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5806ae90 con 0x7fed580b2150 2023-11-28T13:01:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.756+0000 7fed4effd700 1 -- 172.21.15.6:0/1507530280 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fed54017020 con 0x7fed580b2150 2023-11-28T13:01:16.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.756+0000 7fed4effd700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed3c0682f0 0x7fed3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:16.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.756+0000 7fed4effd700 1 -- 172.21.15.6:0/1507530280 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fed54087830 con 0x7fed580b2150 2023-11-28T13:01:16.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.756+0000 7fed5e407700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed3c0682f0 0x7fed3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:16.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.757+0000 7fed5e407700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed3c0682f0 0x7fed3c06a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fed5806e670 tx=0x7fed48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:16.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.761+0000 7fed4effd700 1 -- 172.21.15.6:0/1507530280 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fed54053d60 con 0x7fed580b2150 2023-11-28T13:01:16.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.953+0000 7fed6066b700 1 -- 172.21.15.6:0/1507530280 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed580739f0 con 0x7fed580b2150 2023-11-28T13:01:16.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.954+0000 7fed4effd700 1 -- 172.21.15.6:0/1507530280 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed54057380 con 0x7fed580b2150 2023-11-28T13:01:16.955 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 -- 172.21.15.6:0/1507530280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed3c0682f0 msgr2=0x7fed3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed3c0682f0 0x7fed3c06a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fed5806e670 tx=0x7fed48008040 comp rx=0 tx=0).stop 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 -- 172.21.15.6:0/1507530280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 msgr2=0x7fed58158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 0x7fed58158000 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed54009130 tx=0x7fed54004750 comp rx=0 tx=0).stop 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 -- 172.21.15.6:0/1507530280 shutdown_connections 2023-11-28T13:01:16.958 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed3c0682f0 0x7fed3c06a7a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed5815a490 0x7fed58158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed5806bcd0 0x7fed580ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 --2- 172.21.15.6:0/1507530280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed580b2150 0x7fed58158000 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 -- 172.21.15.6:0/1507530280 >> 172.21.15.6:0/1507530280 conn(0x7fed580b0df0 msgr2=0x7fed580b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 -- 172.21.15.6:0/1507530280 shutdown_connections 2023-11-28T13:01:16.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:16.956+0000 7fed6066b700 1 -- 172.21.15.6:0/1507530280 wait complete. 2023-11-28T13:01:16.969 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:17.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.098+0000 7f90ef425700 1 Processor -- start 2023-11-28T13:01:17.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.098+0000 7f90ef425700 1 -- start start 2023-11-28T13:01:17.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ef425700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90e80b62d0 0x7f90e80cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ef425700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f90e806f8b0 0x7f90e80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ef425700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f90e80b2bc0 0x7f90e80ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ef425700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e80b3020 con 0x7f90e80b2bc0 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ef425700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e80cd2a0 con 0x7f90e806f8b0 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ef425700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80d0400 con 0x7f90e80b62d0 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ed9c2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f90e80b2bc0 0x7f90e80ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ed9c2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f90e80b2bc0 0x7f90e80ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59482/0 (socket says 172.21.15.6:59482) 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ed9c2700 1 -- 172.21.15.6:0/1150850714 learned_addr learned my addr 172.21.15.6:0/1150850714 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ed1c1700 1 --2- 172.21.15.6:0/1150850714 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90e80b62d0 0x7f90e80cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ec9c0700 1 --2- 172.21.15.6:0/1150850714 >> v2:172.21.15.61:3300/0 conn(0x7f90e806f8b0 0x7f90e80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ed9c2700 1 -- 172.21.15.6:0/1150850714 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90e80b62d0 msgr2=0x7f90e80cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ed9c2700 1 --2- 172.21.15.6:0/1150850714 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90e80b62d0 0x7f90e80cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ed9c2700 1 -- 172.21.15.6:0/1150850714 >> v2:172.21.15.61:3300/0 conn(0x7f90e806f8b0 msgr2=0x7f90e80b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ed9c2700 1 --2- 172.21.15.6:0/1150850714 >> v2:172.21.15.61:3300/0 conn(0x7f90e806f8b0 0x7f90e80b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ed9c2700 1 -- 172.21.15.6:0/1150850714 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e80d06e0 con 0x7f90e80b2bc0 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.099+0000 7f90ec9c0700 1 --2- 172.21.15.6:0/1150850714 >> v2:172.21.15.61:3300/0 conn(0x7f90e806f8b0 0x7f90e80b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90ed9c2700 1 --2- 172.21.15.6:0/1150850714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e80b2bc0 0x7f90e80ce7d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f90e8071740 tx=0x7f90e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffdb430128b49fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90dffff700 1 -- 172.21.15.6:0/1150850714 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e000e070 con 0x7f90e80b2bc0 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90dffff700 1 -- 172.21.15.6:0/1150850714 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e000b910 con 0x7f90e80b2bc0 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90dffff700 1 -- 172.21.15.6:0/1150850714 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e000bcc0 con 0x7f90e80b2bc0 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90ef425700 1 -- 172.21.15.6:0/1150850714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e80b2bc0 msgr2=0x7f90e80ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90ef425700 1 --2- 172.21.15.6:0/1150850714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e80b2bc0 0x7f90e80ce7d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f90e8071740 tx=0x7f90e000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90ef425700 1 -- 172.21.15.6:0/1150850714 shutdown_connections 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90ef425700 1 --2- 172.21.15.6:0/1150850714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e80b2bc0 0x7f90e80ce7d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90ef425700 1 --2- 172.21.15.6:0/1150850714 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90e80b62d0 0x7f90e80cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90ef425700 1 --2- 172.21.15.6:0/1150850714 >> v2:172.21.15.61:3300/0 conn(0x7f90e806f8b0 0x7f90e80b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90ef425700 1 -- 172.21.15.6:0/1150850714 >> 172.21.15.6:0/1150850714 conn(0x7f90e80cd680 msgr2=0x7f90e80cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:17.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90ef425700 1 -- 172.21.15.6:0/1150850714 shutdown_connections 2023-11-28T13:01:17.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.100+0000 7f90ef425700 1 -- 172.21.15.6:0/1150850714 wait complete. 2023-11-28T13:01:17.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.101+0000 7f90ef425700 1 Processor -- start 2023-11-28T13:01:17.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.102+0000 7f90ef425700 1 -- start start 2023-11-28T13:01:17.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.102+0000 7f90ef425700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e806f8b0 0x7f90e8156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.102+0000 7f90ef425700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90e80b2bc0 0x7f90e81dfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.102+0000 7f90ef425700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90e80b62d0 0x7f90e81dff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.102+0000 7f90ef425700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80ca250 con 0x7f90e806f8b0 2023-11-28T13:01:17.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.102+0000 7f90ef425700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80ad5e0 con 0x7f90e80b2bc0 2023-11-28T13:01:17.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.102+0000 7f90ef425700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80ae360 con 0x7f90e80b62d0 2023-11-28T13:01:17.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.102+0000 7f90ed1c1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e806f8b0 0x7f90e8156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.102+0000 7f90ed1c1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e806f8b0 0x7f90e8156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59484/0 (socket says 172.21.15.6:59484) 2023-11-28T13:01:17.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.102+0000 7f90ed1c1700 1 -- 172.21.15.6:0/242367454 learned_addr learned my addr 172.21.15.6:0/242367454 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:17.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ed9c2700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90e80b62d0 0x7f90e81dff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ec9c0700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90e80b2bc0 0x7f90e81dfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ed1c1700 1 -- 172.21.15.6:0/242367454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90e80b62d0 msgr2=0x7f90e81dff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ed1c1700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90e80b62d0 0x7f90e81dff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ed1c1700 1 -- 172.21.15.6:0/242367454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90e80b2bc0 msgr2=0x7f90e81dfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ed1c1700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90e80b2bc0 0x7f90e81dfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ed1c1700 1 -- 172.21.15.6:0/242367454 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e0009b30 con 0x7f90e806f8b0 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ed1c1700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e806f8b0 0x7f90e8156a80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f90e400e9b0 tx=0x7f90e400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ddffb700 1 -- 172.21.15.6:0/242367454 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4009710 con 0x7f90e806f8b0 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ddffb700 1 -- 172.21.15.6:0/242367454 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e400c850 con 0x7f90e806f8b0 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ef425700 1 -- 172.21.15.6:0/242367454 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e80b66a0 con 0x7f90e806f8b0 2023-11-28T13:01:17.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ddffb700 1 -- 172.21.15.6:0/242367454 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e400cbc0 con 0x7f90e806f8b0 2023-11-28T13:01:17.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.103+0000 7f90ef425700 1 -- 172.21.15.6:0/242367454 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e80cffb0 con 0x7f90e806f8b0 2023-11-28T13:01:17.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.104+0000 7f90ef425700 1 -- 172.21.15.6:0/242367454 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e806fd20 con 0x7f90e806f8b0 2023-11-28T13:01:17.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.105+0000 7f90ddffb700 1 -- 172.21.15.6:0/242367454 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f90e4019030 con 0x7f90e806f8b0 2023-11-28T13:01:17.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.105+0000 7f90ddffb700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f90cc0682f0 0x7f90cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.105+0000 7f90ddffb700 1 -- 172.21.15.6:0/242367454 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f90e4085fe0 con 0x7f90e806f8b0 2023-11-28T13:01:17.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.105+0000 7f90ec9c0700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f90cc0682f0 0x7f90cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.106+0000 7f90ec9c0700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f90cc0682f0 0x7f90cc06a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f90e81f59b0 tx=0x7f90d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.110+0000 7f90ddffb700 1 -- 172.21.15.6:0/242367454 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90e4052510 con 0x7f90e806f8b0 2023-11-28T13:01:17.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.305+0000 7f90ef425700 1 -- 172.21.15.6:0/242367454 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90e80ae1b0 con 0x7f90e806f8b0 2023-11-28T13:01:17.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.306+0000 7f90ddffb700 1 -- 172.21.15.6:0/242367454 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f90e4021020 con 0x7f90e806f8b0 2023-11-28T13:01:17.307 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:17.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 -- 172.21.15.6:0/242367454 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f90cc0682f0 msgr2=0x7f90cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f90cc0682f0 0x7f90cc06a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f90e81f59b0 tx=0x7f90d8008040 comp rx=0 tx=0).stop 2023-11-28T13:01:17.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 -- 172.21.15.6:0/242367454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e806f8b0 msgr2=0x7f90e8156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e806f8b0 0x7f90e8156a80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f90e400e9b0 tx=0x7f90e400ed70 comp rx=0 tx=0).stop 2023-11-28T13:01:17.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 -- 172.21.15.6:0/242367454 shutdown_connections 2023-11-28T13:01:17.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f90cc0682f0 0x7f90cc06a7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f90e80b62d0 0x7f90e81dff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f90e806f8b0 0x7f90e8156a80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 --2- 172.21.15.6:0/242367454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f90e80b2bc0 0x7f90e81dfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 -- 172.21.15.6:0/242367454 >> 172.21.15.6:0/242367454 conn(0x7f90e80cd680 msgr2=0x7f90e80b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:17.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 -- 172.21.15.6:0/242367454 shutdown_connections 2023-11-28T13:01:17.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.308+0000 7f90ef425700 1 -- 172.21.15.6:0/242367454 wait complete. 2023-11-28T13:01:17.321 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:17.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.449+0000 7fa649398700 1 Processor -- start 2023-11-28T13:01:17.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.449+0000 7fa649398700 1 -- start start 2023-11-28T13:01:17.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.450+0000 7fa649398700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa6440ce560 0x7fa64406f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.450+0000 7fa649398700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64406e470 con 0x7fa6441589c0 2023-11-28T13:01:17.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.450+0000 7fa649398700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64406de60 con 0x7fa644158d90 2023-11-28T13:01:17.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.450+0000 7fa649398700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6440ceb40 con 0x7fa6440ce560 2023-11-28T13:01:17.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.450+0000 7fa642ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fa6441589c0 0x7fa64406aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46812/0 (socket says 172.21.15.6:46812) 2023-11-28T13:01:17.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.450+0000 7fa642ffd700 1 -- 172.21.15.6:0/3125136310 learned_addr learned my addr 172.21.15.6:0/3125136310 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:17.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.450+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486271707 0 0) 0x7fa64406e470 con 0x7fa6441589c0 2023-11-28T13:01:17.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6437fe700 1 --2- 172.21.15.6:0/3125136310 >> v2:172.21.15.61:3300/0 conn(0x7fa6440ce560 0x7fa64406f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa62c0035b0 con 0x7fa6441589c0 2023-11-28T13:01:17.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2631467095 0 0) 0x7fa62c0035b0 con 0x7fa6441589c0 2023-11-28T13:01:17.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 >> v2:172.21.15.61:3300/0 conn(0x7fa6440ce560 msgr2=0x7fa64406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 --2- 172.21.15.6:0/3125136310 >> v2:172.21.15.61:3300/0 conn(0x7fa6440ce560 0x7fa64406f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 >> v1:172.21.15.61:6789/0 conn(0x7fa644158d90 legacy=0x7fa64406e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6440cee20 con 0x7fa6441589c0 2023-11-28T13:01:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fa634001670 con 0x7fa6441589c0 2023-11-28T13:01:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 >> v1:172.21.15.6:6789/0 conn(0x7fa6441589c0 legacy=0x7fa64406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa62c003a00 0x7fa62c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa62c004850 0x7fa62c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa62c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440ceb40 con 0x7fa62c005200 2023-11-28T13:01:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c00e290 con 0x7fa62c004850 2023-11-28T13:01:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c00e450 con 0x7fa62c003a00 2023-11-28T13:01:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6437fe700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa62c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6427fc700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa62c004850 0x7fa62c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa642ffd700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa62c003a00 0x7fa62c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6437fe700 1 -- 172.21.15.6:0/3125136310 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa62c003a00 msgr2=0x7fa62c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6437fe700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa62c003a00 0x7fa62c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6437fe700 1 -- 172.21.15.6:0/3125136310 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa62c004850 msgr2=0x7fa62c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6437fe700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa62c004850 0x7fa62c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6437fe700 1 -- 172.21.15.6:0/3125136310 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c0035b0 con 0x7fa62c005200 2023-11-28T13:01:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6427fc700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa62c004850 0x7fa62c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6437fe700 1 -- 172.21.15.6:0/3125136310 reap_dead start 2023-11-28T13:01:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.451+0000 7fa6437fe700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa62c00ba10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009fc0 tx=0x7fa62c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc36196196a4512c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.458 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.452+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c003a20 con 0x7fa62c005200 2023-11-28T13:01:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.452+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c00f040 con 0x7fa62c005200 2023-11-28T13:01:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.452+0000 7fa641ffb700 1 -- 172.21.15.6:0/3125136310 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c003de0 con 0x7fa62c005200 2023-11-28T13:01:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.452+0000 7fa649398700 1 -- 172.21.15.6:0/3125136310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 msgr2=0x7fa62c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.452+0000 7fa649398700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa62c00ba10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009fc0 tx=0x7fa62c005880 comp rx=0 tx=0).stop 2023-11-28T13:01:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.452+0000 7fa649398700 1 -- 172.21.15.6:0/3125136310 shutdown_connections 2023-11-28T13:01:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.452+0000 7fa649398700 1 --2- 172.21.15.6:0/3125136310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa62c00ba10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.452+0000 7fa649398700 1 -- 172.21.15.6:0/3125136310 >> 172.21.15.6:0/3125136310 conn(0x7fa6440b0df0 msgr2=0x7fa6440b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:17.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.452+0000 7fa649398700 1 -- 172.21.15.6:0/3125136310 shutdown_connections 2023-11-28T13:01:17.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.452+0000 7fa649398700 1 -- 172.21.15.6:0/3125136310 wait complete. 2023-11-28T13:01:17.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.454+0000 7fa649398700 1 Processor -- start 2023-11-28T13:01:17.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.454+0000 7fa649398700 1 -- start start 2023-11-28T13:01:17.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa649398700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa644150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa649398700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6440b2130 0x7fa6440b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa649398700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6440c5770 0x7fa6440c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa649398700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b28a0 con 0x7fa62c005200 2023-11-28T13:01:17.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa649398700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440c6150 con 0x7fa6440b2130 2023-11-28T13:01:17.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa649398700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b25c0 con 0x7fa6440c5770 2023-11-28T13:01:17.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa642ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa644150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa642ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa644150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59504/0 (socket says 172.21.15.6:59504) 2023-11-28T13:01:17.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa642ffd700 1 -- 172.21.15.6:0/3620206045 learned_addr learned my addr 172.21.15.6:0/3620206045 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:17.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa6427fc700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6440b2130 0x7fa6440b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa6437fe700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6440c5770 0x7fa6440c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa642ffd700 1 -- 172.21.15.6:0/3620206045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6440c5770 msgr2=0x7fa6440c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa642ffd700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6440c5770 0x7fa6440c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa642ffd700 1 -- 172.21.15.6:0/3620206045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6440b2130 msgr2=0x7fa6440b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa642ffd700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6440b2130 0x7fa6440b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa642ffd700 1 -- 172.21.15.6:0/3620206045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6440c9d90 con 0x7fa62c005200 2023-11-28T13:01:17.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.455+0000 7fa6427fc700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6440b2130 0x7fa6440b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:17.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.456+0000 7fa642ffd700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa644150d00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e7b0 tx=0x7fa63400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.456+0000 7fa62b7fe700 1 -- 172.21.15.6:0/3620206045 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634012590 con 0x7fa62c005200 2023-11-28T13:01:17.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.456+0000 7fa62b7fe700 1 -- 172.21.15.6:0/3620206045 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6340126f0 con 0x7fa62c005200 2023-11-28T13:01:17.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.456+0000 7fa649398700 1 -- 172.21.15.6:0/3620206045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64406e100 con 0x7fa62c005200 2023-11-28T13:01:17.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.456+0000 7fa62b7fe700 1 -- 172.21.15.6:0/3620206045 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634012aa0 con 0x7fa62c005200 2023-11-28T13:01:17.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.456+0000 7fa649398700 1 -- 172.21.15.6:0/3620206045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64406e900 con 0x7fa62c005200 2023-11-28T13:01:17.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.457+0000 7fa649398700 1 -- 172.21.15.6:0/3620206045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64415b960 con 0x7fa62c005200 2023-11-28T13:01:17.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.457+0000 7fa62b7fe700 1 -- 172.21.15.6:0/3620206045 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa634012c00 con 0x7fa62c005200 2023-11-28T13:01:17.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.458+0000 7fa62b7fe700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa6300682f0 0x7fa63006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.458+0000 7fa62b7fe700 1 -- 172.21.15.6:0/3620206045 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa634085df0 con 0x7fa62c005200 2023-11-28T13:01:17.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.458+0000 7fa6427fc700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa6300682f0 0x7fa63006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.458+0000 7fa6427fc700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa6300682f0 0x7fa63006a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa644154420 tx=0x7fa638009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.462+0000 7fa62b7fe700 1 -- 172.21.15.6:0/3620206045 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa64415b960 con 0x7fa62c005200 2023-11-28T13:01:17.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.654+0000 7fa649398700 1 -- 172.21.15.6:0/3620206045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6440c7420 con 0x7fa62c005200 2023-11-28T13:01:17.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.655+0000 7fa62b7fe700 1 -- 172.21.15.6:0/3620206045 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6440c7420 con 0x7fa62c005200 2023-11-28T13:01:17.656 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:17.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 -- 172.21.15.6:0/3620206045 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa6300682f0 msgr2=0x7fa63006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa6300682f0 0x7fa63006a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa644154420 tx=0x7fa638009040 comp rx=0 tx=0).stop 2023-11-28T13:01:17.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 -- 172.21.15.6:0/3620206045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 msgr2=0x7fa644150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa644150d00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa63400e7b0 tx=0x7fa63400eb70 comp rx=0 tx=0).stop 2023-11-28T13:01:17.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 -- 172.21.15.6:0/3620206045 shutdown_connections 2023-11-28T13:01:17.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa6300682f0 0x7fa63006a7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa6440c5770 0x7fa6440c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa62c005200 0x7fa644150d00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 --2- 172.21.15.6:0/3620206045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa6440b2130 0x7fa6440b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 -- 172.21.15.6:0/3620206045 >> 172.21.15.6:0/3620206045 conn(0x7fa6440b0df0 msgr2=0x7fa6440c9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:17.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 -- 172.21.15.6:0/3620206045 shutdown_connections 2023-11-28T13:01:17.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.657+0000 7fa649398700 1 -- 172.21.15.6:0/3620206045 wait complete. 2023-11-28T13:01:17.670 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.796+0000 7f0c8d3a3700 1 Processor -- start 2023-11-28T13:01:17.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.797+0000 7f0c8d3a3700 1 -- start start 2023-11-28T13:01:17.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.797+0000 7f0c8d3a3700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0c880cf5f0 0x7f0c880caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.797+0000 7f0c8d3a3700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0c880cb3e0 0x7f0c8806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.797+0000 7f0c8d3a3700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c880ce550 con 0x7f0c880b1630 2023-11-28T13:01:17.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.797+0000 7f0c8d3a3700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880ce830 con 0x7f0c880cf5f0 2023-11-28T13:01:17.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.797+0000 7f0c8d3a3700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880ceb10 con 0x7f0c880cb3e0 2023-11-28T13:01:17.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.797+0000 7f0c867fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0c880cf5f0 0x7f0c880caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.797+0000 7f0c867fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0c880cf5f0 0x7f0c880caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59514/0 (socket says 172.21.15.6:59514) 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c867fc700 1 -- 172.21.15.6:0/315508903 learned_addr learned my addr 172.21.15.6:0/315508903 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c877fe700 1 --2- 172.21.15.6:0/315508903 >> v2:172.21.15.61:3300/0 conn(0x7f0c880cb3e0 0x7f0c8806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c867fc700 1 -- 172.21.15.6:0/315508903 >> v2:172.21.15.61:3300/0 conn(0x7f0c880cb3e0 msgr2=0x7f0c8806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c867fc700 1 --2- 172.21.15.6:0/315508903 >> v2:172.21.15.61:3300/0 conn(0x7f0c880cb3e0 0x7f0c8806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c867fc700 1 -- 172.21.15.6:0/315508903 >> v1:172.21.15.61:6789/0 conn(0x7f0c880b1630 legacy=0x7f0c880b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c867fc700 1 -- 172.21.15.6:0/315508903 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c880cedf0 con 0x7f0c880cf5f0 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c877fe700 1 --2- 172.21.15.6:0/315508903 >> v2:172.21.15.61:3300/0 conn(0x7f0c880cb3e0 0x7f0c8806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c867fc700 1 --2- 172.21.15.6:0/315508903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880cf5f0 0x7f0c880caea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0c780098f0 tx=0x7f0c78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faaf78d2419598a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c85ffb700 1 -- 172.21.15.6:0/315508903 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c78004230 con 0x7f0c880cf5f0 2023-11-28T13:01:17.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c85ffb700 1 -- 172.21.15.6:0/315508903 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c78004390 con 0x7f0c880cf5f0 2023-11-28T13:01:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.798+0000 7f0c85ffb700 1 -- 172.21.15.6:0/315508903 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c780036a0 con 0x7f0c880cf5f0 2023-11-28T13:01:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.799+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/315508903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880cf5f0 msgr2=0x7f0c880caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.799+0000 7f0c8d3a3700 1 --2- 172.21.15.6:0/315508903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880cf5f0 0x7f0c880caea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0c780098f0 tx=0x7f0c78009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.799+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/315508903 shutdown_connections 2023-11-28T13:01:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.799+0000 7f0c8d3a3700 1 --2- 172.21.15.6:0/315508903 >> v2:172.21.15.61:3300/0 conn(0x7f0c880cb3e0 0x7f0c8806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.799+0000 7f0c8d3a3700 1 --2- 172.21.15.6:0/315508903 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880cf5f0 0x7f0c880caea0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.799+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/315508903 >> 172.21.15.6:0/315508903 conn(0x7f0c880cc630 msgr2=0x7f0c880cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.799+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/315508903 shutdown_connections 2023-11-28T13:01:17.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.799+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/315508903 wait complete. 2023-11-28T13:01:17.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.800+0000 7f0c8d3a3700 1 Processor -- start 2023-11-28T13:01:17.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.800+0000 7f0c8d3a3700 1 -- start start 2023-11-28T13:01:17.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c8d3a3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880b1630 0x7f0c8815b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c8d3a3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c880cb3e0 0x7f0c880b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c8d3a3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c880cf5f0 0x7f0c880b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c8d3a3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8815c220 con 0x7f0c880b1630 2023-11-28T13:01:17.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c8d3a3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880c84e0 con 0x7f0c880cf5f0 2023-11-28T13:01:17.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c8d3a3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8815bc90 con 0x7f0c880cb3e0 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c86ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880b1630 0x7f0c8815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c86ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880b1630 0x7f0c8815b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59518/0 (socket says 172.21.15.6:59518) 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c86ffd700 1 -- 172.21.15.6:0/37536370 learned_addr learned my addr 172.21.15.6:0/37536370 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c877fe700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c880cf5f0 0x7f0c880b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c867fc700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c880cb3e0 0x7f0c880b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c86ffd700 1 -- 172.21.15.6:0/37536370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c880cb3e0 msgr2=0x7f0c880b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c86ffd700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c880cb3e0 0x7f0c880b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c86ffd700 1 -- 172.21.15.6:0/37536370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c880cf5f0 msgr2=0x7f0c880b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c86ffd700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c880cf5f0 0x7f0c880b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c86ffd700 1 -- 172.21.15.6:0/37536370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c78009550 con 0x7f0c880b1630 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c877fe700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c880cf5f0 0x7f0c880b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:17.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.801+0000 7f0c867fc700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c880cb3e0 0x7f0c880b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.802+0000 7f0c86ffd700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880b1630 0x7f0c8815b750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00c500 tx=0x7f0c7c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.802+0000 7f0c6f7fe700 1 -- 172.21.15.6:0/37536370 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c011790 con 0x7f0c880b1630 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.802+0000 7f0c6f7fe700 1 -- 172.21.15.6:0/37536370 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c0118f0 con 0x7f0c880b1630 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.802+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/37536370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c880ce140 con 0x7f0c880b1630 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.802+0000 7f0c6f7fe700 1 -- 172.21.15.6:0/37536370 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c011ca0 con 0x7f0c880b1630 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.802+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/37536370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c880cf0d0 con 0x7f0c880b1630 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.803+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/37536370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c880ce2d0 con 0x7f0c880b1630 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.803+0000 7f0c6f7fe700 1 -- 172.21.15.6:0/37536370 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0c7c010050 con 0x7f0c880b1630 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.804+0000 7f0c6f7fe700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0c740682f0 0x7f0c7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.804+0000 7f0c867fc700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0c740682f0 0x7f0c7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:17.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.804+0000 7f0c6f7fe700 1 -- 172.21.15.6:0/37536370 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0c7c085c80 con 0x7f0c880b1630 2023-11-28T13:01:17.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.804+0000 7f0c867fc700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0c740682f0 0x7f0c7406a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009130 tx=0x7f0c78008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:17.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.808+0000 7f0c6f7fe700 1 -- 172.21.15.6:0/37536370 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0c7c0521b0 con 0x7f0c880b1630 2023-11-28T13:01:18.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:17.999+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/37536370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0c880739f0 con 0x7f0c880b1630 2023-11-28T13:01:18.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.000+0000 7f0c6f7fe700 1 -- 172.21.15.6:0/37536370 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0c7c0557d0 con 0x7f0c880b1630 2023-11-28T13:01:18.001 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:18.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.001+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/37536370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0c740682f0 msgr2=0x7f0c7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.001+0000 7f0c8d3a3700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0c740682f0 0x7f0c7406a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009130 tx=0x7f0c78008030 comp rx=0 tx=0).stop 2023-11-28T13:01:18.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.001+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/37536370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880b1630 msgr2=0x7f0c8815b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.001+0000 7f0c8d3a3700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880b1630 0x7f0c8815b750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00c500 tx=0x7f0c7c00c8c0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.002+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/37536370 shutdown_connections 2023-11-28T13:01:18.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.002+0000 7f0c8d3a3700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0c880cf5f0 0x7f0c880b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.004 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.002+0000 7f0c8d3a3700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c880b1630 0x7f0c8815b750 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.004 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.002+0000 7f0c8d3a3700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0c740682f0 0x7f0c7406a7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.004 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.002+0000 7f0c8d3a3700 1 --2- 172.21.15.6:0/37536370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0c880cb3e0 0x7f0c880b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.004 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.002+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/37536370 >> 172.21.15.6:0/37536370 conn(0x7f0c880cc630 msgr2=0x7f0c8806beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:18.004 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.002+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/37536370 shutdown_connections 2023-11-28T13:01:18.004 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.002+0000 7f0c8d3a3700 1 -- 172.21.15.6:0/37536370 wait complete. 2023-11-28T13:01:18.015 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:18.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.140+0000 7f2c1df22700 1 Processor -- start 2023-11-28T13:01:18.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.140+0000 7f2c1df22700 1 -- start start 2023-11-28T13:01:18.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c1df22700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2c1806dda0 0x7f2c1815b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c1df22700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1806c3a0 con 0x7f2c1806bc30 2023-11-28T13:01:18.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c1df22700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1806c0c0 con 0x7f2c1806d9d0 2023-11-28T13:01:18.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c1df22700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c1815ab60 con 0x7f2c1806dda0 2023-11-28T13:01:18.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c16ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f2c1806bc30 0x7f2c180c9930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46848/0 (socket says 172.21.15.6:46848) 2023-11-28T13:01:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c16ffd700 1 -- 172.21.15.6:0/1272596892 learned_addr learned my addr 172.21.15.6:0/1272596892 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123751104 0 0) 0x7f2c1806c3a0 con 0x7f2c1806bc30 2023-11-28T13:01:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c17fff700 1 --2- 172.21.15.6:0/1272596892 >> v2:172.21.15.61:3300/0 conn(0x7f2c1806dda0 0x7f2c1815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c000035b0 con 0x7f2c1806bc30 2023-11-28T13:01:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2784646080 0 0) 0x7f2c000035b0 con 0x7f2c1806bc30 2023-11-28T13:01:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 >> v2:172.21.15.61:3300/0 conn(0x7f2c1806dda0 msgr2=0x7f2c1815b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 --2- 172.21.15.6:0/1272596892 >> v2:172.21.15.61:3300/0 conn(0x7f2c1806dda0 0x7f2c1815b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 >> v1:172.21.15.61:6789/0 conn(0x7f2c1806d9d0 legacy=0x7f2c18150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1815ae40 con 0x7f2c1806bc30 2023-11-28T13:01:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f2c0c001670 con 0x7f2c1806bc30 2023-11-28T13:01:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 >> v1:172.21.15.6:6789/0 conn(0x7f2c1806bc30 legacy=0x7f2c180c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c00003a00 0x7f2c00003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c00004850 0x7f2c00004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c00005200 0x7f2c0000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1815ab60 con 0x7f2c00005200 2023-11-28T13:01:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0000c080 con 0x7f2c00003a00 2023-11-28T13:01:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.141+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0000c240 con 0x7f2c00004850 2023-11-28T13:01:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c17fff700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c00005200 0x7f2c0000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c177fe700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c00003a00 0x7f2c00003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c16ffd700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c00004850 0x7f2c00004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c17fff700 1 -- 172.21.15.6:0/1272596892 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c00004850 msgr2=0x7f2c00004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:18.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c17fff700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c00004850 0x7f2c00004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c17fff700 1 -- 172.21.15.6:0/1272596892 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c00003a00 msgr2=0x7f2c00003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c17fff700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c00003a00 0x7f2c00003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c17fff700 1 -- 172.21.15.6:0/1272596892 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c000035b0 con 0x7f2c00005200 2023-11-28T13:01:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c16ffd700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c00004850 0x7f2c00004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c17fff700 1 -- 172.21.15.6:0/1272596892 reap_dead start 2023-11-28T13:01:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c17fff700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c00005200 0x7f2c0000ba10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009fc0 tx=0x7f2c00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6145e24b610176b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10003a20 con 0x7f2c00005200 2023-11-28T13:01:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1000f040 con 0x7f2c00005200 2023-11-28T13:01:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.142+0000 7f2c167fc700 1 -- 172.21.15.6:0/1272596892 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10003da0 con 0x7f2c00005200 2023-11-28T13:01:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.143+0000 7f2c1df22700 1 -- 172.21.15.6:0/1272596892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c00005200 msgr2=0x7f2c0000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.143+0000 7f2c1df22700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c00005200 0x7f2c0000ba10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009fc0 tx=0x7f2c00005880 comp rx=0 tx=0).stop 2023-11-28T13:01:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.143+0000 7f2c1df22700 1 -- 172.21.15.6:0/1272596892 shutdown_connections 2023-11-28T13:01:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.143+0000 7f2c1df22700 1 --2- 172.21.15.6:0/1272596892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c00005200 0x7f2c0000ba10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.143+0000 7f2c1df22700 1 -- 172.21.15.6:0/1272596892 >> 172.21.15.6:0/1272596892 conn(0x7f2c180b0dd0 msgr2=0x7f2c180b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.143+0000 7f2c1df22700 1 -- 172.21.15.6:0/1272596892 shutdown_connections 2023-11-28T13:01:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.143+0000 7f2c1df22700 1 -- 172.21.15.6:0/1272596892 wait complete. 2023-11-28T13:01:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.144+0000 7f2c1df22700 1 Processor -- start 2023-11-28T13:01:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.144+0000 7f2c1df22700 1 -- start start 2023-11-28T13:01:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c1df22700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c00005200 0x7f2c180cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c1df22700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c1814b620 0x7f2c180c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c1df22700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c1814ba90 0x7f2c180c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c1df22700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180c5c00 con 0x7f2c1814ba90 2023-11-28T13:01:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c1df22700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180ccf50 con 0x7f2c00005200 2023-11-28T13:01:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c1df22700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180cd230 con 0x7f2c1814b620 2023-11-28T13:01:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c17fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c1814ba90 0x7f2c180c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c17fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c1814ba90 0x7f2c180c56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59542/0 (socket says 172.21.15.6:59542) 2023-11-28T13:01:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c17fff700 1 -- 172.21.15.6:0/512720322 learned_addr learned my addr 172.21.15.6:0/512720322 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:18.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c177fe700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c00005200 0x7f2c180cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c16ffd700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c1814b620 0x7f2c180c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c17fff700 1 -- 172.21.15.6:0/512720322 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c1814b620 msgr2=0x7f2c180c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c17fff700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c1814b620 0x7f2c180c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c17fff700 1 -- 172.21.15.6:0/512720322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c00005200 msgr2=0x7f2c180cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c17fff700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c00005200 0x7f2c180cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c17fff700 1 -- 172.21.15.6:0/512720322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1806fac0 con 0x7f2c1814ba90 2023-11-28T13:01:18.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2c17fff700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c1814ba90 0x7f2c180c56c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2c180c2de0 tx=0x7f2c10009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.145+0000 7f2bfffff700 1 -- 172.21.15.6:0/512720322 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10003e10 con 0x7f2c1814ba90 2023-11-28T13:01:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.146+0000 7f2bfffff700 1 -- 172.21.15.6:0/512720322 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c10004250 con 0x7f2c1814ba90 2023-11-28T13:01:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.146+0000 7f2c1df22700 1 -- 172.21.15.6:0/512720322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1815b120 con 0x7f2c1814ba90 2023-11-28T13:01:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.146+0000 7f2bfffff700 1 -- 172.21.15.6:0/512720322 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10004600 con 0x7f2c1814ba90 2023-11-28T13:01:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.146+0000 7f2c1df22700 1 -- 172.21.15.6:0/512720322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1815b640 con 0x7f2c1814ba90 2023-11-28T13:01:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.147+0000 7f2c1df22700 1 -- 172.21.15.6:0/512720322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c180675a0 con 0x7f2c1814ba90 2023-11-28T13:01:18.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.147+0000 7f2bfffff700 1 -- 172.21.15.6:0/512720322 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2c10004760 con 0x7f2c1814ba90 2023-11-28T13:01:18.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.148+0000 7f2bfffff700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c040682f0 0x7f2c0406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.148+0000 7f2bfffff700 1 -- 172.21.15.6:0/512720322 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2c10088250 con 0x7f2c1814ba90 2023-11-28T13:01:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.148+0000 7f2c177fe700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c040682f0 0x7f2c0406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.148+0000 7f2c177fe700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c040682f0 0x7f2c0406a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c08007fd0 tx=0x7f2c08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.152+0000 7f2bfffff700 1 -- 172.21.15.6:0/512720322 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2c10054720 con 0x7f2c1814ba90 2023-11-28T13:01:18.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.365+0000 7f2c1df22700 1 -- 172.21.15.6:0/512720322 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2c180b1b70 con 0x7f2c1814ba90 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.365+0000 7f2bfffff700 1 -- 172.21.15.6:0/512720322 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2c10018020 con 0x7f2c1814ba90 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 -- 172.21.15.6:0/512720322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c040682f0 msgr2=0x7f2c0406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c040682f0 0x7f2c0406a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2c08007fd0 tx=0x7f2c08009040 comp rx=0 tx=0).stop 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 -- 172.21.15.6:0/512720322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c1814ba90 msgr2=0x7f2c180c56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c1814ba90 0x7f2c180c56c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2c180c2de0 tx=0x7f2c10009750 comp rx=0 tx=0).stop 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 -- 172.21.15.6:0/512720322 shutdown_connections 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c1814ba90 0x7f2c180c56c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c00005200 0x7f2c180cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c040682f0 0x7f2c0406a7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 --2- 172.21.15.6:0/512720322 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c1814b620 0x7f2c180c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 -- 172.21.15.6:0/512720322 >> 172.21.15.6:0/512720322 conn(0x7f2c180b0dd0 msgr2=0x7f2c18159ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:18.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 -- 172.21.15.6:0/512720322 shutdown_connections 2023-11-28T13:01:18.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.367+0000 7f2c1df22700 1 -- 172.21.15.6:0/512720322 wait complete. 2023-11-28T13:01:18.370 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:01:18.380 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:01:18.381 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:18.398 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:18.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.561+0000 7f6a612d3700 1 Processor -- start 2023-11-28T13:01:18.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.561+0000 7f6a612d3700 1 -- start start 2023-11-28T13:01:18.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a612d3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a5c15b5e0 0x7f6a5c0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a612d3700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6a5c0b2ba0 0x7f6a5c15bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a612d3700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c0b3000 con 0x7f6a5c143fa0 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a612d3700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c15abd0 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a612d3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c15b150 con 0x7f6a5c15b5e0 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a5b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6a5c0b2ba0 0x7f6a5c15bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a5b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6a5c0b2ba0 0x7f6a5c15bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59550/0 (socket says 172.21.15.6:59550) 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a5b7fe700 1 -- 172.21.15.6:0/1707971199 learned_addr learned my addr 172.21.15.6:0/1707971199 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a5a7fc700 1 --2- 172.21.15.6:0/1707971199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a5c15b5e0 0x7f6a5c0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a5b7fe700 1 -- 172.21.15.6:0/1707971199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a5c15b5e0 msgr2=0x7f6a5c0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a5b7fe700 1 --2- 172.21.15.6:0/1707971199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a5c15b5e0 0x7f6a5c0b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a5b7fe700 1 -- 172.21.15.6:0/1707971199 >> v1:172.21.15.61:6789/0 conn(0x7f6a5c143fa0 legacy=0x7f6a5c15ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.562+0000 7f6a5b7fe700 1 -- 172.21.15.6:0/1707971199 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0cb230 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a5b7fe700 1 --2- 172.21.15.6:0/1707971199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 0x7f6a5c15bd90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009e50 tx=0x7f6a5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f73f6af4ec27554 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a59ffb700 1 -- 172.21.15.6:0/1707971199 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400e070 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a59ffb700 1 -- 172.21.15.6:0/1707971199 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5400b910 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a59ffb700 1 -- 172.21.15.6:0/1707971199 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400bcc0 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a612d3700 1 -- 172.21.15.6:0/1707971199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 msgr2=0x7f6a5c15bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a612d3700 1 --2- 172.21.15.6:0/1707971199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 0x7f6a5c15bd90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009e50 tx=0x7f6a5400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:18.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a612d3700 1 -- 172.21.15.6:0/1707971199 shutdown_connections 2023-11-28T13:01:18.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a612d3700 1 --2- 172.21.15.6:0/1707971199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 0x7f6a5c15bd90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a612d3700 1 --2- 172.21.15.6:0/1707971199 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a5c15b5e0 0x7f6a5c0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a612d3700 1 -- 172.21.15.6:0/1707971199 >> 172.21.15.6:0/1707971199 conn(0x7f6a5c06e9b0 msgr2=0x7f6a5c0cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:18.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a612d3700 1 -- 172.21.15.6:0/1707971199 shutdown_connections 2023-11-28T13:01:18.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.563+0000 7f6a612d3700 1 -- 172.21.15.6:0/1707971199 wait complete. 2023-11-28T13:01:18.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.565+0000 7f6a612d3700 1 Processor -- start 2023-11-28T13:01:18.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.565+0000 7f6a612d3700 1 -- start start 2023-11-28T13:01:18.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.565+0000 7f6a612d3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 0x7f6a5c151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.565+0000 7f6a612d3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5c143fa0 0x7f6a5c1ef960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.565+0000 7f6a612d3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a5c15b5e0 0x7f6a5c1583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.565+0000 7f6a612d3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0c8d60 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.565+0000 7f6a612d3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c1592f0 con 0x7f6a5c143fa0 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.565+0000 7f6a612d3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c1595d0 con 0x7f6a5c15b5e0 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 0x7f6a5c151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 0x7f6a5c151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59552/0 (socket says 172.21.15.6:59552) 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5affd700 1 -- 172.21.15.6:0/3980181096 learned_addr learned my addr 172.21.15.6:0/3980181096 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5b7fe700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a5c15b5e0 0x7f6a5c1583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5a7fc700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5c143fa0 0x7f6a5c1ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5affd700 1 -- 172.21.15.6:0/3980181096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a5c15b5e0 msgr2=0x7f6a5c1583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5affd700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a5c15b5e0 0x7f6a5c1583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5affd700 1 -- 172.21.15.6:0/3980181096 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5c143fa0 msgr2=0x7f6a5c1ef960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5affd700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5c143fa0 0x7f6a5c1ef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5affd700 1 -- 172.21.15.6:0/3980181096 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54009b30 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a5affd700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 0x7f6a5c151dd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000c4e0 tx=0x7f6a5000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a437fe700 1 -- 172.21.15.6:0/3980181096 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a500052d0 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a437fe700 1 -- 172.21.15.6:0/3980181096 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a50005430 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a612d3700 1 -- 172.21.15.6:0/3980181096 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c0cb4d0 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a437fe700 1 -- 172.21.15.6:0/3980181096 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a500038c0 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.566+0000 7f6a612d3700 1 -- 172.21.15.6:0/3980181096 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c0cba90 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.567+0000 7f6a612d3700 1 -- 172.21.15.6:0/3980181096 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c0cb7b0 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.568+0000 7f6a437fe700 1 -- 172.21.15.6:0/3980181096 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a50019030 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.568+0000 7f6a437fe700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a480682f0 0x7f6a4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.568+0000 7f6a437fe700 1 -- 172.21.15.6:0/3980181096 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6a50085370 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.568+0000 7f6a5a7fc700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a480682f0 0x7f6a4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.569+0000 7f6a5a7fc700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a480682f0 0x7f6a4806a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009a10 tx=0x7f6a4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.572+0000 7f6a437fe700 1 -- 172.21.15.6:0/3980181096 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a500518a0 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.771+0000 7f6a612d3700 1 -- 172.21.15.6:0/3980181096 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a5c068570 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.773+0000 7f6a437fe700 1 -- 172.21.15.6:0/3980181096 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19690 (secure 0 0 0) 0x7f6a50054ec0 con 0x7f6a5c0b2ba0 2023-11-28T13:01:18.774 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:18.774 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:10.053012+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:18.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 -- 172.21.15.6:0/3980181096 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a480682f0 msgr2=0x7f6a4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a480682f0 0x7f6a4806a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009a10 tx=0x7f6a4c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:18.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 -- 172.21.15.6:0/3980181096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 msgr2=0x7f6a5c151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 0x7f6a5c151dd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000c4e0 tx=0x7f6a5000c8a0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 -- 172.21.15.6:0/3980181096 shutdown_connections 2023-11-28T13:01:18.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a480682f0 0x7f6a4806a7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a5c15b5e0 0x7f6a5c1583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a5c0b2ba0 0x7f6a5c151dd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 --2- 172.21.15.6:0/3980181096 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5c143fa0 0x7f6a5c1ef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 -- 172.21.15.6:0/3980181096 >> 172.21.15.6:0/3980181096 conn(0x7f6a5c06e9b0 msgr2=0x7f6a5c152700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:18.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 -- 172.21.15.6:0/3980181096 shutdown_connections 2023-11-28T13:01:18.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.775+0000 7f6a612d3700 1 -- 172.21.15.6:0/3980181096 wait complete. 2023-11-28T13:01:18.789 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:18.790 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.915+0000 7ff3b1f32700 1 Processor -- start 2023-11-28T13:01:18.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.915+0000 7ff3b1f32700 1 -- start start 2023-11-28T13:01:18.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.916+0000 7ff3b1f32700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff3ac159500 0x7ff3ac0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.916+0000 7ff3b1f32700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac15e040 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.916+0000 7ff3b1f32700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac0ceff0 con 0x7ff3ac0b1d40 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.916+0000 7ff3b1f32700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac0cf2d0 con 0x7ff3ac159500 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.916+0000 7ff3ab7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff3ac159500 0x7ff3ac0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.916+0000 7ff3aaffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff3ac15dbf0 0x7ff3ac0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46880/0 (socket says 172.21.15.6:46880) 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.916+0000 7ff3aaffd700 1 -- 172.21.15.6:0/1363944922 learned_addr learned my addr 172.21.15.6:0/1363944922 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.917+0000 7ff3ab7fe700 1 -- 172.21.15.6:0/1363944922 >> v1:172.21.15.61:6789/0 conn(0x7ff3ac0b1d40 legacy=0x7ff3ac06bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.917+0000 7ff3ab7fe700 1 -- 172.21.15.6:0/1363944922 >> v1:172.21.15.6:6789/0 conn(0x7ff3ac15dbf0 legacy=0x7ff3ac0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.917+0000 7ff3ab7fe700 1 -- 172.21.15.6:0/1363944922 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac15d110 con 0x7ff3ac159500 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.917+0000 7ff3aa7fc700 1 -- 172.21.15.6:0/1363944922 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910540388 0 0) 0x7ff3ac15e040 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.917+0000 7ff3ab7fe700 1 --2- 172.21.15.6:0/1363944922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac159500 0x7ff3ac0cd140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009840 tx=0x7ff39c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0320fab8c800bff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.917+0000 7ff3aa7fc700 1 -- 172.21.15.6:0/1363944922 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c004230 con 0x7ff3ac159500 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.917+0000 7ff3aa7fc700 1 -- 172.21.15.6:0/1363944922 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c004390 con 0x7ff3ac159500 2023-11-28T13:01:18.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.917+0000 7ff3aa7fc700 1 -- 172.21.15.6:0/1363944922 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0036a0 con 0x7ff3ac159500 2023-11-28T13:01:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.918+0000 7ff3b1f32700 1 -- 172.21.15.6:0/1363944922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac159500 msgr2=0x7ff3ac0cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.918+0000 7ff3b1f32700 1 --2- 172.21.15.6:0/1363944922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac159500 0x7ff3ac0cd140 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009840 tx=0x7ff39c009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.918+0000 7ff3b1f32700 1 -- 172.21.15.6:0/1363944922 shutdown_connections 2023-11-28T13:01:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.918+0000 7ff3b1f32700 1 --2- 172.21.15.6:0/1363944922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac159500 0x7ff3ac0cd140 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.918+0000 7ff3b1f32700 1 -- 172.21.15.6:0/1363944922 >> 172.21.15.6:0/1363944922 conn(0x7ff3ac0cc630 msgr2=0x7ff3ac0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.918+0000 7ff3b1f32700 1 -- 172.21.15.6:0/1363944922 shutdown_connections 2023-11-28T13:01:18.921 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.918+0000 7ff3b1f32700 1 -- 172.21.15.6:0/1363944922 wait complete. 2023-11-28T13:01:18.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.919+0000 7ff3b1f32700 1 Processor -- start 2023-11-28T13:01:18.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.919+0000 7ff3b1f32700 1 -- start start 2023-11-28T13:01:18.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3b1f32700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff3ac0b1d40 0x7ff3ac1ec700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3b1f32700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3ac159500 0x7ff3ac0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3b1f32700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac15dbf0 0x7ff3ac0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3b1f32700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1ecc40 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3b1f32700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1f2b00 con 0x7ff3ac159500 2023-11-28T13:01:18.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3b1f32700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac1f2de0 con 0x7ff3ac0b1d40 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3abfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac15dbf0 0x7ff3ac0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3abfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac15dbf0 0x7ff3ac0b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59570/0 (socket says 172.21.15.6:59570) 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3abfff700 1 -- 172.21.15.6:0/2170595204 learned_addr learned my addr 172.21.15.6:0/2170595204 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3aaffd700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3ac159500 0x7ff3ac0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3ab7fe700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff3ac0b1d40 0x7ff3ac1ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3abfff700 1 -- 172.21.15.6:0/2170595204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff3ac0b1d40 msgr2=0x7ff3ac1ec700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3abfff700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff3ac0b1d40 0x7ff3ac1ec700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3abfff700 1 -- 172.21.15.6:0/2170595204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3ac159500 msgr2=0x7ff3ac0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3abfff700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3ac159500 0x7ff3ac0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3abfff700 1 -- 172.21.15.6:0/2170595204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c0094a0 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.920+0000 7ff3abfff700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac15dbf0 0x7ff3ac0b50f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400cc10 tx=0x7ff3a4004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.921+0000 7ff3a3fff700 1 -- 172.21.15.6:0/2170595204 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400ab80 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.921+0000 7ff3a3fff700 1 -- 172.21.15.6:0/2170595204 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a400ace0 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.921+0000 7ff3b1f32700 1 -- 172.21.15.6:0/2170595204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac15d6d0 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.921+0000 7ff3a3fff700 1 -- 172.21.15.6:0/2170595204 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4012660 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.921+0000 7ff3b1f32700 1 -- 172.21.15.6:0/2170595204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac15d3f0 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.922+0000 7ff3b1f32700 1 -- 172.21.15.6:0/2170595204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac1598d0 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.922+0000 7ff3a3fff700 1 -- 172.21.15.6:0/2170595204 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff3a4022070 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.922+0000 7ff3a3fff700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff38c0682f0 0x7ff38c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:18.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.923+0000 7ff3ab7fe700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff38c0682f0 0x7ff38c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:18.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.923+0000 7ff3a3fff700 1 -- 172.21.15.6:0/2170595204 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff3a40865b0 con 0x7ff3ac15dbf0 2023-11-28T13:01:18.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.923+0000 7ff3ab7fe700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff38c0682f0 0x7ff38c06a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac06c380 tx=0x7ff39c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:18.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:18.927+0000 7ff3a3fff700 1 -- 172.21.15.6:0/2170595204 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff3a4052ae0 con 0x7ff3ac15dbf0 2023-11-28T13:01:19.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.119+0000 7ff3b1f32700 1 -- 172.21.15.6:0/2170595204 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff3ac0739f0 con 0x7ff3ac15dbf0 2023-11-28T13:01:19.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.120+0000 7ff3a3fff700 1 -- 172.21.15.6:0/2170595204 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff3a4016020 con 0x7ff3ac15dbf0 2023-11-28T13:01:19.121 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:19.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 -- 172.21.15.6:0/2170595204 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff38c0682f0 msgr2=0x7ff38c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff38c0682f0 0x7ff38c06a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac06c380 tx=0x7ff39c01c040 comp rx=0 tx=0).stop 2023-11-28T13:01:19.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 -- 172.21.15.6:0/2170595204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac15dbf0 msgr2=0x7ff3ac0b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac15dbf0 0x7ff3ac0b50f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400cc10 tx=0x7ff3a4004990 comp rx=0 tx=0).stop 2023-11-28T13:01:19.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 -- 172.21.15.6:0/2170595204 shutdown_connections 2023-11-28T13:01:19.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff3ac15dbf0 0x7ff3ac0b50f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff38c0682f0 0x7ff38c06a7a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff3ac0b1d40 0x7ff3ac1ec700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 --2- 172.21.15.6:0/2170595204 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3ac159500 0x7ff3ac0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 -- 172.21.15.6:0/2170595204 >> 172.21.15.6:0/2170595204 conn(0x7ff3ac0cc630 msgr2=0x7ff3ac0c5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:19.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 -- 172.21.15.6:0/2170595204 shutdown_connections 2023-11-28T13:01:19.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.122+0000 7ff3b1f32700 1 -- 172.21.15.6:0/2170595204 wait complete. 2023-11-28T13:01:19.136 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:19.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.262+0000 7f809e087700 1 Processor -- start 2023-11-28T13:01:19.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.263+0000 7f809e087700 1 -- start start 2023-11-28T13:01:19.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.263+0000 7f809e087700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8098165670 0x7f8098167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.263+0000 7f809e087700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80980ce550 con 0x7f80980b1630 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.263+0000 7f809e087700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80980ce830 con 0x7f80980cf5f0 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.263+0000 7f809e087700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80980ceb10 con 0x7f8098165670 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.263+0000 7f8097fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8098165670 0x7f8098167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f80977fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f80980b1630 0x7f80980b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46894/0 (socket says 172.21.15.6:46894) 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f8097fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8098165670 0x7f8098167a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59578/0 (socket says 172.21.15.6:59578) 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f80977fe700 1 -- 172.21.15.6:0/3021884156 learned_addr learned my addr 172.21.15.6:0/3021884156 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f8097fff700 1 -- 172.21.15.6:0/3021884156 >> v1:172.21.15.61:6789/0 conn(0x7f80980cf5f0 legacy=0x7f80980caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f8097fff700 1 -- 172.21.15.6:0/3021884156 >> v1:172.21.15.6:6789/0 conn(0x7f80980b1630 legacy=0x7f80980b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f8097fff700 1 -- 172.21.15.6:0/3021884156 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80980cedf0 con 0x7f8098165670 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f80967fc700 1 -- 172.21.15.6:0/3021884156 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329569662 0 0) 0x7f80980ce550 con 0x7f80980b1630 2023-11-28T13:01:19.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f8097fff700 1 --2- 172.21.15.6:0/3021884156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8098165670 0x7f8098167a50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8090009dc0 tx=0x7f809000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5c9bd19481f2f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f80967fc700 1 -- 172.21.15.6:0/3021884156 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809000e070 con 0x7f8098165670 2023-11-28T13:01:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f80967fc700 1 -- 172.21.15.6:0/3021884156 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809000bb50 con 0x7f8098165670 2023-11-28T13:01:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.264+0000 7f80967fc700 1 -- 172.21.15.6:0/3021884156 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8090003a20 con 0x7f8098165670 2023-11-28T13:01:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.265+0000 7f809e087700 1 -- 172.21.15.6:0/3021884156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8098165670 msgr2=0x7f8098167a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.265+0000 7f809e087700 1 --2- 172.21.15.6:0/3021884156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8098165670 0x7f8098167a50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8090009dc0 tx=0x7f809000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.265+0000 7f809e087700 1 -- 172.21.15.6:0/3021884156 shutdown_connections 2023-11-28T13:01:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.265+0000 7f809e087700 1 --2- 172.21.15.6:0/3021884156 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8098165670 0x7f8098167a50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.265+0000 7f809e087700 1 -- 172.21.15.6:0/3021884156 >> 172.21.15.6:0/3021884156 conn(0x7f80980cc630 msgr2=0x7f80980cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.265+0000 7f809e087700 1 -- 172.21.15.6:0/3021884156 shutdown_connections 2023-11-28T13:01:19.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.265+0000 7f809e087700 1 -- 172.21.15.6:0/3021884156 wait complete. 2023-11-28T13:01:19.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.266+0000 7f809e087700 1 Processor -- start 2023-11-28T13:01:19.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.266+0000 7f809e087700 1 -- start start 2023-11-28T13:01:19.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f809e087700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b1630 0x7f809814ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f809e087700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80980cf5f0 0x7f80980ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f809e087700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8098165670 0x7f8098153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f809e087700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8098154d40 con 0x7f80980b1630 2023-11-28T13:01:19.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f809e087700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80981540c0 con 0x7f8098165670 2023-11-28T13:01:19.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f809e087700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80981543a0 con 0x7f80980cf5f0 2023-11-28T13:01:19.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f80977fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b1630 0x7f809814ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f80977fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b1630 0x7f809814ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59580/0 (socket says 172.21.15.6:59580) 2023-11-28T13:01:19.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f80977fe700 1 -- 172.21.15.6:0/1614846588 learned_addr learned my addr 172.21.15.6:0/1614846588 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:19.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f8097fff700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8098165670 0x7f8098153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f8096ffd700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80980cf5f0 0x7f80980ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f80977fe700 1 -- 172.21.15.6:0/1614846588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80980cf5f0 msgr2=0x7f80980ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f80977fe700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80980cf5f0 0x7f80980ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f80977fe700 1 -- 172.21.15.6:0/1614846588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8098165670 msgr2=0x7f8098153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f80977fe700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8098165670 0x7f8098153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f80977fe700 1 -- 172.21.15.6:0/1614846588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8090009a70 con 0x7f80980b1630 2023-11-28T13:01:19.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f8096ffd700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80980cf5f0 0x7f80980ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:19.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.267+0000 7f80977fe700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b1630 0x7f809814ac40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80800011e0 tx=0x7f808000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:19.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.268+0000 7f808ffff700 1 -- 172.21.15.6:0/1614846588 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080004c80 con 0x7f80980b1630 2023-11-28T13:01:19.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.268+0000 7f808ffff700 1 -- 172.21.15.6:0/1614846588 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8080004de0 con 0x7f80980b1630 2023-11-28T13:01:19.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.268+0000 7f809e087700 1 -- 172.21.15.6:0/1614846588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80980cb5b0 con 0x7f80980b1630 2023-11-28T13:01:19.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.268+0000 7f808ffff700 1 -- 172.21.15.6:0/1614846588 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080007310 con 0x7f80980b1630 2023-11-28T13:01:19.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.268+0000 7f809e087700 1 -- 172.21.15.6:0/1614846588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80980cf0d0 con 0x7f80980b1630 2023-11-28T13:01:19.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.269+0000 7f809e087700 1 -- 172.21.15.6:0/1614846588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80980ce140 con 0x7f80980b1630 2023-11-28T13:01:19.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.269+0000 7f808ffff700 1 -- 172.21.15.6:0/1614846588 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f808001a020 con 0x7f80980b1630 2023-11-28T13:01:19.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.270+0000 7f808ffff700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80780682f0 0x7f807806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.270+0000 7f808ffff700 1 -- 172.21.15.6:0/1614846588 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8080085b70 con 0x7f80980b1630 2023-11-28T13:01:19.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.270+0000 7f8096ffd700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80780682f0 0x7f807806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.270+0000 7f8096ffd700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80780682f0 0x7f807806a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f80980ca160 tx=0x7f8088009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:19.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.274+0000 7f808ffff700 1 -- 172.21.15.6:0/1614846588 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f80800520a0 con 0x7f80980b1630 2023-11-28T13:01:19.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.469+0000 7f809e087700 1 -- 172.21.15.6:0/1614846588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80980739f0 con 0x7f80980b1630 2023-11-28T13:01:19.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.469+0000 7f808ffff700 1 -- 172.21.15.6:0/1614846588 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80800556c0 con 0x7f80980b1630 2023-11-28T13:01:19.471 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:19.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 -- 172.21.15.6:0/1614846588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80780682f0 msgr2=0x7f807806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80780682f0 0x7f807806a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f80980ca160 tx=0x7f8088009040 comp rx=0 tx=0).stop 2023-11-28T13:01:19.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 -- 172.21.15.6:0/1614846588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b1630 msgr2=0x7f809814ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b1630 0x7f809814ac40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80800011e0 tx=0x7f808000ca30 comp rx=0 tx=0).stop 2023-11-28T13:01:19.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 -- 172.21.15.6:0/1614846588 shutdown_connections 2023-11-28T13:01:19.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8098165670 0x7f8098153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80980b1630 0x7f809814ac40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80780682f0 0x7f807806a7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 --2- 172.21.15.6:0/1614846588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80980cf5f0 0x7f80980ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 -- 172.21.15.6:0/1614846588 >> 172.21.15.6:0/1614846588 conn(0x7f80980cc630 msgr2=0x7f80980cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:19.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.471+0000 7f809e087700 1 -- 172.21.15.6:0/1614846588 shutdown_connections 2023-11-28T13:01:19.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.472+0000 7f809e087700 1 -- 172.21.15.6:0/1614846588 wait complete. 2023-11-28T13:01:19.485 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:19.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.611+0000 7f8288eef700 1 Processor -- start 2023-11-28T13:01:19.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.611+0000 7f8288eef700 1 -- start start 2023-11-28T13:01:19.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8288eef700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8284159500 0x7f82840cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8288eef700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f828415dbf0 0x7f82840ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8288eef700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f82840b1630 0x7f82840b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8288eef700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82840b2010 con 0x7f82840b1630 2023-11-28T13:01:19.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8288eef700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82840cede0 con 0x7f8284159500 2023-11-28T13:01:19.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8288eef700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828415e040 con 0x7f828415dbf0 2023-11-28T13:01:19.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8282d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f82840b1630 0x7f82840b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8282d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f82840b1630 0x7f82840b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59592/0 (socket says 172.21.15.6:59592) 2023-11-28T13:01:19.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8282d9d700 1 -- 172.21.15.6:0/285052157 learned_addr learned my addr 172.21.15.6:0/285052157 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:19.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f828259c700 1 --2- 172.21.15.6:0/285052157 >> v2:172.21.15.61:3300/0 conn(0x7f8284159500 0x7f82840cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8282d9d700 1 -- 172.21.15.6:0/285052157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f828415dbf0 msgr2=0x7f82840ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8281d9b700 1 --2- 172.21.15.6:0/285052157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f828415dbf0 0x7f82840ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8282d9d700 1 --2- 172.21.15.6:0/285052157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f828415dbf0 0x7f82840ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8282d9d700 1 -- 172.21.15.6:0/285052157 >> v2:172.21.15.61:3300/0 conn(0x7f8284159500 msgr2=0x7f82840cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8282d9d700 1 --2- 172.21.15.6:0/285052157 >> v2:172.21.15.61:3300/0 conn(0x7f8284159500 0x7f82840cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8282d9d700 1 -- 172.21.15.6:0/285052157 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82840ceff0 con 0x7f82840b1630 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f828259c700 1 --2- 172.21.15.6:0/285052157 >> v2:172.21.15.61:3300/0 conn(0x7f8284159500 0x7f82840cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f8282d9d700 1 --2- 172.21.15.6:0/285052157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82840b1630 0x7f82840b1ad0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f827c00d150 tx=0x7f827c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c10f142085b43fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.612+0000 7f828159a700 1 -- 172.21.15.6:0/285052157 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c010070 con 0x7f82840b1630 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f828159a700 1 -- 172.21.15.6:0/285052157 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c00dd80 con 0x7f82840b1630 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f828159a700 1 -- 172.21.15.6:0/285052157 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00b310 con 0x7f82840b1630 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f8288eef700 1 -- 172.21.15.6:0/285052157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82840b1630 msgr2=0x7f82840b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f8288eef700 1 --2- 172.21.15.6:0/285052157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82840b1630 0x7f82840b1ad0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f827c00d150 tx=0x7f827c00d460 comp rx=0 tx=0).stop 2023-11-28T13:01:19.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f8288eef700 1 -- 172.21.15.6:0/285052157 shutdown_connections 2023-11-28T13:01:19.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f8288eef700 1 --2- 172.21.15.6:0/285052157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82840b1630 0x7f82840b1ad0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f8288eef700 1 --2- 172.21.15.6:0/285052157 >> v2:172.21.15.61:3300/0 conn(0x7f8284159500 0x7f82840cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f8288eef700 1 --2- 172.21.15.6:0/285052157 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f828415dbf0 0x7f82840ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f8288eef700 1 -- 172.21.15.6:0/285052157 >> 172.21.15.6:0/285052157 conn(0x7f82840cc630 msgr2=0x7f82840cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:19.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f8288eef700 1 -- 172.21.15.6:0/285052157 shutdown_connections 2023-11-28T13:01:19.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.613+0000 7f8288eef700 1 -- 172.21.15.6:0/285052157 wait complete. 2023-11-28T13:01:19.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.614+0000 7f8288eef700 1 Processor -- start 2023-11-28T13:01:19.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.614+0000 7f8288eef700 1 -- start start 2023-11-28T13:01:19.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.615+0000 7f8288eef700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82840b1630 0x7f82840b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.615+0000 7f8288eef700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8284159500 0x7f82841f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.615+0000 7f8288eef700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f828415dbf0 0x7f82841f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.615+0000 7f8288eef700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82840b3a30 con 0x7f828415dbf0 2023-11-28T13:01:19.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.615+0000 7f8288eef700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82840b3cd0 con 0x7f8284159500 2023-11-28T13:01:19.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.615+0000 7f8288eef700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82840b3fb0 con 0x7f82840b1630 2023-11-28T13:01:19.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.615+0000 7f8282d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f828415dbf0 0x7f82841f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.615+0000 7f8282d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f828415dbf0 0x7f82841f19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59596/0 (socket says 172.21.15.6:59596) 2023-11-28T13:01:19.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.615+0000 7f8282d9d700 1 -- 172.21.15.6:0/856292491 learned_addr learned my addr 172.21.15.6:0/856292491 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:19.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.615+0000 7f8281d9b700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8284159500 0x7f82841f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f828259c700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82840b1630 0x7f82840b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8282d9d700 1 -- 172.21.15.6:0/856292491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82840b1630 msgr2=0x7f82840b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8282d9d700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82840b1630 0x7f82840b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8282d9d700 1 -- 172.21.15.6:0/856292491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8284159500 msgr2=0x7f82841f14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8282d9d700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8284159500 0x7f82841f14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8282d9d700 1 -- 172.21.15.6:0/856292491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c00bd40 con 0x7f828415dbf0 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f828259c700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82840b1630 0x7f82840b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8282d9d700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f828415dbf0 0x7f82841f19f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f827c00d860 tx=0x7f827c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8272ffd700 1 -- 172.21.15.6:0/856292491 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c009810 con 0x7f828415dbf0 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8272ffd700 1 -- 172.21.15.6:0/856292491 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c009970 con 0x7f828415dbf0 2023-11-28T13:01:19.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8288eef700 1 -- 172.21.15.6:0/856292491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828415d110 con 0x7f828415dbf0 2023-11-28T13:01:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8272ffd700 1 -- 172.21.15.6:0/856292491 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c009d00 con 0x7f828415dbf0 2023-11-28T13:01:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.616+0000 7f8288eef700 1 -- 172.21.15.6:0/856292491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82840cf2d0 con 0x7f828415dbf0 2023-11-28T13:01:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.617+0000 7f8288eef700 1 -- 172.21.15.6:0/856292491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828415d3f0 con 0x7f828415dbf0 2023-11-28T13:01:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.617+0000 7f8272ffd700 1 -- 172.21.15.6:0/856292491 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f827c004040 con 0x7f828415dbf0 2023-11-28T13:01:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.618+0000 7f8272ffd700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82640682f0 0x7f826406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.618+0000 7f8272ffd700 1 -- 172.21.15.6:0/856292491 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f827c089790 con 0x7f828415dbf0 2023-11-28T13:01:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.618+0000 7f828259c700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82640682f0 0x7f826406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.618+0000 7f828259c700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82640682f0 0x7f826406a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8274006fd0 tx=0x7f8274008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:19.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.622+0000 7f8272ffd700 1 -- 172.21.15.6:0/856292491 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f827c055cc0 con 0x7f828415dbf0 2023-11-28T13:01:19.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.813+0000 7f8288eef700 1 -- 172.21.15.6:0/856292491 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82840739f0 con 0x7f828415dbf0 2023-11-28T13:01:19.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.814+0000 7f8272ffd700 1 -- 172.21.15.6:0/856292491 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f827c009e60 con 0x7f828415dbf0 2023-11-28T13:01:19.815 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.815+0000 7f8288eef700 1 -- 172.21.15.6:0/856292491 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82640682f0 msgr2=0x7f826406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.815+0000 7f8288eef700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82640682f0 0x7f826406a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8274006fd0 tx=0x7f8274008040 comp rx=0 tx=0).stop 2023-11-28T13:01:19.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.815+0000 7f8288eef700 1 -- 172.21.15.6:0/856292491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f828415dbf0 msgr2=0x7f82841f19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.815+0000 7f8288eef700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f828415dbf0 0x7f82841f19f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f827c00d860 tx=0x7f827c004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.816+0000 7f8288eef700 1 -- 172.21.15.6:0/856292491 shutdown_connections 2023-11-28T13:01:19.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.816+0000 7f8288eef700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f828415dbf0 0x7f82841f19f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.816+0000 7f8288eef700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82640682f0 0x7f826406a7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.816+0000 7f8288eef700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f82840b1630 0x7f82840b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.816+0000 7f8288eef700 1 --2- 172.21.15.6:0/856292491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8284159500 0x7f82841f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.816+0000 7f8288eef700 1 -- 172.21.15.6:0/856292491 >> 172.21.15.6:0/856292491 conn(0x7f82840cc630 msgr2=0x7f82840cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.816+0000 7f8288eef700 1 -- 172.21.15.6:0/856292491 shutdown_connections 2023-11-28T13:01:19.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.816+0000 7f8288eef700 1 -- 172.21.15.6:0/856292491 wait complete. 2023-11-28T13:01:19.829 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:19.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.955+0000 7f3ab0b6a700 1 Processor -- start 2023-11-28T13:01:19.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.956+0000 7f3ab0b6a700 1 -- start start 2023-11-28T13:01:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.956+0000 7f3ab0b6a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3aac0cd140 0x7f3aac06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3ab0b6a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac0ae730 con 0x7f3aac15a640 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3ab0b6a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac0c3e10 con 0x7f3aac06d140 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3ab0b6a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aac0cb210 con 0x7f3aac0cd140 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aa9d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3aac0cd140 0x7f3aac06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aaa59c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f3aac15a640 0x7f3aac0c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46922/0 (socket says 172.21.15.6:46922) 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aaa59c700 1 -- 172.21.15.6:0/219398679 learned_addr learned my addr 172.21.15.6:0/219398679 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aa9d9b700 1 -- 172.21.15.6:0/219398679 >> v1:172.21.15.61:6789/0 conn(0x7f3aac06d140 legacy=0x7f3aac0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aa9d9b700 1 -- 172.21.15.6:0/219398679 >> v1:172.21.15.6:6789/0 conn(0x7f3aac15a640 legacy=0x7f3aac0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aa959a700 1 -- 172.21.15.6:0/219398679 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651159464 0 0) 0x7f3aac0ae730 con 0x7f3aac15a640 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aa9d9b700 1 -- 172.21.15.6:0/219398679 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac06e9e0 con 0x7f3aac0cd140 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aa9d9b700 1 --2- 172.21.15.6:0/219398679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac0cd140 0x7f3aac06b1f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009840 tx=0x7f3a9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3595d7e92af8739 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aa959a700 1 -- 172.21.15.6:0/219398679 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c011830 con 0x7f3aac0cd140 2023-11-28T13:01:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aa959a700 1 -- 172.21.15.6:0/219398679 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c011990 con 0x7f3aac0cd140 2023-11-28T13:01:19.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.957+0000 7f3aa959a700 1 -- 172.21.15.6:0/219398679 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c011d20 con 0x7f3aac0cd140 2023-11-28T13:01:19.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.958+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/219398679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac0cd140 msgr2=0x7f3aac06b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.958+0000 7f3ab0b6a700 1 --2- 172.21.15.6:0/219398679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac0cd140 0x7f3aac06b1f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009840 tx=0x7f3a9c009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:19.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.958+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/219398679 shutdown_connections 2023-11-28T13:01:19.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.958+0000 7f3ab0b6a700 1 --2- 172.21.15.6:0/219398679 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac0cd140 0x7f3aac06b1f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.958+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/219398679 >> 172.21.15.6:0/219398679 conn(0x7f3aac0cc630 msgr2=0x7f3aac0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:19.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.958+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/219398679 shutdown_connections 2023-11-28T13:01:19.961 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.958+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/219398679 wait complete. 2023-11-28T13:01:19.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.959+0000 7f3ab0b6a700 1 Processor -- start 2023-11-28T13:01:19.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.959+0000 7f3ab0b6a700 1 -- start start 2023-11-28T13:01:19.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3ab0b6a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3aac06d140 0x7f3aac14c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3ab0b6a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3aac0cd140 0x7f3aac153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3ab0b6a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac15a640 0x7f3aac0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3ab0b6a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac1530f0 con 0x7f3aac15a640 2023-11-28T13:01:19.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3ab0b6a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac150de0 con 0x7f3aac06d140 2023-11-28T13:01:19.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3ab0b6a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac1510c0 con 0x7f3aac0cd140 2023-11-28T13:01:19.964 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3aaad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac15a640 0x7f3aac0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3aaad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac15a640 0x7f3aac0cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59612/0 (socket says 172.21.15.6:59612) 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3aaad9d700 1 -- 172.21.15.6:0/1577352022 learned_addr learned my addr 172.21.15.6:0/1577352022 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3aaa59c700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3aac06d140 0x7f3aac14c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.960+0000 7f3aa9d9b700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3aac0cd140 0x7f3aac153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3aaad9d700 1 -- 172.21.15.6:0/1577352022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3aac0cd140 msgr2=0x7f3aac153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3aaad9d700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3aac0cd140 0x7f3aac153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3aaad9d700 1 -- 172.21.15.6:0/1577352022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3aac06d140 msgr2=0x7f3aac14c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3aaad9d700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3aac06d140 0x7f3aac14c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3aaad9d700 1 -- 172.21.15.6:0/1577352022 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0094a0 con 0x7f3aac15a640 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3aaad9d700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac15a640 0x7f3aac0cf0b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400cc50 tx=0x7f3aa400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:19.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3a9affd700 1 -- 172.21.15.6:0/1577352022 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4003c00 con 0x7f3aac15a640 2023-11-28T13:01:19.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3a9affd700 1 -- 172.21.15.6:0/1577352022 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa4003d60 con 0x7f3aac15a640 2023-11-28T13:01:19.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/1577352022 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac0aec60 con 0x7f3aac15a640 2023-11-28T13:01:19.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3a9affd700 1 -- 172.21.15.6:0/1577352022 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa400aab0 con 0x7f3aac15a640 2023-11-28T13:01:19.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.961+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/1577352022 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac0ae980 con 0x7f3aac15a640 2023-11-28T13:01:19.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.962+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/1577352022 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac0aef40 con 0x7f3aac15a640 2023-11-28T13:01:19.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.962+0000 7f3a9affd700 1 -- 172.21.15.6:0/1577352022 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3aa4010050 con 0x7f3aac15a640 2023-11-28T13:01:19.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.963+0000 7f3a9affd700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a8c0682f0 0x7f3a8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:19.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.963+0000 7f3a9affd700 1 -- 172.21.15.6:0/1577352022 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3aa4085df0 con 0x7f3aac15a640 2023-11-28T13:01:19.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.963+0000 7f3aaa59c700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a8c0682f0 0x7f3a8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:19.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.963+0000 7f3aaa59c700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a8c0682f0 0x7f3a8c06a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0007c40 tx=0x7f3aa0007b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:19.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:19.967+0000 7f3a9affd700 1 -- 172.21.15.6:0/1577352022 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3aa4052320 con 0x7f3aac15a640 2023-11-28T13:01:20.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.164+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/1577352022 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3aac1515e0 con 0x7f3aac15a640 2023-11-28T13:01:20.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.164+0000 7f3a9affd700 1 -- 172.21.15.6:0/1577352022 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3aa4055940 con 0x7f3aac15a640 2023-11-28T13:01:20.166 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:20.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.166+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/1577352022 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a8c0682f0 msgr2=0x7f3a8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.166+0000 7f3ab0b6a700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a8c0682f0 0x7f3a8c06a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0007c40 tx=0x7f3aa0007b90 comp rx=0 tx=0).stop 2023-11-28T13:01:20.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.166+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/1577352022 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac15a640 msgr2=0x7f3aac0cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.166+0000 7f3ab0b6a700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac15a640 0x7f3aac0cf0b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400cc50 tx=0x7f3aa400cf60 comp rx=0 tx=0).stop 2023-11-28T13:01:20.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.167+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/1577352022 shutdown_connections 2023-11-28T13:01:20.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.167+0000 7f3ab0b6a700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3aac15a640 0x7f3aac0cf0b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.167+0000 7f3ab0b6a700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3aac06d140 0x7f3aac14c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.167+0000 7f3ab0b6a700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a8c0682f0 0x7f3a8c06a7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.167+0000 7f3ab0b6a700 1 --2- 172.21.15.6:0/1577352022 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3aac0cd140 0x7f3aac153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.167+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/1577352022 >> 172.21.15.6:0/1577352022 conn(0x7f3aac0cc630 msgr2=0x7f3aac0c36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:20.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.167+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/1577352022 shutdown_connections 2023-11-28T13:01:20.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.167+0000 7f3ab0b6a700 1 -- 172.21.15.6:0/1577352022 wait complete. 2023-11-28T13:01:20.180 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:20.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.307+0000 7fdfa67f0700 1 Processor -- start 2023-11-28T13:01:20.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.307+0000 7fdfa67f0700 1 -- start start 2023-11-28T13:01:20.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.307+0000 7fdfa67f0700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdfa015a640 0x7fdfa00c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.307+0000 7fdfa67f0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfa006b900 0x7fdfa00cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.307+0000 7fdfa67f0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00ae730 con 0x7fdfa00cd140 2023-11-28T13:01:20.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.307+0000 7fdfa67f0700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa00cb210 con 0x7fdfa015a640 2023-11-28T13:01:20.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.307+0000 7fdfa67f0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa006e9e0 con 0x7fdfa006b900 2023-11-28T13:01:20.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9ffff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdfa015a640 0x7fdfa00c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9ffff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdfa015a640 0x7fdfa00c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59616/0 (socket says 172.21.15.6:59616) 2023-11-28T13:01:20.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9ffff700 1 -- 172.21.15.6:0/793595348 learned_addr learned my addr 172.21.15.6:0/793595348 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:20.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdfa4d8d700 1 --2- 172.21.15.6:0/793595348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfa006b900 0x7fdfa00cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9ffff700 1 -- 172.21.15.6:0/793595348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfa006b900 msgr2=0x7fdfa00cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:20.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9ffff700 1 --2- 172.21.15.6:0/793595348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfa006b900 0x7fdfa00cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9ffff700 1 -- 172.21.15.6:0/793595348 >> v1:172.21.15.61:6789/0 conn(0x7fdfa00cd140 legacy=0x7fdfa006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:20.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9ffff700 1 -- 172.21.15.6:0/793595348 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa00ae980 con 0x7fdfa015a640 2023-11-28T13:01:20.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9ffff700 1 --2- 172.21.15.6:0/793595348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa015a640 0x7fdfa00c3700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdf900098f0 tx=0x7fdf90009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e592ea668a91d99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:20.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9effd700 1 -- 172.21.15.6:0/793595348 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90004230 con 0x7fdfa015a640 2023-11-28T13:01:20.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9effd700 1 -- 172.21.15.6:0/793595348 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf90004390 con 0x7fdfa015a640 2023-11-28T13:01:20.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.308+0000 7fdf9effd700 1 -- 172.21.15.6:0/793595348 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf900036a0 con 0x7fdfa015a640 2023-11-28T13:01:20.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.309+0000 7fdfa67f0700 1 -- 172.21.15.6:0/793595348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa015a640 msgr2=0x7fdfa00c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.309+0000 7fdfa67f0700 1 --2- 172.21.15.6:0/793595348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa015a640 0x7fdfa00c3700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdf900098f0 tx=0x7fdf90009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:20.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.309+0000 7fdfa67f0700 1 -- 172.21.15.6:0/793595348 shutdown_connections 2023-11-28T13:01:20.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.309+0000 7fdfa67f0700 1 --2- 172.21.15.6:0/793595348 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfa006b900 0x7fdfa00cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.309+0000 7fdfa67f0700 1 --2- 172.21.15.6:0/793595348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa015a640 0x7fdfa00c3700 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.309+0000 7fdfa67f0700 1 -- 172.21.15.6:0/793595348 >> 172.21.15.6:0/793595348 conn(0x7fdfa00cc630 msgr2=0x7fdfa00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:20.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.309+0000 7fdfa67f0700 1 -- 172.21.15.6:0/793595348 shutdown_connections 2023-11-28T13:01:20.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.309+0000 7fdfa67f0700 1 -- 172.21.15.6:0/793595348 wait complete. 2023-11-28T13:01:20.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.310+0000 7fdfa67f0700 1 Processor -- start 2023-11-28T13:01:20.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.310+0000 7fdfa67f0700 1 -- start start 2023-11-28T13:01:20.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdfa67f0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa006b900 0x7fdfa015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdfa67f0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdfa00cd140 0x7fdfa00cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdfa67f0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfa015a640 0x7fdfa01571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdfa67f0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00cf1d0 con 0x7fdfa006b900 2023-11-28T13:01:20.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdfa67f0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa015b060 con 0x7fdfa00cd140 2023-11-28T13:01:20.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdfa67f0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa015b340 con 0x7fdfa015a640 2023-11-28T13:01:20.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdf9ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa006b900 0x7fdfa015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdf9ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa006b900 0x7fdfa015aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59622/0 (socket says 172.21.15.6:59622) 2023-11-28T13:01:20.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdf9ffff700 1 -- 172.21.15.6:0/1102245099 learned_addr learned my addr 172.21.15.6:0/1102245099 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdf9f7fe700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdfa00cd140 0x7fdfa00cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdfa4d8d700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfa015a640 0x7fdfa01571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdf9ffff700 1 -- 172.21.15.6:0/1102245099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfa015a640 msgr2=0x7fdfa01571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.311+0000 7fdf9ffff700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfa015a640 0x7fdfa01571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.312+0000 7fdf9ffff700 1 -- 172.21.15.6:0/1102245099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdfa00cd140 msgr2=0x7fdfa00cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.312+0000 7fdf9ffff700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdfa00cd140 0x7fdfa00cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.312+0000 7fdf9ffff700 1 -- 172.21.15.6:0/1102245099 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf90009550 con 0x7fdfa006b900 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.312+0000 7fdf9f7fe700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdfa00cd140 0x7fdfa00cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.312+0000 7fdf9ffff700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa006b900 0x7fdfa015aa10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0066980 tx=0x7fdf90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.312+0000 7fdf9cff9700 1 -- 172.21.15.6:0/1102245099 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90003e70 con 0x7fdfa006b900 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.312+0000 7fdf9cff9700 1 -- 172.21.15.6:0/1102245099 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9001f070 con 0x7fdfa006b900 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.312+0000 7fdfa67f0700 1 -- 172.21.15.6:0/1102245099 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa00aef40 con 0x7fdfa006b900 2023-11-28T13:01:20.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.312+0000 7fdf9cff9700 1 -- 172.21.15.6:0/1102245099 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9000b8c0 con 0x7fdfa006b900 2023-11-28T13:01:20.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.312+0000 7fdfa67f0700 1 -- 172.21.15.6:0/1102245099 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa00aec60 con 0x7fdfa006b900 2023-11-28T13:01:20.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.313+0000 7fdfa67f0700 1 -- 172.21.15.6:0/1102245099 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa00af220 con 0x7fdfa006b900 2023-11-28T13:01:20.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.313+0000 7fdf9cff9700 1 -- 172.21.15.6:0/1102245099 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdf9002a020 con 0x7fdfa006b900 2023-11-28T13:01:20.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.314+0000 7fdf9cff9700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf8c0682f0 0x7fdf8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.314+0000 7fdf9cff9700 1 -- 172.21.15.6:0/1102245099 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdf900877d0 con 0x7fdfa006b900 2023-11-28T13:01:20.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.314+0000 7fdf9f7fe700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf8c0682f0 0x7fdf8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.314+0000 7fdf9f7fe700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf8c0682f0 0x7fdf8c06a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0157ba0 tx=0x7fdf94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:20.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.318+0000 7fdf9cff9700 1 -- 172.21.15.6:0/1102245099 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdf90053d00 con 0x7fdfa006b900 2023-11-28T13:01:20.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.512+0000 7fdfa67f0700 1 -- 172.21.15.6:0/1102245099 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdfa00af220 con 0x7fdfa006b900 2023-11-28T13:01:20.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.512+0000 7fdf9cff9700 1 -- 172.21.15.6:0/1102245099 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdf90057320 con 0x7fdfa006b900 2023-11-28T13:01:20.514 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 -- 172.21.15.6:0/1102245099 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf8c0682f0 msgr2=0x7fdf8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf8c0682f0 0x7fdf8c06a7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0157ba0 tx=0x7fdf94009040 comp rx=0 tx=0).stop 2023-11-28T13:01:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 -- 172.21.15.6:0/1102245099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa006b900 msgr2=0x7fdfa015aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa006b900 0x7fdfa015aa10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0066980 tx=0x7fdf90004750 comp rx=0 tx=0).stop 2023-11-28T13:01:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 -- 172.21.15.6:0/1102245099 shutdown_connections 2023-11-28T13:01:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf8c0682f0 0x7fdf8c06a7a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfa015a640 0x7fdfa01571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfa006b900 0x7fdfa015aa10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 --2- 172.21.15.6:0/1102245099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdfa00cd140 0x7fdfa00cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 -- 172.21.15.6:0/1102245099 >> 172.21.15.6:0/1102245099 conn(0x7fdfa00cc630 msgr2=0x7fdfa00b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:20.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 -- 172.21.15.6:0/1102245099 shutdown_connections 2023-11-28T13:01:20.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.514+0000 7fdfa67f0700 1 -- 172.21.15.6:0/1102245099 wait complete. 2023-11-28T13:01:20.528 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:20.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.652+0000 7f7ea1cca700 1 Processor -- start 2023-11-28T13:01:20.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.652+0000 7f7ea1cca700 1 -- start start 2023-11-28T13:01:20.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7ea1cca700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7e9c15a650 0x7f7e9c0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7ea1cca700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 0x7f7e9c0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7ea1cca700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c0ca990 con 0x7f7e9c15aa20 2023-11-28T13:01:20.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7ea1cca700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c0caf50 con 0x7f7e9c15a650 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7ea1cca700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0aef10 con 0x7f7e9c0b5500 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7e9b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7e9c15a650 0x7f7e9c0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7e9affd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f7e9c15aa20 0x7f7e9c0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46952/0 (socket says 172.21.15.6:46952) 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7e9affd700 1 -- 172.21.15.6:0/2415992660 learned_addr learned my addr 172.21.15.6:0/2415992660 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7e9bfff700 1 --2- 172.21.15.6:0/2415992660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 0x7f7e9c0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7e9b7fe700 1 -- 172.21.15.6:0/2415992660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 msgr2=0x7f7e9c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7e9b7fe700 1 --2- 172.21.15.6:0/2415992660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 0x7f7e9c0cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7e9b7fe700 1 -- 172.21.15.6:0/2415992660 >> v1:172.21.15.6:6789/0 conn(0x7f7e9c15aa20 legacy=0x7f7e9c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7e9a7fc700 1 -- 172.21.15.6:0/2415992660 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132640860 0 0) 0x7f7e9c0ca990 con 0x7f7e9c15aa20 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.653+0000 7f7e9b7fe700 1 -- 172.21.15.6:0/2415992660 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0af1f0 con 0x7f7e9c15a650 2023-11-28T13:01:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7e9bfff700 1 --2- 172.21.15.6:0/2415992660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 0x7f7e9c0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7e9b7fe700 1 --2- 172.21.15.6:0/2415992660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15a650 0x7f7e9c0b2950 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009840 tx=0x7f7e84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0525deabde17b0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7e9a7fc700 1 -- 172.21.15.6:0/2415992660 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84011830 con 0x7f7e9c15a650 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7e9a7fc700 1 -- 172.21.15.6:0/2415992660 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e84011990 con 0x7f7e9c15a650 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7e9a7fc700 1 -- 172.21.15.6:0/2415992660 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84011d20 con 0x7f7e9c15a650 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2415992660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15a650 msgr2=0x7f7e9c0b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7ea1cca700 1 --2- 172.21.15.6:0/2415992660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15a650 0x7f7e9c0b2950 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009840 tx=0x7f7e84009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2415992660 shutdown_connections 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7ea1cca700 1 --2- 172.21.15.6:0/2415992660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 0x7f7e9c0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7ea1cca700 1 --2- 172.21.15.6:0/2415992660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15a650 0x7f7e9c0b2950 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2415992660 >> 172.21.15.6:0/2415992660 conn(0x7f7e9c0b1650 msgr2=0x7f7e9c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2415992660 shutdown_connections 2023-11-28T13:01:20.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.654+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2415992660 wait complete. 2023-11-28T13:01:20.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.656+0000 7f7ea1cca700 1 Processor -- start 2023-11-28T13:01:20.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.656+0000 7f7ea1cca700 1 -- start start 2023-11-28T13:01:20.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7ea1cca700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 0x7f7e9c15b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7ea1cca700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e9c15a650 0x7f7e9c151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7ea1cca700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15aa20 0x7f7e9c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7ea1cca700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1efd50 con 0x7f7e9c15aa20 2023-11-28T13:01:20.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7ea1cca700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0b2950 con 0x7f7e9c15a650 2023-11-28T13:01:20.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7ea1cca700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c0b2b60 con 0x7f7e9c0b5500 2023-11-28T13:01:20.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15aa20 0x7f7e9c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15aa20 0x7f7e9c0b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59640/0 (socket says 172.21.15.6:59640) 2023-11-28T13:01:20.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9bfff700 1 -- 172.21.15.6:0/2127937498 learned_addr learned my addr 172.21.15.6:0/2127937498 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:20.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9affd700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e9c15a650 0x7f7e9c151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9b7fe700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 0x7f7e9c15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9bfff700 1 -- 172.21.15.6:0/2127937498 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 msgr2=0x7f7e9c15b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9bfff700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 0x7f7e9c15b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9bfff700 1 -- 172.21.15.6:0/2127937498 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e9c15a650 msgr2=0x7f7e9c151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9bfff700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e9c15a650 0x7f7e9c151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9bfff700 1 -- 172.21.15.6:0/2127937498 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e840094a0 con 0x7f7e9c15aa20 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e9bfff700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15aa20 0x7f7e9c0b3c00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400cef0 tx=0x7f7e9400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e93fff700 1 -- 172.21.15.6:0/2127937498 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400ad00 con 0x7f7e9c15aa20 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e93fff700 1 -- 172.21.15.6:0/2127937498 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9400ae60 con 0x7f7e9c15aa20 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2127937498 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c0cc9f0 con 0x7f7e9c15aa20 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.657+0000 7f7e93fff700 1 -- 172.21.15.6:0/2127937498 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94015680 con 0x7f7e9c15aa20 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.658+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2127937498 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c0cabe0 con 0x7f7e9c15aa20 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.658+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2127937498 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c0ccb80 con 0x7f7e9c15aa20 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.659+0000 7f7e93fff700 1 -- 172.21.15.6:0/2127937498 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7e9401c030 con 0x7f7e9c15aa20 2023-11-28T13:01:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.659+0000 7f7e93fff700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e7c0682f0 0x7f7e7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.659+0000 7f7e93fff700 1 -- 172.21.15.6:0/2127937498 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7e94086310 con 0x7f7e9c15aa20 2023-11-28T13:01:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.659+0000 7f7e9b7fe700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e7c0682f0 0x7f7e7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.660+0000 7f7e9b7fe700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e7c0682f0 0x7f7e7c06a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e840090b0 tx=0x7f7e84008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.664+0000 7f7e93fff700 1 -- 172.21.15.6:0/2127937498 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7e94052840 con 0x7f7e9c15aa20 2023-11-28T13:01:20.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.871+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2127937498 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e9c0675a0 con 0x7f7e9c15aa20 2023-11-28T13:01:20.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.871+0000 7f7e93fff700 1 -- 172.21.15.6:0/2127937498 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7e94055e60 con 0x7f7e9c15aa20 2023-11-28T13:01:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2127937498 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e7c0682f0 msgr2=0x7f7e7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e7c0682f0 0x7f7e7c06a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7e840090b0 tx=0x7f7e84008030 comp rx=0 tx=0).stop 2023-11-28T13:01:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2127937498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15aa20 msgr2=0x7f7e9c0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15aa20 0x7f7e9c0b3c00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400cef0 tx=0x7f7e9400cf20 comp rx=0 tx=0).stop 2023-11-28T13:01:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2127937498 shutdown_connections 2023-11-28T13:01:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7e9c15aa20 0x7f7e9c0b3c00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7e7c0682f0 0x7f7e7c06a7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7e9c0b5500 0x7f7e9c15b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 --2- 172.21.15.6:0/2127937498 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e9c15a650 0x7f7e9c151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2127937498 >> 172.21.15.6:0/2127937498 conn(0x7f7e9c0b1650 msgr2=0x7f7e9c06ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2127937498 shutdown_connections 2023-11-28T13:01:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:20.873+0000 7f7ea1cca700 1 -- 172.21.15.6:0/2127937498 wait complete. 2023-11-28T13:01:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:01:20.887 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2023-11-28T13:01:20.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:20.887 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-28T13:01:20.923 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:20.924 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:01:20.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:20.924 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T13:01:20.981 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:01:20.982 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:21.003 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:01:21.004 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:21.064 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:21.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.188+0000 7fb6c53e6700 1 Processor -- start 2023-11-28T13:01:21.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.188+0000 7fb6c53e6700 1 -- start start 2023-11-28T13:01:21.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.189+0000 7fb6c53e6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb6c006d9d0 0x7fb6c015b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.189+0000 7fb6c53e6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb6c006bc30 0x7fb6c015e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.189+0000 7fb6c53e6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6c006dda0 0x7fb6c0150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.189+0000 7fb6c53e6700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c015ee90 con 0x7fb6c006d9d0 2023-11-28T13:01:21.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.189+0000 7fb6c53e6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c006c3a0 con 0x7fb6c006bc30 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.189+0000 7fb6c53e6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c006c0c0 con 0x7fb6c006dda0 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.189+0000 7fb6beffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb6c006d9d0 0x7fb6c015b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.189+0000 7fb6beffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb6c006d9d0 0x7fb6c015b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59644/0 (socket says 172.21.15.6:59644) 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.189+0000 7fb6beffd700 1 -- 172.21.15.6:0/3701097055 learned_addr learned my addr 172.21.15.6:0/3701097055 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6be7fc700 1 --2- 172.21.15.6:0/3701097055 >> v2:172.21.15.61:3300/0 conn(0x7fb6c006bc30 0x7fb6c015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6bf7fe700 1 --2- 172.21.15.6:0/3701097055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6c006dda0 0x7fb6c0150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6beffd700 1 -- 172.21.15.6:0/3701097055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6c006dda0 msgr2=0x7fb6c0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6beffd700 1 --2- 172.21.15.6:0/3701097055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6c006dda0 0x7fb6c0150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6beffd700 1 -- 172.21.15.6:0/3701097055 >> v2:172.21.15.61:3300/0 conn(0x7fb6c006bc30 msgr2=0x7fb6c015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6beffd700 1 --2- 172.21.15.6:0/3701097055 >> v2:172.21.15.61:3300/0 conn(0x7fb6c006bc30 0x7fb6c015e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6beffd700 1 -- 172.21.15.6:0/3701097055 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c015ab60 con 0x7fb6c006d9d0 2023-11-28T13:01:21.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6beffd700 1 --2- 172.21.15.6:0/3701097055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006d9d0 0x7fb6c015b640 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00b5420 tx=0x7fb6b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b930791455fb4893 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6bdffb700 1 -- 172.21.15.6:0/3701097055 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000b870 con 0x7fb6c006d9d0 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6bdffb700 1 -- 172.21.15.6:0/3701097055 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b000b9d0 con 0x7fb6c006d9d0 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6bdffb700 1 -- 172.21.15.6:0/3701097055 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000bd60 con 0x7fb6c006d9d0 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6c53e6700 1 -- 172.21.15.6:0/3701097055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006d9d0 msgr2=0x7fb6c015b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6c53e6700 1 --2- 172.21.15.6:0/3701097055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006d9d0 0x7fb6c015b640 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00b5420 tx=0x7fb6b0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6c53e6700 1 -- 172.21.15.6:0/3701097055 shutdown_connections 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6c53e6700 1 --2- 172.21.15.6:0/3701097055 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6c006dda0 0x7fb6c0150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6c53e6700 1 --2- 172.21.15.6:0/3701097055 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006d9d0 0x7fb6c015b640 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6c53e6700 1 --2- 172.21.15.6:0/3701097055 >> v2:172.21.15.61:3300/0 conn(0x7fb6c006bc30 0x7fb6c015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.190+0000 7fb6c53e6700 1 -- 172.21.15.6:0/3701097055 >> 172.21.15.6:0/3701097055 conn(0x7fb6c00b0dd0 msgr2=0x7fb6c00b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.191+0000 7fb6c53e6700 1 -- 172.21.15.6:0/3701097055 shutdown_connections 2023-11-28T13:01:21.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.191+0000 7fb6c53e6700 1 -- 172.21.15.6:0/3701097055 wait complete. 2023-11-28T13:01:21.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.192+0000 7fb6c53e6700 1 Processor -- start 2023-11-28T13:01:21.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.192+0000 7fb6c53e6700 1 -- start start 2023-11-28T13:01:21.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6c53e6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006bc30 0x7fb6c01485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6c53e6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6c006d9d0 0x7fb6c00c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6c53e6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6c006dda0 0x7fb6c00c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6c53e6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c01f1d80 con 0x7fb6c006bc30 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6c53e6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00b2110 con 0x7fb6c006dda0 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6c53e6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c01f1100 con 0x7fb6c006d9d0 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6beffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006bc30 0x7fb6c01485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6beffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006bc30 0x7fb6c01485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59650/0 (socket says 172.21.15.6:59650) 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6beffd700 1 -- 172.21.15.6:0/825243148 learned_addr learned my addr 172.21.15.6:0/825243148 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6beffd700 1 -- 172.21.15.6:0/825243148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6c006d9d0 msgr2=0x7fb6c00c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6be7fc700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6c006d9d0 0x7fb6c00c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6bf7fe700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6c006dda0 0x7fb6c00c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6beffd700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6c006d9d0 0x7fb6c00c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6beffd700 1 -- 172.21.15.6:0/825243148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6c006dda0 msgr2=0x7fb6c00c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6beffd700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6c006dda0 0x7fb6c00c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6beffd700 1 -- 172.21.15.6:0/825243148 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b0009550 con 0x7fb6c006bc30 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6beffd700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006bc30 0x7fb6c01485e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6c006d160 tx=0x7fb6b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6b77fe700 1 -- 172.21.15.6:0/825243148 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0004310 con 0x7fb6c006bc30 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6b77fe700 1 -- 172.21.15.6:0/825243148 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b001f070 con 0x7fb6c006bc30 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6c53e6700 1 -- 172.21.15.6:0/825243148 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c015b120 con 0x7fb6c006bc30 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6b77fe700 1 -- 172.21.15.6:0/825243148 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b00036a0 con 0x7fb6c006bc30 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.193+0000 7fb6c53e6700 1 -- 172.21.15.6:0/825243148 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c015ae40 con 0x7fb6c006bc30 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.194+0000 7fb6c53e6700 1 -- 172.21.15.6:0/825243148 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c006fac0 con 0x7fb6c006bc30 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.195+0000 7fb6b77fe700 1 -- 172.21.15.6:0/825243148 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb6b002a020 con 0x7fb6c006bc30 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.195+0000 7fb6b77fe700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6a00682f0 0x7fb6a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.195+0000 7fb6b77fe700 1 -- 172.21.15.6:0/825243148 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb6b0087600 con 0x7fb6c006bc30 2023-11-28T13:01:21.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.195+0000 7fb6be7fc700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6a00682f0 0x7fb6a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.196+0000 7fb6be7fc700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6a00682f0 0x7fb6a006a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6c015c590 tx=0x7fb6a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.200+0000 7fb6b77fe700 1 -- 172.21.15.6:0/825243148 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb6b0053b30 con 0x7fb6c006bc30 2023-11-28T13:01:21.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.391+0000 7fb6c53e6700 1 -- 172.21.15.6:0/825243148 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6c01f1ec0 con 0x7fb6c006bc30 2023-11-28T13:01:21.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.392+0000 7fb6b77fe700 1 -- 172.21.15.6:0/825243148 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19690 (secure 0 0 0) 0x7fb6b0057150 con 0x7fb6c006bc30 2023-11-28T13:01:21.394 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:21.395 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:10.053012+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:21.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 -- 172.21.15.6:0/825243148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6a00682f0 msgr2=0x7fb6a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6a00682f0 0x7fb6a006a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6c015c590 tx=0x7fb6a8008040 comp rx=0 tx=0).stop 2023-11-28T13:01:21.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 -- 172.21.15.6:0/825243148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006bc30 msgr2=0x7fb6c01485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006bc30 0x7fb6c01485e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb6c006d160 tx=0x7fb6b0004750 comp rx=0 tx=0).stop 2023-11-28T13:01:21.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 -- 172.21.15.6:0/825243148 shutdown_connections 2023-11-28T13:01:21.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6c006dda0 0x7fb6c00c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c006bc30 0x7fb6c01485e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6a00682f0 0x7fb6a006a7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 --2- 172.21.15.6:0/825243148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6c006d9d0 0x7fb6c00c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 -- 172.21.15.6:0/825243148 >> 172.21.15.6:0/825243148 conn(0x7fb6c00b0dd0 msgr2=0x7fb6c00715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:21.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 -- 172.21.15.6:0/825243148 shutdown_connections 2023-11-28T13:01:21.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.394+0000 7fb6c53e6700 1 -- 172.21.15.6:0/825243148 wait complete. 2023-11-28T13:01:21.409 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:21.410 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:21.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.539+0000 7fd1460c9700 1 Processor -- start 2023-11-28T13:01:21.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.539+0000 7fd1460c9700 1 -- start start 2023-11-28T13:01:21.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd1460c9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140159500 0x7fd1400cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd1460c9700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14015e040 con 0x7fd1400b1d40 2023-11-28T13:01:21.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd1460c9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1400ceff0 con 0x7fd14015dbf0 2023-11-28T13:01:21.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd1460c9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400cf2d0 con 0x7fd140159500 2023-11-28T13:01:21.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13ffff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd1400b1d40 0x7fd14006bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:46982/0 (socket says 172.21.15.6:46982) 2023-11-28T13:01:21.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13ffff700 1 -- 172.21.15.6:0/1494196027 learned_addr learned my addr 172.21.15.6:0/1494196027 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:21.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13f7fe700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140159500 0x7fd1400cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249652544 0 0) 0x7fd14015e040 con 0x7fd1400b1d40 2023-11-28T13:01:21.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1280035b0 con 0x7fd1400b1d40 2023-11-28T13:01:21.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1956845193 0 0) 0x7fd1280035b0 con 0x7fd1400b1d40 2023-11-28T13:01:21.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140159500 msgr2=0x7fd1400cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140159500 0x7fd1400cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 >> v1:172.21.15.61:6789/0 conn(0x7fd14015dbf0 legacy=0x7fd1400b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14015d110 con 0x7fd1400b1d40 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fd1380019f0 con 0x7fd1400b1d40 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 >> v1:172.21.15.6:6789/0 conn(0x7fd1400b1d40 legacy=0x7fd14006bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd128003a30 0x7fd128003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.540+0000 7fd13e7fc700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140159500 0x7fd128004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13e7fc700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128004d90 0x7fd128005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1400cf2d0 con 0x7fd128004d90 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12800ba10 con 0x7fd128003a30 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12800bba0 con 0x7fd140159500 2023-11-28T13:01:21.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13ffff700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128004d90 0x7fd128005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13f7fe700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd128003a30 0x7fd128003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13effd700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140159500 0x7fd128004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13ffff700 1 -- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140159500 msgr2=0x7fd128004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13ffff700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140159500 0x7fd128004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13ffff700 1 -- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd128003a30 msgr2=0x7fd128003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13ffff700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd128003a30 0x7fd128003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13ffff700 1 -- 172.21.15.6:0/1494196027 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1280035b0 con 0x7fd128004d90 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13ffff700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128004d90 0x7fd128005200 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd13800a830 tx=0x7fd128005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5430d7bc54852ef2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138012600 con 0x7fd128004d90 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1380127a0 con 0x7fd128004d90 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.541+0000 7fd13e7fc700 1 -- 172.21.15.6:0/1494196027 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138012b50 con 0x7fd128004d90 2023-11-28T13:01:21.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.542+0000 7fd1460c9700 1 -- 172.21.15.6:0/1494196027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128004d90 msgr2=0x7fd128005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.542+0000 7fd1460c9700 1 --2- 172.21.15.6:0/1494196027 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd128004d90 0x7fd128005200 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd13800a830 tx=0x7fd128005880 comp rx=0 tx=0).stop 2023-11-28T13:01:21.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.542+0000 7fd13ffff700 1 -- 172.21.15.6:0/1494196027 reap_dead start 2023-11-28T13:01:21.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.542+0000 7fd1460c9700 1 -- 172.21.15.6:0/1494196027 shutdown_connections 2023-11-28T13:01:21.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.542+0000 7fd1460c9700 1 -- 172.21.15.6:0/1494196027 >> 172.21.15.6:0/1494196027 conn(0x7fd1400cc630 msgr2=0x7fd1400cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:21.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.542+0000 7fd1460c9700 1 -- 172.21.15.6:0/1494196027 shutdown_connections 2023-11-28T13:01:21.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.542+0000 7fd1460c9700 1 -- 172.21.15.6:0/1494196027 wait complete. 2023-11-28T13:01:21.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.543+0000 7fd1460c9700 1 Processor -- start 2023-11-28T13:01:21.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.543+0000 7fd1460c9700 1 -- start start 2023-11-28T13:01:21.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd1460c9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1401f3280 0x7fd1401589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd1460c9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd140157200 0x7fd140155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd1460c9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140146090 0x7fd140146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd1460c9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd140146a40 con 0x7fd140157200 2023-11-28T13:01:21.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd1460c9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd140155d70 con 0x7fd1401f3280 2023-11-28T13:01:21.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd1460c9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd140157950 con 0x7fd140146090 2023-11-28T13:01:21.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd140157200 0x7fd140155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd140157200 0x7fd140155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59672/0 (socket says 172.21.15.6:59672) 2023-11-28T13:01:21.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13effd700 1 -- 172.21.15.6:0/1772530982 learned_addr learned my addr 172.21.15.6:0/1772530982 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:21.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13f7fe700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1401f3280 0x7fd1401589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13ffff700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140146090 0x7fd140146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13effd700 1 -- 172.21.15.6:0/1772530982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140146090 msgr2=0x7fd140146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13effd700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140146090 0x7fd140146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13effd700 1 -- 172.21.15.6:0/1772530982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1401f3280 msgr2=0x7fd1401589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13effd700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1401f3280 0x7fd1401589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13effd700 1 -- 172.21.15.6:0/1772530982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14015d6d0 con 0x7fd140157200 2023-11-28T13:01:21.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.544+0000 7fd13effd700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd140157200 0x7fd140155830 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd13000f770 tx=0x7fd13000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.545+0000 7fd127fff700 1 -- 172.21.15.6:0/1772530982 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1300052e0 con 0x7fd140157200 2023-11-28T13:01:21.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.545+0000 7fd127fff700 1 -- 172.21.15.6:0/1772530982 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd130005460 con 0x7fd140157200 2023-11-28T13:01:21.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.545+0000 7fd127fff700 1 -- 172.21.15.6:0/1772530982 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1300038c0 con 0x7fd140157200 2023-11-28T13:01:21.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.545+0000 7fd1460c9700 1 -- 172.21.15.6:0/1772530982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14015d3f0 con 0x7fd140157200 2023-11-28T13:01:21.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.545+0000 7fd1460c9700 1 -- 172.21.15.6:0/1772530982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1401598d0 con 0x7fd140157200 2023-11-28T13:01:21.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.546+0000 7fd1460c9700 1 -- 172.21.15.6:0/1772530982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1400739f0 con 0x7fd140157200 2023-11-28T13:01:21.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.546+0000 7fd127fff700 1 -- 172.21.15.6:0/1772530982 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd130003a20 con 0x7fd140157200 2023-11-28T13:01:21.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.547+0000 7fd127fff700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd12c068320 0x7fd12c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.547+0000 7fd127fff700 1 -- 172.21.15.6:0/1772530982 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd130085e40 con 0x7fd140157200 2023-11-28T13:01:21.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.547+0000 7fd13f7fe700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd12c068320 0x7fd12c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.547+0000 7fd13f7fe700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd12c068320 0x7fd12c06a7d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd134009c60 tx=0x7fd134009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.551+0000 7fd127fff700 1 -- 172.21.15.6:0/1772530982 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd130052370 con 0x7fd140157200 2023-11-28T13:01:21.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.746+0000 7fd1460c9700 1 -- 172.21.15.6:0/1772530982 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd1400b2230 con 0x7fd140157200 2023-11-28T13:01:21.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.746+0000 7fd127fff700 1 -- 172.21.15.6:0/1772530982 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd130003cb0 con 0x7fd140157200 2023-11-28T13:01:21.748 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:21.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 -- 172.21.15.6:0/1772530982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd12c068320 msgr2=0x7fd12c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd12c068320 0x7fd12c06a7d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd134009c60 tx=0x7fd134009210 comp rx=0 tx=0).stop 2023-11-28T13:01:21.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 -- 172.21.15.6:0/1772530982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd140157200 msgr2=0x7fd140155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd140157200 0x7fd140155830 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd13000f770 tx=0x7fd13000fa80 comp rx=0 tx=0).stop 2023-11-28T13:01:21.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 -- 172.21.15.6:0/1772530982 shutdown_connections 2023-11-28T13:01:21.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd12c068320 0x7fd12c06a7d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd140146090 0x7fd140146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1401f3280 0x7fd1401589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 --2- 172.21.15.6:0/1772530982 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd140157200 0x7fd140155830 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 -- 172.21.15.6:0/1772530982 >> 172.21.15.6:0/1772530982 conn(0x7fd1400cc630 msgr2=0x7fd140153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:21.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 -- 172.21.15.6:0/1772530982 shutdown_connections 2023-11-28T13:01:21.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.748+0000 7fd1460c9700 1 -- 172.21.15.6:0/1772530982 wait complete. 2023-11-28T13:01:21.765 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:21.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.892+0000 7f9986c25700 1 Processor -- start 2023-11-28T13:01:21.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.892+0000 7f9986c25700 1 -- start start 2023-11-28T13:01:21.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.892+0000 7f9986c25700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800cb630 0x7f99800b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f9986c25700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99800cd2a0 con 0x7f99800b2bc0 2023-11-28T13:01:21.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f9986c25700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99800d0400 con 0x7f99800b62d0 2023-11-28T13:01:21.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f9986c25700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99800cba80 con 0x7f99800cb630 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f99851c2700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f99800b2bc0 0x7f99800ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47002/0 (socket says 172.21.15.6:47002) 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f99851c2700 1 -- 172.21.15.6:0/11060868 learned_addr learned my addr 172.21.15.6:0/11060868 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997ffff700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800cb630 0x7f99800b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350110077 0 0) 0x7f99800cd2a0 con 0x7f99800b2bc0 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99700035b0 con 0x7f99800b2bc0 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 902503948 0 0) 0x7f99700035b0 con 0x7f99800b2bc0 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800cb630 msgr2=0x7f99800b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800cb630 0x7f99800b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 >> v1:172.21.15.61:6789/0 conn(0x7f99800b62d0 legacy=0x7f998006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99800d06e0 con 0x7f99800b2bc0 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f99780019f0 con 0x7f99800b2bc0 2023-11-28T13:01:21.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 >> v1:172.21.15.6:6789/0 conn(0x7f99800b2bc0 legacy=0x7f99800ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800cb630 0x7f9970003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9970003fb0 0x7f9970004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 0x7f9970005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997000bac0 con 0x7f9970004d90 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997000bc30 con 0x7f9970003fb0 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997000bdf0 con 0x7f99800cb630 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f99851c2700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 0x7f9970005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f99849c1700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800cb630 0x7f9970003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.893+0000 7f997ffff700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9970003fb0 0x7f9970004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f99851c2700 1 -- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800cb630 msgr2=0x7f9970003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f99851c2700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800cb630 0x7f9970003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f99851c2700 1 -- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9970003fb0 msgr2=0x7f9970004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f99851c2700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9970003fb0 0x7f9970004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f99851c2700 1 -- 172.21.15.6:0/11060868 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99700035b0 con 0x7f9970004d90 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f99851c2700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 0x7f9970005200 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f997800a7f0 tx=0x7f9970005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d322c5b3d3414f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997800a2f0 con 0x7f9970004d90 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9978012620 con 0x7f9970004d90 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f997f7fe700 1 -- 172.21.15.6:0/11060868 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99780129d0 con 0x7f9970004d90 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f9986c25700 1 -- 172.21.15.6:0/11060868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 msgr2=0x7f9970005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f9986c25700 1 --2- 172.21.15.6:0/11060868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 0x7f9970005200 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f997800a7f0 tx=0x7f9970005880 comp rx=0 tx=0).stop 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f99851c2700 1 -- 172.21.15.6:0/11060868 reap_dead start 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f9986c25700 1 -- 172.21.15.6:0/11060868 shutdown_connections 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.894+0000 7f9986c25700 1 -- 172.21.15.6:0/11060868 >> 172.21.15.6:0/11060868 conn(0x7f99800cd680 msgr2=0x7f99800cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.895+0000 7f9986c25700 1 -- 172.21.15.6:0/11060868 shutdown_connections 2023-11-28T13:01:21.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.895+0000 7f9986c25700 1 -- 172.21.15.6:0/11060868 wait complete. 2023-11-28T13:01:21.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.896+0000 7f9986c25700 1 Processor -- start 2023-11-28T13:01:21.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.896+0000 7f9986c25700 1 -- start start 2023-11-28T13:01:21.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f9986c25700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 0x7f9980158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f9986c25700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800c9820 0x7f99800aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f9986c25700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99801f5550 0x7f99801f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f9986c25700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99801f5f30 con 0x7f9970004d90 2023-11-28T13:01:21.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f9986c25700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9980158d70 con 0x7f99801f5550 2023-11-28T13:01:21.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f9986c25700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998006bcb0 con 0x7f99800c9820 2023-11-28T13:01:21.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99849c1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 0x7f9980158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99849c1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 0x7f9980158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59690/0 (socket says 172.21.15.6:59690) 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99849c1700 1 -- 172.21.15.6:0/1945679341 learned_addr learned my addr 172.21.15.6:0/1945679341 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f997ffff700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800c9820 0x7f99800aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99851c2700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99801f5550 0x7f99801f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99849c1700 1 -- 172.21.15.6:0/1945679341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800c9820 msgr2=0x7f99800aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99849c1700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800c9820 0x7f99800aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99849c1700 1 -- 172.21.15.6:0/1945679341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99801f5550 msgr2=0x7f99801f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99849c1700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99801f5550 0x7f99801f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99849c1700 1 -- 172.21.15.6:0/1945679341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99800b66a0 con 0x7f9970004d90 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99851c2700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99801f5550 0x7f99801f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f997ffff700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800c9820 0x7f99800aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f99849c1700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 0x7f9980158190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f997400f7f0 tx=0x7f997400fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f997d7fa700 1 -- 172.21.15.6:0/1945679341 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740052e0 con 0x7f9970004d90 2023-11-28T13:01:21.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f997d7fa700 1 -- 172.21.15.6:0/1945679341 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9974005440 con 0x7f9970004d90 2023-11-28T13:01:21.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f9986c25700 1 -- 172.21.15.6:0/1945679341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99800cffb0 con 0x7f9970004d90 2023-11-28T13:01:21.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f997d7fa700 1 -- 172.21.15.6:0/1945679341 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740038c0 con 0x7f9970004d90 2023-11-28T13:01:21.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.897+0000 7f9986c25700 1 -- 172.21.15.6:0/1945679341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99800cce50 con 0x7f9970004d90 2023-11-28T13:01:21.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.898+0000 7f9986c25700 1 -- 172.21.15.6:0/1945679341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99800739f0 con 0x7f9970004d90 2023-11-28T13:01:21.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.899+0000 7f997d7fa700 1 -- 172.21.15.6:0/1945679341 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9974003a20 con 0x7f9970004d90 2023-11-28T13:01:21.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.899+0000 7f997d7fa700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f99640682f0 0x7f996406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:21.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.899+0000 7f997d7fa700 1 -- 172.21.15.6:0/1945679341 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9974085f80 con 0x7f9970004d90 2023-11-28T13:01:21.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.899+0000 7f997ffff700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f99640682f0 0x7f996406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:21.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.900+0000 7f997ffff700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f99640682f0 0x7f996406a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99800c9c50 tx=0x7f996c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:21.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:21.904+0000 7f997d7fa700 1 -- 172.21.15.6:0/1945679341 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f99740524b0 con 0x7f9970004d90 2023-11-28T13:01:22.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.097+0000 7f9986c25700 1 -- 172.21.15.6:0/1945679341 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99800c44b0 con 0x7f9970004d90 2023-11-28T13:01:22.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.097+0000 7f997d7fa700 1 -- 172.21.15.6:0/1945679341 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9974003cb0 con 0x7f9970004d90 2023-11-28T13:01:22.099 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:22.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 -- 172.21.15.6:0/1945679341 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f99640682f0 msgr2=0x7f996406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f99640682f0 0x7f996406a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99800c9c50 tx=0x7f996c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:22.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 -- 172.21.15.6:0/1945679341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 msgr2=0x7f9980158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 0x7f9980158190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f997400f7f0 tx=0x7f997400fbb0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 -- 172.21.15.6:0/1945679341 shutdown_connections 2023-11-28T13:01:22.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f99801f5550 0x7f99801f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9970004d90 0x7f9980158190 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f99640682f0 0x7f996406a7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 --2- 172.21.15.6:0/1945679341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f99800c9820 0x7f99800aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 -- 172.21.15.6:0/1945679341 >> 172.21.15.6:0/1945679341 conn(0x7f99800cd680 msgr2=0x7f99800b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 -- 172.21.15.6:0/1945679341 shutdown_connections 2023-11-28T13:01:22.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.099+0000 7f9986c25700 1 -- 172.21.15.6:0/1945679341 wait complete. 2023-11-28T13:01:22.114 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:22.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.240+0000 7f19f704e700 1 Processor -- start 2023-11-28T13:01:22.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.240+0000 7f19f704e700 1 -- start start 2023-11-28T13:01:22.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19f704e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f00cb630 0x7f19f00b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19f704e700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f00cd2a0 con 0x7f19f00b62d0 2023-11-28T13:01:22.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19f704e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f00d0400 con 0x7f19f00b2bc0 2023-11-28T13:01:22.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19f704e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f00cba80 con 0x7f19f00cb630 2023-11-28T13:01:22.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19f4dea700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f19f00b62d0 0x7f19f006f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47018/0 (socket says 172.21.15.6:47018) 2023-11-28T13:01:22.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19f4dea700 1 -- 172.21.15.6:0/3898752750 learned_addr learned my addr 172.21.15.6:0/3898752750 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:22.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19effff700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f00cb630 0x7f19f00b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567400226 0 0) 0x7f19f00cd2a0 con 0x7f19f00b62d0 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e00035b0 con 0x7f19f00b62d0 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4289762139 0 0) 0x7f19e00035b0 con 0x7f19f00b62d0 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f00cb630 msgr2=0x7f19f00b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19ef7fe700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f00cb630 0x7f19f00b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 >> v1:172.21.15.61:6789/0 conn(0x7f19f00b2bc0 legacy=0x7f19f00ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00d06e0 con 0x7f19f00b62d0 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.241+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f19e4001670 con 0x7f19f00b62d0 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 >> v1:172.21.15.6:6789/0 conn(0x7f19f00b62d0 legacy=0x7f19f006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19ef7fe700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19e0003a30 0x7f19e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19ef7fe700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f00cb630 0x7f19e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19ef7fe700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19e0004d90 0x7f19e0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e000ba10 con 0x7f19e0004d90 2023-11-28T13:01:22.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e000bb50 con 0x7f19e0003a30 2023-11-28T13:01:22.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e000bce0 con 0x7f19f00cb630 2023-11-28T13:01:22.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19f55eb700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19e0004d90 0x7f19e0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19effff700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f00cb630 0x7f19e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19f4dea700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19e0003a30 0x7f19e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19f55eb700 1 -- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f00cb630 msgr2=0x7f19e0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:22.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19f55eb700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f00cb630 0x7f19e0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19f55eb700 1 -- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19e0003a30 msgr2=0x7f19e0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:22.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19f55eb700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19e0003a30 0x7f19e0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19f55eb700 1 -- 172.21.15.6:0/3898752750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00035b0 con 0x7f19e0004d90 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19f55eb700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19e0004d90 0x7f19e0005200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c290 tx=0x7f19e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9052b52b0a9fbf03 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8001e70 con 0x7f19e0004d90 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e800ce50 con 0x7f19e0004d90 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.242+0000 7f19ef7fe700 1 -- 172.21.15.6:0/3898752750 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8012660 con 0x7f19e0004d90 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.243+0000 7f19f704e700 1 -- 172.21.15.6:0/3898752750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19e0004d90 msgr2=0x7f19e0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.243+0000 7f19f704e700 1 --2- 172.21.15.6:0/3898752750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19e0004d90 0x7f19e0005200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c290 tx=0x7f19e0005880 comp rx=0 tx=0).stop 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.243+0000 7f19f55eb700 1 -- 172.21.15.6:0/3898752750 reap_dead start 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.243+0000 7f19f704e700 1 -- 172.21.15.6:0/3898752750 shutdown_connections 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.243+0000 7f19f704e700 1 -- 172.21.15.6:0/3898752750 >> 172.21.15.6:0/3898752750 conn(0x7f19f00cd680 msgr2=0x7f19f00cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.243+0000 7f19f704e700 1 -- 172.21.15.6:0/3898752750 shutdown_connections 2023-11-28T13:01:22.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.243+0000 7f19f704e700 1 -- 172.21.15.6:0/3898752750 wait complete. 2023-11-28T13:01:22.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.244+0000 7f19f704e700 1 Processor -- start 2023-11-28T13:01:22.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.244+0000 7f19f704e700 1 -- start start 2023-11-28T13:01:22.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f704e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f00c70d0 0x7f19f015a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f704e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f01564c0 0x7f19f015af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f704e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f014a470 0x7f19f0157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f704e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f0147cd0 con 0x7f19f00c70d0 2023-11-28T13:01:22.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f704e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f0147fb0 con 0x7f19f01564c0 2023-11-28T13:01:22.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f704e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f0148290 con 0x7f19f014a470 2023-11-28T13:01:22.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f4dea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f00c70d0 0x7f19f015a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f4dea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f00c70d0 0x7f19f015a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59710/0 (socket says 172.21.15.6:59710) 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f4dea700 1 -- 172.21.15.6:0/1718202844 learned_addr learned my addr 172.21.15.6:0/1718202844 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19effff700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f01564c0 0x7f19f015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f55eb700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f014a470 0x7f19f0157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f4dea700 1 -- 172.21.15.6:0/1718202844 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f014a470 msgr2=0x7f19f0157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f4dea700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f014a470 0x7f19f0157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f4dea700 1 -- 172.21.15.6:0/1718202844 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f01564c0 msgr2=0x7f19f015af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f4dea700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f01564c0 0x7f19f015af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.245+0000 7f19f4dea700 1 -- 172.21.15.6:0/1718202844 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00b66a0 con 0x7f19f00c70d0 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.246+0000 7f19f4dea700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f00c70d0 0x7f19f015a9e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f19e400eb70 tx=0x7f19e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.246+0000 7f19ed7fa700 1 -- 172.21.15.6:0/1718202844 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4005e10 con 0x7f19f00c70d0 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.246+0000 7f19ed7fa700 1 -- 172.21.15.6:0/1718202844 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e400c7e0 con 0x7f19f00c70d0 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.246+0000 7f19f704e700 1 -- 172.21.15.6:0/1718202844 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f00cffb0 con 0x7f19f00c70d0 2023-11-28T13:01:22.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.246+0000 7f19ed7fa700 1 -- 172.21.15.6:0/1718202844 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e400cb90 con 0x7f19f00c70d0 2023-11-28T13:01:22.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.246+0000 7f19f704e700 1 -- 172.21.15.6:0/1718202844 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f00cce50 con 0x7f19f00c70d0 2023-11-28T13:01:22.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.247+0000 7f19f704e700 1 -- 172.21.15.6:0/1718202844 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f00739f0 con 0x7f19f00c70d0 2023-11-28T13:01:22.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.247+0000 7f19ed7fa700 1 -- 172.21.15.6:0/1718202844 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19e400ccf0 con 0x7f19f00c70d0 2023-11-28T13:01:22.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.248+0000 7f19ed7fa700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d40682f0 0x7f19d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.248+0000 7f19ed7fa700 1 -- 172.21.15.6:0/1718202844 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f19e4085e90 con 0x7f19f00c70d0 2023-11-28T13:01:22.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.248+0000 7f19effff700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d40682f0 0x7f19d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.248+0000 7f19effff700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d40682f0 0x7f19d406a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f19f0066110 tx=0x7f19dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.252+0000 7f19ed7fa700 1 -- 172.21.15.6:0/1718202844 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19e40523c0 con 0x7f19f00c70d0 2023-11-28T13:01:22.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.447+0000 7f19f704e700 1 -- 172.21.15.6:0/1718202844 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f19f00cf380 con 0x7f19f00c70d0 2023-11-28T13:01:22.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.447+0000 7f19ed7fa700 1 -- 172.21.15.6:0/1718202844 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f19e40559e0 con 0x7f19f00c70d0 2023-11-28T13:01:22.449 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:22.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.449+0000 7f19f704e700 1 -- 172.21.15.6:0/1718202844 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d40682f0 msgr2=0x7f19d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.449+0000 7f19f704e700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d40682f0 0x7f19d406a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f19f0066110 tx=0x7f19dc008040 comp rx=0 tx=0).stop 2023-11-28T13:01:22.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.449+0000 7f19f704e700 1 -- 172.21.15.6:0/1718202844 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f00c70d0 msgr2=0x7f19f015a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.449+0000 7f19f704e700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f00c70d0 0x7f19f015a9e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f19e400eb70 tx=0x7f19e400ee80 comp rx=0 tx=0).stop 2023-11-28T13:01:22.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.450+0000 7f19f704e700 1 -- 172.21.15.6:0/1718202844 shutdown_connections 2023-11-28T13:01:22.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.450+0000 7f19f704e700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d40682f0 0x7f19d406a7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.450+0000 7f19f704e700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f014a470 0x7f19f0157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.450+0000 7f19f704e700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f00c70d0 0x7f19f015a9e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.450+0000 7f19f704e700 1 --2- 172.21.15.6:0/1718202844 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f01564c0 0x7f19f015af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.450+0000 7f19f704e700 1 -- 172.21.15.6:0/1718202844 >> 172.21.15.6:0/1718202844 conn(0x7f19f00cd680 msgr2=0x7f19f00ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.450+0000 7f19f704e700 1 -- 172.21.15.6:0/1718202844 shutdown_connections 2023-11-28T13:01:22.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.450+0000 7f19f704e700 1 -- 172.21.15.6:0/1718202844 wait complete. 2023-11-28T13:01:22.463 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:22.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.590+0000 7f2a92e73700 1 Processor -- start 2023-11-28T13:01:22.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.590+0000 7f2a92e73700 1 -- start start 2023-11-28T13:01:22.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.591+0000 7f2a92e73700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f2a8c0b1d40 0x7f2a8c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.591+0000 7f2a92e73700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2a8c15dbf0 0x7f2a8c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.591+0000 7f2a92e73700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8c0ceff0 con 0x7f2a8c159500 2023-11-28T13:01:22.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.591+0000 7f2a92e73700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8c15e090 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.591+0000 7f2a92e73700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cede0 con 0x7f2a8c0b1d40 2023-11-28T13:01:22.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.591+0000 7f2a91410700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2a8c15dbf0 0x7f2a8c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.591+0000 7f2a91410700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2a8c15dbf0 0x7f2a8c0ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59722/0 (socket says 172.21.15.6:59722) 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.591+0000 7f2a91410700 1 -- 172.21.15.6:0/3236014428 learned_addr learned my addr 172.21.15.6:0/3236014428 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a8bfff700 1 --2- 172.21.15.6:0/3236014428 >> v2:172.21.15.61:3300/0 conn(0x7f2a8c0b1d40 0x7f2a8c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a91410700 1 -- 172.21.15.6:0/3236014428 >> v2:172.21.15.61:3300/0 conn(0x7f2a8c0b1d40 msgr2=0x7f2a8c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a91410700 1 --2- 172.21.15.6:0/3236014428 >> v2:172.21.15.61:3300/0 conn(0x7f2a8c0b1d40 0x7f2a8c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a91410700 1 -- 172.21.15.6:0/3236014428 >> v1:172.21.15.61:6789/0 conn(0x7f2a8c159500 legacy=0x7f2a8c0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a91410700 1 -- 172.21.15.6:0/3236014428 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c0cf2d0 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a91410700 1 --2- 172.21.15.6:0/3236014428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 0x7f2a8c0ce8a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0d0c50 tx=0x7f2a8400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c61276e0b446736 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a8b7fe700 1 -- 172.21.15.6:0/3236014428 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84010070 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a8b7fe700 1 -- 172.21.15.6:0/3236014428 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8400dd50 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a8b7fe700 1 -- 172.21.15.6:0/3236014428 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400b310 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a92e73700 1 -- 172.21.15.6:0/3236014428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 msgr2=0x7f2a8c0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.592+0000 7f2a92e73700 1 --2- 172.21.15.6:0/3236014428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 0x7f2a8c0ce8a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0d0c50 tx=0x7f2a8400d430 comp rx=0 tx=0).stop 2023-11-28T13:01:22.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.593+0000 7f2a92e73700 1 -- 172.21.15.6:0/3236014428 shutdown_connections 2023-11-28T13:01:22.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.593+0000 7f2a92e73700 1 --2- 172.21.15.6:0/3236014428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 0x7f2a8c0ce8a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.593+0000 7f2a92e73700 1 --2- 172.21.15.6:0/3236014428 >> v2:172.21.15.61:3300/0 conn(0x7f2a8c0b1d40 0x7f2a8c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.593+0000 7f2a92e73700 1 -- 172.21.15.6:0/3236014428 >> 172.21.15.6:0/3236014428 conn(0x7f2a8c0cc630 msgr2=0x7f2a8c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.593+0000 7f2a92e73700 1 -- 172.21.15.6:0/3236014428 shutdown_connections 2023-11-28T13:01:22.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.593+0000 7f2a92e73700 1 -- 172.21.15.6:0/3236014428 wait complete. 2023-11-28T13:01:22.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.594+0000 7f2a92e73700 1 Processor -- start 2023-11-28T13:01:22.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.594+0000 7f2a92e73700 1 -- start start 2023-11-28T13:01:22.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.595+0000 7f2a92e73700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a8c0b1d40 0x7f2a8c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.595+0000 7f2a92e73700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a8c159500 0x7f2a8c0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.595+0000 7f2a92e73700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 0x7f2a8c154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.595+0000 7f2a92e73700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0b31f0 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.595+0000 7f2a92e73700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c06c720 con 0x7f2a8c159500 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.595+0000 7f2a92e73700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c06ca00 con 0x7f2a8c0b1d40 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.595+0000 7f2a91410700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 0x7f2a8c154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.595+0000 7f2a91410700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 0x7f2a8c154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59728/0 (socket says 172.21.15.6:59728) 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.595+0000 7f2a91410700 1 -- 172.21.15.6:0/2496742495 learned_addr learned my addr 172.21.15.6:0/2496742495 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a8bfff700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a8c159500 0x7f2a8c0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a90c0f700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a8c0b1d40 0x7f2a8c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a91410700 1 -- 172.21.15.6:0/2496742495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a8c0b1d40 msgr2=0x7f2a8c159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a91410700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a8c0b1d40 0x7f2a8c159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a91410700 1 -- 172.21.15.6:0/2496742495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a8c159500 msgr2=0x7f2a8c0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a91410700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a8c159500 0x7f2a8c0b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a91410700 1 -- 172.21.15.6:0/2496742495 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8400bd40 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a91410700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 0x7f2a8c154290 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2a84017040 tx=0x7f2a84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a897fa700 1 -- 172.21.15.6:0/2496742495 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84009a10 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a897fa700 1 -- 172.21.15.6:0/2496742495 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a84009b90 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a92e73700 1 -- 172.21.15.6:0/2496742495 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c15d3f0 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a897fa700 1 -- 172.21.15.6:0/2496742495 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8400b4c0 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.596+0000 7f2a92e73700 1 -- 172.21.15.6:0/2496742495 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c15d110 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.597+0000 7f2a92e73700 1 -- 172.21.15.6:0/2496742495 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8c15d6d0 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.598+0000 7f2a897fa700 1 -- 172.21.15.6:0/2496742495 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2a84004030 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.598+0000 7f2a897fa700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2a700682f0 0x7f2a7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.598+0000 7f2a897fa700 1 -- 172.21.15.6:0/2496742495 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2a840896b0 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.598+0000 7f2a90c0f700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2a700682f0 0x7f2a7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.599+0000 7f2a90c0f700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2a700682f0 0x7f2a7006a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2a80007fd0 tx=0x7f2a80007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.603+0000 7f2a897fa700 1 -- 172.21.15.6:0/2496742495 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2a84055be0 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.797+0000 7f2a92e73700 1 -- 172.21.15.6:0/2496742495 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a8c0739f0 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.798+0000 7f2a897fa700 1 -- 172.21.15.6:0/2496742495 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a84059200 con 0x7f2a8c15dbf0 2023-11-28T13:01:22.799 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 -- 172.21.15.6:0/2496742495 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2a700682f0 msgr2=0x7f2a7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2a700682f0 0x7f2a7006a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2a80007fd0 tx=0x7f2a80007ec0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 -- 172.21.15.6:0/2496742495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 msgr2=0x7f2a8c154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 0x7f2a8c154290 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2a84017040 tx=0x7f2a84004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 -- 172.21.15.6:0/2496742495 shutdown_connections 2023-11-28T13:01:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a8c15dbf0 0x7f2a8c154290 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2a700682f0 0x7f2a7006a7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2a8c0b1d40 0x7f2a8c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 --2- 172.21.15.6:0/2496742495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a8c159500 0x7f2a8c0b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 -- 172.21.15.6:0/2496742495 >> 172.21.15.6:0/2496742495 conn(0x7f2a8c0cc630 msgr2=0x7f2a8c157140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 -- 172.21.15.6:0/2496742495 shutdown_connections 2023-11-28T13:01:22.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.800+0000 7f2a92e73700 1 -- 172.21.15.6:0/2496742495 wait complete. 2023-11-28T13:01:22.814 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:22.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.940+0000 7fb961ae2700 1 Processor -- start 2023-11-28T13:01:22.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.940+0000 7fb961ae2700 1 -- start start 2023-11-28T13:01:22.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb961ae2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb95c0b62d0 0x7fb95c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb961ae2700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c0cd2a0 con 0x7fb95c06f8b0 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb961ae2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c0d0400 con 0x7fb95c0b2d90 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb961ae2700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c0d06e0 con 0x7fb95c0b62d0 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb95b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb95c0b62d0 0x7fb95c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb95b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb95c0b62d0 0x7fb95c0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59732/0 (socket says 172.21.15.6:59732) 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb95affd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb95c06f8b0 0x7fb95c0b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47052/0 (socket says 172.21.15.6:47052) 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb95b7fe700 1 -- 172.21.15.6:0/3247071005 learned_addr learned my addr 172.21.15.6:0/3247071005 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb95b7fe700 1 -- 172.21.15.6:0/3247071005 >> v1:172.21.15.61:6789/0 conn(0x7fb95c0b2d90 legacy=0x7fb95c0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb95b7fe700 1 -- 172.21.15.6:0/3247071005 >> v1:172.21.15.6:6789/0 conn(0x7fb95c06f8b0 legacy=0x7fb95c0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb95a7fc700 1 -- 172.21.15.6:0/3247071005 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326542169 0 0) 0x7fb95c0cd2a0 con 0x7fb95c06f8b0 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb95b7fe700 1 -- 172.21.15.6:0/3247071005 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c0cffb0 con 0x7fb95c0b62d0 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb95b7fe700 1 --2- 172.21.15.6:0/3247071005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c0b62d0 0x7fb95c0cb630 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb9500098f0 tx=0x7fb950009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e1e9ce77f40361 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.941+0000 7fb95a7fc700 1 -- 172.21.15.6:0/3247071005 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950004230 con 0x7fb95c0b62d0 2023-11-28T13:01:22.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.942+0000 7fb95a7fc700 1 -- 172.21.15.6:0/3247071005 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb950004390 con 0x7fb95c0b62d0 2023-11-28T13:01:22.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.942+0000 7fb95a7fc700 1 -- 172.21.15.6:0/3247071005 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9500036a0 con 0x7fb95c0b62d0 2023-11-28T13:01:22.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.942+0000 7fb961ae2700 1 -- 172.21.15.6:0/3247071005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c0b62d0 msgr2=0x7fb95c0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.942+0000 7fb961ae2700 1 --2- 172.21.15.6:0/3247071005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c0b62d0 0x7fb95c0cb630 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb9500098f0 tx=0x7fb950009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:22.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.942+0000 7fb961ae2700 1 -- 172.21.15.6:0/3247071005 shutdown_connections 2023-11-28T13:01:22.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.942+0000 7fb961ae2700 1 --2- 172.21.15.6:0/3247071005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c0b62d0 0x7fb95c0cb630 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.942+0000 7fb961ae2700 1 -- 172.21.15.6:0/3247071005 >> 172.21.15.6:0/3247071005 conn(0x7fb95c0cd680 msgr2=0x7fb95c0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:22.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.942+0000 7fb961ae2700 1 -- 172.21.15.6:0/3247071005 shutdown_connections 2023-11-28T13:01:22.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.942+0000 7fb961ae2700 1 -- 172.21.15.6:0/3247071005 wait complete. 2023-11-28T13:01:22.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.943+0000 7fb961ae2700 1 Processor -- start 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.943+0000 7fb961ae2700 1 -- start start 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.944+0000 7fb961ae2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c06f8b0 0x7fb95c0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.944+0000 7fb961ae2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb95c0b2d90 0x7fb95c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.944+0000 7fb961ae2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb95c0b62d0 0x7fb95c06cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.944+0000 7fb961ae2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0c8710 con 0x7fb95c06f8b0 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.944+0000 7fb961ae2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0c9070 con 0x7fb95c0b2d90 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.944+0000 7fb961ae2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c1545a0 con 0x7fb95c0b62d0 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.944+0000 7fb95b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c06f8b0 0x7fb95c0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.944+0000 7fb95b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c06f8b0 0x7fb95c0c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59738/0 (socket says 172.21.15.6:59738) 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.944+0000 7fb95b7fe700 1 -- 172.21.15.6:0/1757505957 learned_addr learned my addr 172.21.15.6:0/1757505957 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:22.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.944+0000 7fb95affd700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb95c0b2d90 0x7fb95c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb95bfff700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb95c0b62d0 0x7fb95c06cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb95b7fe700 1 -- 172.21.15.6:0/1757505957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb95c0b62d0 msgr2=0x7fb95c06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb95b7fe700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb95c0b62d0 0x7fb95c06cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb95b7fe700 1 -- 172.21.15.6:0/1757505957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb95c0b2d90 msgr2=0x7fb95c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb95b7fe700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb95c0b2d90 0x7fb95c14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb95b7fe700 1 -- 172.21.15.6:0/1757505957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb950009550 con 0x7fb95c06f8b0 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb95b7fe700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c06f8b0 0x7fb95c0c8b30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb950008000 tx=0x7fb950004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb943fff700 1 -- 172.21.15.6:0/1757505957 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000b660 con 0x7fb95c06f8b0 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb943fff700 1 -- 172.21.15.6:0/1757505957 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95000b7c0 con 0x7fb95c06f8b0 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb961ae2700 1 -- 172.21.15.6:0/1757505957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95c0b66a0 con 0x7fb95c06f8b0 2023-11-28T13:01:22.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb943fff700 1 -- 172.21.15.6:0/1757505957 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000bb70 con 0x7fb95c06f8b0 2023-11-28T13:01:22.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.945+0000 7fb961ae2700 1 -- 172.21.15.6:0/1757505957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c06fd30 con 0x7fb95c06f8b0 2023-11-28T13:01:22.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.946+0000 7fb961ae2700 1 -- 172.21.15.6:0/1757505957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c0cce50 con 0x7fb95c06f8b0 2023-11-28T13:01:22.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.946+0000 7fb943fff700 1 -- 172.21.15.6:0/1757505957 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb950017020 con 0x7fb95c06f8b0 2023-11-28T13:01:22.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.947+0000 7fb943fff700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9480682f0 0x7fb94806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:22.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.947+0000 7fb943fff700 1 -- 172.21.15.6:0/1757505957 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb950087830 con 0x7fb95c06f8b0 2023-11-28T13:01:22.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.947+0000 7fb95affd700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9480682f0 0x7fb94806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:22.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.947+0000 7fb95affd700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9480682f0 0x7fb94806a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0b4cf0 tx=0x7fb94c007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:22.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:22.951+0000 7fb943fff700 1 -- 172.21.15.6:0/1757505957 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb950053d60 con 0x7fb95c06f8b0 2023-11-28T13:01:23.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.145+0000 7fb961ae2700 1 -- 172.21.15.6:0/1757505957 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb95c0c8980 con 0x7fb95c06f8b0 2023-11-28T13:01:23.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.146+0000 7fb943fff700 1 -- 172.21.15.6:0/1757505957 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb950010380 con 0x7fb95c06f8b0 2023-11-28T13:01:23.147 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:23.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 -- 172.21.15.6:0/1757505957 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9480682f0 msgr2=0x7fb94806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9480682f0 0x7fb94806a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0b4cf0 tx=0x7fb94c007de0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 -- 172.21.15.6:0/1757505957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c06f8b0 msgr2=0x7fb95c0c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c06f8b0 0x7fb95c0c8b30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb950008000 tx=0x7fb950004750 comp rx=0 tx=0).stop 2023-11-28T13:01:23.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 -- 172.21.15.6:0/1757505957 shutdown_connections 2023-11-28T13:01:23.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb9480682f0 0x7fb94806a7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb95c0b62d0 0x7fb95c06cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb95c06f8b0 0x7fb95c0c8b30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 --2- 172.21.15.6:0/1757505957 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb95c0b2d90 0x7fb95c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 -- 172.21.15.6:0/1757505957 >> 172.21.15.6:0/1757505957 conn(0x7fb95c0cd680 msgr2=0x7fb95c0b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 -- 172.21.15.6:0/1757505957 shutdown_connections 2023-11-28T13:01:23.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.148+0000 7fb961ae2700 1 -- 172.21.15.6:0/1757505957 wait complete. 2023-11-28T13:01:23.161 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:23.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.286+0000 7f3685918700 1 Processor -- start 2023-11-28T13:01:23.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.286+0000 7f3685918700 1 -- start start 2023-11-28T13:01:23.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.287+0000 7f3685918700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f368015c990 0x7f36800cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.287+0000 7f3685918700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f36800cc310 0x7f36800cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.287+0000 7f3685918700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800ce340 con 0x7f3680161090 2023-11-28T13:01:23.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.287+0000 7f3685918700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36800cebe0 con 0x7f368015c990 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.287+0000 7f3685918700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36800cccb0 con 0x7f36800cc310 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.287+0000 7f367effd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f368015c990 0x7f36800cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.287+0000 7f367e7fc700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f3680161090 0x7f36800ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47066/0 (socket says 172.21.15.6:47066) 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.287+0000 7f367e7fc700 1 -- 172.21.15.6:0/404331649 learned_addr learned my addr 172.21.15.6:0/404331649 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.288+0000 7f367f7fe700 1 --2- 172.21.15.6:0/404331649 >> v2:172.21.15.61:3300/0 conn(0x7f36800cc310 0x7f36800cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.288+0000 7f367effd700 1 -- 172.21.15.6:0/404331649 >> v2:172.21.15.61:3300/0 conn(0x7f36800cc310 msgr2=0x7f36800cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.288+0000 7f367effd700 1 --2- 172.21.15.6:0/404331649 >> v2:172.21.15.61:3300/0 conn(0x7f36800cc310 0x7f36800cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.288+0000 7f367effd700 1 -- 172.21.15.6:0/404331649 >> v1:172.21.15.6:6789/0 conn(0x7f3680161090 legacy=0x7f36800ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.288+0000 7f367effd700 1 -- 172.21.15.6:0/404331649 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800cb3f0 con 0x7f368015c990 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.288+0000 7f367dffb700 1 -- 172.21.15.6:0/404331649 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764466612 0 0) 0x7f36800ce340 con 0x7f3680161090 2023-11-28T13:01:23.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.288+0000 7f367effd700 1 --2- 172.21.15.6:0/404331649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f368015c990 0x7f36800cf070 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3680071a50 tx=0x7f3674009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4516190c47733a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.288+0000 7f367dffb700 1 -- 172.21.15.6:0/404331649 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674011830 con 0x7f368015c990 2023-11-28T13:01:23.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.288+0000 7f367dffb700 1 -- 172.21.15.6:0/404331649 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3674011990 con 0x7f368015c990 2023-11-28T13:01:23.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.288+0000 7f367dffb700 1 -- 172.21.15.6:0/404331649 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674011d20 con 0x7f368015c990 2023-11-28T13:01:23.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.289+0000 7f3685918700 1 -- 172.21.15.6:0/404331649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f368015c990 msgr2=0x7f36800cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.289+0000 7f3685918700 1 --2- 172.21.15.6:0/404331649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f368015c990 0x7f36800cf070 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3680071a50 tx=0x7f3674009b20 comp rx=0 tx=0).stop 2023-11-28T13:01:23.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.289+0000 7f3685918700 1 -- 172.21.15.6:0/404331649 shutdown_connections 2023-11-28T13:01:23.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.289+0000 7f3685918700 1 --2- 172.21.15.6:0/404331649 >> v2:172.21.15.61:3300/0 conn(0x7f36800cc310 0x7f36800cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.289+0000 7f3685918700 1 --2- 172.21.15.6:0/404331649 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f368015c990 0x7f36800cf070 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.289+0000 7f3685918700 1 -- 172.21.15.6:0/404331649 >> 172.21.15.6:0/404331649 conn(0x7f36800b1650 msgr2=0x7f36800b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.289+0000 7f3685918700 1 -- 172.21.15.6:0/404331649 shutdown_connections 2023-11-28T13:01:23.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.289+0000 7f3685918700 1 -- 172.21.15.6:0/404331649 wait complete. 2023-11-28T13:01:23.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.290+0000 7f3685918700 1 Processor -- start 2023-11-28T13:01:23.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.290+0000 7f3685918700 1 -- start start 2023-11-28T13:01:23.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f3685918700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36800cc310 0x7f368014f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f3685918700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368015c990 0x7f36800b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f3685918700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3680161090 0x7f36800c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f3685918700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800b3150 con 0x7f36800cc310 2023-11-28T13:01:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f3685918700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800b2e70 con 0x7f368015c990 2023-11-28T13:01:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f3685918700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800b3ed0 con 0x7f3680161090 2023-11-28T13:01:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f367effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36800cc310 0x7f368014f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f367effd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36800cc310 0x7f368014f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59752/0 (socket says 172.21.15.6:59752) 2023-11-28T13:01:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f367effd700 1 -- 172.21.15.6:0/2348112090 learned_addr learned my addr 172.21.15.6:0/2348112090 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f367effd700 1 -- 172.21.15.6:0/2348112090 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3680161090 msgr2=0x7f36800c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f367effd700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3680161090 0x7f36800c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f367effd700 1 -- 172.21.15.6:0/2348112090 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368015c990 msgr2=0x7f36800b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f367e7fc700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368015c990 0x7f36800b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f367effd700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368015c990 0x7f36800b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f367effd700 1 -- 172.21.15.6:0/2348112090 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36740094a0 con 0x7f36800cc310 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f367effd700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36800cc310 0x7f368014f0c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f36740090b0 tx=0x7f3674004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.291+0000 7f3667fff700 1 -- 172.21.15.6:0/2348112090 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36740045b0 con 0x7f36800cc310 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.292+0000 7f3667fff700 1 -- 172.21.15.6:0/2348112090 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367401f070 con 0x7f36800cc310 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.292+0000 7f3685918700 1 -- 172.21.15.6:0/2348112090 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36800ce870 con 0x7f36800cc310 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.292+0000 7f3667fff700 1 -- 172.21.15.6:0/2348112090 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36740038c0 con 0x7f36800cc310 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.292+0000 7f3685918700 1 -- 172.21.15.6:0/2348112090 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36800ce590 con 0x7f36800cc310 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.293+0000 7f3685918700 1 -- 172.21.15.6:0/2348112090 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36800cb080 con 0x7f36800cc310 2023-11-28T13:01:23.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.293+0000 7f3667fff700 1 -- 172.21.15.6:0/2348112090 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f367402a020 con 0x7f36800cc310 2023-11-28T13:01:23.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.293+0000 7f3667fff700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f366c068320 0x7f366c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.294+0000 7f367e7fc700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f366c068320 0x7f366c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.294+0000 7f3667fff700 1 -- 172.21.15.6:0/2348112090 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3674087450 con 0x7f36800cc310 2023-11-28T13:01:23.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.294+0000 7f367e7fc700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f366c068320 0x7f366c06a7d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3670000f60 tx=0x7f3670009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.298+0000 7f3667fff700 1 -- 172.21.15.6:0/2348112090 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3674053980 con 0x7f36800cc310 2023-11-28T13:01:23.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.513+0000 7f3685918700 1 -- 172.21.15.6:0/2348112090 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f36800675a0 con 0x7f36800cc310 2023-11-28T13:01:23.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.513+0000 7f3667fff700 1 -- 172.21.15.6:0/2348112090 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3674056fa0 con 0x7f36800cc310 2023-11-28T13:01:23.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 -- 172.21.15.6:0/2348112090 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f366c068320 msgr2=0x7f366c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f366c068320 0x7f366c06a7d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3670000f60 tx=0x7f3670009040 comp rx=0 tx=0).stop 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 -- 172.21.15.6:0/2348112090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36800cc310 msgr2=0x7f368014f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36800cc310 0x7f368014f0c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f36740090b0 tx=0x7f3674004750 comp rx=0 tx=0).stop 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 -- 172.21.15.6:0/2348112090 shutdown_connections 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f366c068320 0x7f366c06a7d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3680161090 0x7f36800c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36800cc310 0x7f368014f0c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 --2- 172.21.15.6:0/2348112090 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f368015c990 0x7f36800b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 -- 172.21.15.6:0/2348112090 >> 172.21.15.6:0/2348112090 conn(0x7f36800b1650 msgr2=0x7f368006b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 -- 172.21.15.6:0/2348112090 shutdown_connections 2023-11-28T13:01:23.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.515+0000 7f3685918700 1 -- 172.21.15.6:0/2348112090 wait complete. 2023-11-28T13:01:23.518 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:01:23.528 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:01:23.529 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:23.550 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:23.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.674+0000 7ff6b3793700 1 Processor -- start 2023-11-28T13:01:23.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.674+0000 7ff6b3793700 1 -- start start 2023-11-28T13:01:23.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b3793700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6ac15a6c0 0x7ff6ac0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b3793700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff6ac0d0020 0x7ff6ac0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b3793700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac0cfb90 con 0x7ff6ac0b2660 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b3793700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ac0cb930 con 0x7ff6ac0d0020 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b3793700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0cf260 con 0x7ff6ac15a6c0 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b0d2e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff6ac0d0020 0x7ff6ac0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b0d2e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff6ac0d0020 0x7ff6ac0ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59760/0 (socket says 172.21.15.6:59760) 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b0d2e700 1 -- 172.21.15.6:0/851445400 learned_addr learned my addr 172.21.15.6:0/851445400 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b1d30700 1 --1- 172.21.15.6:0/851445400 >> v1:172.21.15.6:6789/0 conn(0x7ff6ac0b2660 0x7ff6ac0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47082/0 (socket says 172.21.15.6:47082) 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b152f700 1 --2- 172.21.15.6:0/851445400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6ac15a6c0 0x7ff6ac0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b0d2e700 1 -- 172.21.15.6:0/851445400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6ac15a6c0 msgr2=0x7ff6ac0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:23.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b0d2e700 1 --2- 172.21.15.6:0/851445400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6ac15a6c0 0x7ff6ac0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b0d2e700 1 -- 172.21.15.6:0/851445400 >> v1:172.21.15.6:6789/0 conn(0x7ff6ac0b2660 legacy=0x7ff6ac0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6b0d2e700 1 -- 172.21.15.6:0/851445400 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac0cf540 con 0x7ff6ac0d0020 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.675+0000 7ff6a3fff700 1 -- 172.21.15.6:0/851445400 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264883724 0 0) 0x7ff6ac0cfb90 con 0x7ff6ac0b2660 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6b0d2e700 1 --2- 172.21.15.6:0/851445400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0d0020 0x7ff6ac0ccea0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0098f0 tx=0x7ff69c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b4cca048fab9e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6a3fff700 1 -- 172.21.15.6:0/851445400 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00b8a0 con 0x7ff6ac0d0020 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6a3fff700 1 -- 172.21.15.6:0/851445400 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c00ba00 con 0x7ff6ac0d0020 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6a3fff700 1 -- 172.21.15.6:0/851445400 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00bd90 con 0x7ff6ac0d0020 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6b3793700 1 -- 172.21.15.6:0/851445400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0d0020 msgr2=0x7ff6ac0ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6b3793700 1 --2- 172.21.15.6:0/851445400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0d0020 0x7ff6ac0ccea0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0098f0 tx=0x7ff69c009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6b3793700 1 -- 172.21.15.6:0/851445400 shutdown_connections 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6b3793700 1 --2- 172.21.15.6:0/851445400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6ac15a6c0 0x7ff6ac0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6b3793700 1 --2- 172.21.15.6:0/851445400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0d0020 0x7ff6ac0ccea0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6b3793700 1 -- 172.21.15.6:0/851445400 >> 172.21.15.6:0/851445400 conn(0x7ff6ac0cd660 msgr2=0x7ff6ac0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6b3793700 1 -- 172.21.15.6:0/851445400 shutdown_connections 2023-11-28T13:01:23.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.676+0000 7ff6b3793700 1 -- 172.21.15.6:0/851445400 wait complete. 2023-11-28T13:01:23.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.678+0000 7ff6b3793700 1 Processor -- start 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.678+0000 7ff6b3793700 1 -- start start 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.678+0000 7ff6b3793700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0b2660 0x7ff6ac1f1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.678+0000 7ff6b3793700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6ac0d0020 0x7ff6ac15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.678+0000 7ff6b3793700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6ac15a6c0 0x7ff6ac0b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.678+0000 7ff6b3793700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac06bb80 con 0x7ff6ac0b2660 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.678+0000 7ff6b3793700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac15c4c0 con 0x7ff6ac15a6c0 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.678+0000 7ff6b3793700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac15c7a0 con 0x7ff6ac0d0020 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.678+0000 7ff6b152f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0b2660 0x7ff6ac1f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6b152f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0b2660 0x7ff6ac1f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59764/0 (socket says 172.21.15.6:59764) 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6b152f700 1 -- 172.21.15.6:0/4242589658 learned_addr learned my addr 172.21.15.6:0/4242589658 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6b152f700 1 -- 172.21.15.6:0/4242589658 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6ac0d0020 msgr2=0x7ff6ac15b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6b152f700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6ac0d0020 0x7ff6ac15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6b152f700 1 -- 172.21.15.6:0/4242589658 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6ac15a6c0 msgr2=0x7ff6ac0b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6b152f700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6ac15a6c0 0x7ff6ac0b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6b152f700 1 -- 172.21.15.6:0/4242589658 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c009550 con 0x7ff6ac0b2660 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6b152f700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0b2660 0x7ff6ac1f1430 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800d800 tx=0x7ff6a800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6a1ffb700 1 -- 172.21.15.6:0/4242589658 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a80125f0 con 0x7ff6ac0b2660 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6a1ffb700 1 -- 172.21.15.6:0/4242589658 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a8012750 con 0x7ff6ac0b2660 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6a1ffb700 1 -- 172.21.15.6:0/4242589658 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8012b00 con 0x7ff6ac0b2660 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6b3793700 1 -- 172.21.15.6:0/4242589658 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac0cd3e0 con 0x7ff6ac0b2660 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.679+0000 7ff6b3793700 1 -- 172.21.15.6:0/4242589658 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac0cf820 con 0x7ff6ac0b2660 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.680+0000 7ff6b3793700 1 -- 172.21.15.6:0/4242589658 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac0d0470 con 0x7ff6ac0b2660 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.680+0000 7ff6a1ffb700 1 -- 172.21.15.6:0/4242589658 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff6a8012c60 con 0x7ff6ac0b2660 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.681+0000 7ff6a1ffb700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6900682f0 0x7ff69006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:23.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.681+0000 7ff6a1ffb700 1 -- 172.21.15.6:0/4242589658 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff6a80864e0 con 0x7ff6ac0b2660 2023-11-28T13:01:23.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.681+0000 7ff6b0d2e700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6900682f0 0x7ff69006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:23.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.681+0000 7ff6b0d2e700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6900682f0 0x7ff69006a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0b6190 tx=0x7ff69c010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:23.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.685+0000 7ff6a1ffb700 1 -- 172.21.15.6:0/4242589658 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff6a8052a10 con 0x7ff6ac0b2660 2023-11-28T13:01:23.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.882+0000 7ff6b3793700 1 -- 172.21.15.6:0/4242589658 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6ac0675a0 con 0x7ff6ac0b2660 2023-11-28T13:01:23.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.883+0000 7ff6a1ffb700 1 -- 172.21.15.6:0/4242589658 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19690 (secure 0 0 0) 0x7ff6a8056030 con 0x7ff6ac0b2660 2023-11-28T13:01:23.884 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:23.885 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:10.053012+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 -- 172.21.15.6:0/4242589658 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6900682f0 msgr2=0x7ff69006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6900682f0 0x7ff69006a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0b6190 tx=0x7ff69c010000 comp rx=0 tx=0).stop 2023-11-28T13:01:23.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 -- 172.21.15.6:0/4242589658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0b2660 msgr2=0x7ff6ac1f1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0b2660 0x7ff6ac1f1430 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800d800 tx=0x7ff6a800dbc0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 -- 172.21.15.6:0/4242589658 shutdown_connections 2023-11-28T13:01:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6ac15a6c0 0x7ff6ac0b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff6ac0b2660 0x7ff6ac1f1430 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6900682f0 0x7ff69006a7a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 --2- 172.21.15.6:0/4242589658 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff6ac0d0020 0x7ff6ac15b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 -- 172.21.15.6:0/4242589658 >> 172.21.15.6:0/4242589658 conn(0x7ff6ac0cd660 msgr2=0x7ff6ac0b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 -- 172.21.15.6:0/4242589658 shutdown_connections 2023-11-28T13:01:23.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:23.885+0000 7ff6b3793700 1 -- 172.21.15.6:0/4242589658 wait complete. 2023-11-28T13:01:23.899 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:23.899 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:24.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.026+0000 7f1a45592700 1 Processor -- start 2023-11-28T13:01:24.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.026+0000 7f1a45592700 1 -- start start 2023-11-28T13:01:24.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a45592700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a40159500 0x7f1a400cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a45592700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1a4015dbf0 0x7f1a400ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a45592700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1a400b1630 0x7f1a400b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a45592700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a400cede0 con 0x7f1a400b1630 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a45592700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a400b1ff0 con 0x7f1a4015dbf0 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a45592700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4015e040 con 0x7f1a40159500 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a3f7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1a400b1630 0x7f1a400b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a3f7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1a400b1630 0x7f1a400b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59774/0 (socket says 172.21.15.6:59774) 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a3f7fe700 1 -- 172.21.15.6:0/961928732 learned_addr learned my addr 172.21.15.6:0/961928732 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a3e7fc700 1 --2- 172.21.15.6:0/961928732 >> v2:172.21.15.61:3300/0 conn(0x7f1a4015dbf0 0x7f1a400ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.027+0000 7f1a3effd700 1 --2- 172.21.15.6:0/961928732 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a40159500 0x7f1a400cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3f7fe700 1 -- 172.21.15.6:0/961928732 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a40159500 msgr2=0x7f1a400cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3f7fe700 1 --2- 172.21.15.6:0/961928732 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a40159500 0x7f1a400cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3f7fe700 1 -- 172.21.15.6:0/961928732 >> v2:172.21.15.61:3300/0 conn(0x7f1a4015dbf0 msgr2=0x7f1a400ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3f7fe700 1 --2- 172.21.15.6:0/961928732 >> v2:172.21.15.61:3300/0 conn(0x7f1a4015dbf0 0x7f1a400ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3f7fe700 1 -- 172.21.15.6:0/961928732 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400ceff0 con 0x7f1a400b1630 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3e7fc700 1 --2- 172.21.15.6:0/961928732 >> v2:172.21.15.61:3300/0 conn(0x7f1a4015dbf0 0x7f1a400ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3effd700 1 --2- 172.21.15.6:0/961928732 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a40159500 0x7f1a400cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3f7fe700 1 --2- 172.21.15.6:0/961928732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a400b1630 0x7f1a400b1ab0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1a400714f0 tx=0x7f1a3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6a69b3a0dc7777 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3dffb700 1 -- 172.21.15.6:0/961928732 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800e070 con 0x7f1a400b1630 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3dffb700 1 -- 172.21.15.6:0/961928732 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3800b9c0 con 0x7f1a400b1630 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a3dffb700 1 -- 172.21.15.6:0/961928732 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3800bd70 con 0x7f1a400b1630 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a45592700 1 -- 172.21.15.6:0/961928732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a400b1630 msgr2=0x7f1a400b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a45592700 1 --2- 172.21.15.6:0/961928732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a400b1630 0x7f1a400b1ab0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1a400714f0 tx=0x7f1a3800b330 comp rx=0 tx=0).stop 2023-11-28T13:01:24.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a45592700 1 -- 172.21.15.6:0/961928732 shutdown_connections 2023-11-28T13:01:24.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a45592700 1 --2- 172.21.15.6:0/961928732 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a400b1630 0x7f1a400b1ab0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a45592700 1 --2- 172.21.15.6:0/961928732 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a40159500 0x7f1a400cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a45592700 1 --2- 172.21.15.6:0/961928732 >> v2:172.21.15.61:3300/0 conn(0x7f1a4015dbf0 0x7f1a400ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a45592700 1 -- 172.21.15.6:0/961928732 >> 172.21.15.6:0/961928732 conn(0x7f1a400cc630 msgr2=0x7f1a400cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:24.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.028+0000 7f1a45592700 1 -- 172.21.15.6:0/961928732 shutdown_connections 2023-11-28T13:01:24.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.029+0000 7f1a45592700 1 -- 172.21.15.6:0/961928732 wait complete. 2023-11-28T13:01:24.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.030+0000 7f1a45592700 1 Processor -- start 2023-11-28T13:01:24.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.030+0000 7f1a45592700 1 -- start start 2023-11-28T13:01:24.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a45592700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a400b1630 0x7f1a40155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a45592700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a40159500 0x7f1a4014c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a45592700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a4015dbf0 0x7f1a4014cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a45592700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4015b810 con 0x7f1a40159500 2023-11-28T13:01:24.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a45592700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401552f0 con 0x7f1a4015dbf0 2023-11-28T13:01:24.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a45592700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a40154a00 con 0x7f1a400b1630 2023-11-28T13:01:24.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a40159500 0x7f1a4014c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.036 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3e7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a40159500 0x7f1a4014c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59778/0 (socket says 172.21.15.6:59778) 2023-11-28T13:01:24.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3e7fc700 1 -- 172.21.15.6:0/3636748871 learned_addr learned my addr 172.21.15.6:0/3636748871 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:24.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3f7fe700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a4015dbf0 0x7f1a4014cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3effd700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a400b1630 0x7f1a40155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3e7fc700 1 -- 172.21.15.6:0/3636748871 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a400b1630 msgr2=0x7f1a40155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3e7fc700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a400b1630 0x7f1a40155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3e7fc700 1 -- 172.21.15.6:0/3636748871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a4015dbf0 msgr2=0x7f1a4014cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3e7fc700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a4015dbf0 0x7f1a4014cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3e7fc700 1 -- 172.21.15.6:0/3636748871 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a38009b30 con 0x7f1a40159500 2023-11-28T13:01:24.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3f7fe700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a4015dbf0 0x7f1a4014cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:24.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3effd700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a400b1630 0x7f1a40155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a3e7fc700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a40159500 0x7f1a4014c960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000e810 tx=0x7f1a3000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a277fe700 1 -- 172.21.15.6:0/3636748871 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a30004230 con 0x7f1a40159500 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a277fe700 1 -- 172.21.15.6:0/3636748871 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a30004390 con 0x7f1a40159500 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a45592700 1 -- 172.21.15.6:0/3636748871 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4015d110 con 0x7f1a40159500 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a277fe700 1 -- 172.21.15.6:0/3636748871 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a300036a0 con 0x7f1a40159500 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.031+0000 7f1a45592700 1 -- 172.21.15.6:0/3636748871 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a400cf2d0 con 0x7f1a40159500 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.032+0000 7f1a45592700 1 -- 172.21.15.6:0/3636748871 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4015d3f0 con 0x7f1a40159500 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.033+0000 7f1a277fe700 1 -- 172.21.15.6:0/3636748871 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a30022070 con 0x7f1a40159500 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.033+0000 7f1a277fe700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a2c0682f0 0x7f1a2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.033+0000 7f1a277fe700 1 -- 172.21.15.6:0/3636748871 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1a30085be0 con 0x7f1a40159500 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.033+0000 7f1a3effd700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a2c0682f0 0x7f1a2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.034+0000 7f1a3effd700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a2c0682f0 0x7f1a2c06a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a400c6700 tx=0x7f1a34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:24.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.038+0000 7f1a277fe700 1 -- 172.21.15.6:0/3636748871 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a30051fc0 con 0x7f1a40159500 2023-11-28T13:01:24.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.228+0000 7f1a45592700 1 -- 172.21.15.6:0/3636748871 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1a401550a0 con 0x7f1a40159500 2023-11-28T13:01:24.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.228+0000 7f1a277fe700 1 -- 172.21.15.6:0/3636748871 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1a300555e0 con 0x7f1a40159500 2023-11-28T13:01:24.230 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 -- 172.21.15.6:0/3636748871 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a2c0682f0 msgr2=0x7f1a2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a2c0682f0 0x7f1a2c06a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a400c6700 tx=0x7f1a34008040 comp rx=0 tx=0).stop 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 -- 172.21.15.6:0/3636748871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a40159500 msgr2=0x7f1a4014c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a40159500 0x7f1a4014c960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000e810 tx=0x7f1a3000ebd0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 -- 172.21.15.6:0/3636748871 shutdown_connections 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a4015dbf0 0x7f1a4014cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a2c0682f0 0x7f1a2c06a7a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a400b1630 0x7f1a40155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 --2- 172.21.15.6:0/3636748871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a40159500 0x7f1a4014c960 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 -- 172.21.15.6:0/3636748871 >> 172.21.15.6:0/3636748871 conn(0x7f1a400cc630 msgr2=0x7f1a400affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:24.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 -- 172.21.15.6:0/3636748871 shutdown_connections 2023-11-28T13:01:24.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.230+0000 7f1a45592700 1 -- 172.21.15.6:0/3636748871 wait complete. 2023-11-28T13:01:24.244 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:24.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.370+0000 7f80f9651700 1 Processor -- start 2023-11-28T13:01:24.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.371+0000 7f80f9651700 1 -- start start 2023-11-28T13:01:24.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.371+0000 7f80f9651700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f40cf5f0 0x7f80f40caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.371+0000 7f80f9651700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f80f40cb3e0 0x7f80f406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.371+0000 7f80f9651700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f40ce550 con 0x7f80f40b1630 2023-11-28T13:01:24.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.371+0000 7f80f9651700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f40cfa60 con 0x7f80f40cb3e0 2023-11-28T13:01:24.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.371+0000 7f80f9651700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40ce830 con 0x7f80f40cf5f0 2023-11-28T13:01:24.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.371+0000 7f80f2ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f80f40b1630 0x7f80f40b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47104/0 (socket says 172.21.15.6:47104) 2023-11-28T13:01:24.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.371+0000 7f80f2ffd700 1 -- 172.21.15.6:0/2693593233 learned_addr learned my addr 172.21.15.6:0/2693593233 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:24.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f27fc700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f40cf5f0 0x7f80f40caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f37fe700 1 --2- 172.21.15.6:0/2693593233 >> v2:172.21.15.61:3300/0 conn(0x7f80f40cb3e0 0x7f80f406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025279415 0 0) 0x7f80f40ce550 con 0x7f80f40b1630 2023-11-28T13:01:24.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80dc0035b0 con 0x7f80f40b1630 2023-11-28T13:01:24.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1623174477 0 0) 0x7f80dc0035b0 con 0x7f80f40b1630 2023-11-28T13:01:24.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f40cf5f0 msgr2=0x7f80f40caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:24.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f40cf5f0 0x7f80f40caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 >> v2:172.21.15.61:3300/0 conn(0x7f80f40cb3e0 msgr2=0x7f80f406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:24.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 --2- 172.21.15.6:0/2693593233 >> v2:172.21.15.61:3300/0 conn(0x7f80f40cb3e0 0x7f80f406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f40ceb10 con 0x7f80f40b1630 2023-11-28T13:01:24.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f80e8001670 con 0x7f80f40b1630 2023-11-28T13:01:24.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 >> v1:172.21.15.6:6789/0 conn(0x7f80f40b1630 legacy=0x7f80f40b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80dc003a00 0x7f80dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f40cf5f0 0x7f80dc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80dc004d90 0x7f80dc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40ce830 con 0x7f80dc004d90 2023-11-28T13:01:24.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc00ba10 con 0x7f80dc003a00 2023-11-28T13:01:24.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc00bb50 con 0x7f80f40cf5f0 2023-11-28T13:01:24.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f37fe700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80dc004d90 0x7f80dc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f2ffd700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80dc003a00 0x7f80dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.372+0000 7f80f27fc700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f40cf5f0 0x7f80dc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f37fe700 1 -- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f40cf5f0 msgr2=0x7f80dc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:24.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f37fe700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f40cf5f0 0x7f80dc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f37fe700 1 -- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80dc003a00 msgr2=0x7f80dc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:24.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f37fe700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80dc003a00 0x7f80dc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f37fe700 1 -- 172.21.15.6:0/2693593233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc0035b0 con 0x7f80dc004d90 2023-11-28T13:01:24.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f37fe700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80dc004d90 0x7f80dc005200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009fc0 tx=0x7f80dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ce9be4ef810462 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:24.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec00ebd0 con 0x7f80dc004d90 2023-11-28T13:01:24.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ec00f040 con 0x7f80dc004d90 2023-11-28T13:01:24.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f1ffb700 1 -- 172.21.15.6:0/2693593233 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec014480 con 0x7f80dc004d90 2023-11-28T13:01:24.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f9651700 1 -- 172.21.15.6:0/2693593233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80dc004d90 msgr2=0x7f80dc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f9651700 1 --2- 172.21.15.6:0/2693593233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80dc004d90 0x7f80dc005200 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009fc0 tx=0x7f80dc005880 comp rx=0 tx=0).stop 2023-11-28T13:01:24.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.373+0000 7f80f37fe700 1 -- 172.21.15.6:0/2693593233 reap_dead start 2023-11-28T13:01:24.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.374+0000 7f80f9651700 1 -- 172.21.15.6:0/2693593233 shutdown_connections 2023-11-28T13:01:24.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.374+0000 7f80f9651700 1 -- 172.21.15.6:0/2693593233 >> 172.21.15.6:0/2693593233 conn(0x7f80f40cc630 msgr2=0x7f80f40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:24.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.374+0000 7f80f9651700 1 -- 172.21.15.6:0/2693593233 shutdown_connections 2023-11-28T13:01:24.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.374+0000 7f80f9651700 1 -- 172.21.15.6:0/2693593233 wait complete. 2023-11-28T13:01:24.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.375+0000 7f80f9651700 1 Processor -- start 2023-11-28T13:01:24.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.375+0000 7f80f9651700 1 -- start start 2023-11-28T13:01:24.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f9651700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80f415ae60 0x7f80f40c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f9651700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80f415b230 0x7f80f40c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f9651700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f406bf00 0x7f80f406c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f9651700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40c52a0 con 0x7f80f415b230 2023-11-28T13:01:24.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f9651700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f406c8c0 con 0x7f80f415ae60 2023-11-28T13:01:24.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f9651700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40c8950 con 0x7f80f406bf00 2023-11-28T13:01:24.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80f415b230 0x7f80f40c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80f415b230 0x7f80f40c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59798/0 (socket says 172.21.15.6:59798) 2023-11-28T13:01:24.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f27fc700 1 -- 172.21.15.6:0/3566766155 learned_addr learned my addr 172.21.15.6:0/3566766155 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:24.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f2ffd700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80f415ae60 0x7f80f40c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f37fe700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f406bf00 0x7f80f406c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f27fc700 1 -- 172.21.15.6:0/3566766155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f406bf00 msgr2=0x7f80f406c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f27fc700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f406bf00 0x7f80f406c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f27fc700 1 -- 172.21.15.6:0/3566766155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80f415ae60 msgr2=0x7f80f40c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f27fc700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80f415ae60 0x7f80f40c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f27fc700 1 -- 172.21.15.6:0/3566766155 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f40cf0d0 con 0x7f80f415b230 2023-11-28T13:01:24.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80f27fc700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80f415b230 0x7f80f40c8410 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f80e4011aa0 tx=0x7f80e4011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:24.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80db7fe700 1 -- 172.21.15.6:0/3566766155 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4011ee0 con 0x7f80f415b230 2023-11-28T13:01:24.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.376+0000 7f80db7fe700 1 -- 172.21.15.6:0/3566766155 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e400f940 con 0x7f80f415b230 2023-11-28T13:01:24.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.377+0000 7f80f9651700 1 -- 172.21.15.6:0/3566766155 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f40cedf0 con 0x7f80f415b230 2023-11-28T13:01:24.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.377+0000 7f80db7fe700 1 -- 172.21.15.6:0/3566766155 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e400fcf0 con 0x7f80f415b230 2023-11-28T13:01:24.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.377+0000 7f80f9651700 1 -- 172.21.15.6:0/3566766155 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f40ce140 con 0x7f80f415b230 2023-11-28T13:01:24.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.377+0000 7f80f9651700 1 -- 172.21.15.6:0/3566766155 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f40cb1d0 con 0x7f80f415b230 2023-11-28T13:01:24.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.378+0000 7f80db7fe700 1 -- 172.21.15.6:0/3566766155 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f80e4014440 con 0x7f80f415b230 2023-11-28T13:01:24.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.378+0000 7f80db7fe700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80e00682f0 0x7f80e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.378+0000 7f80db7fe700 1 -- 172.21.15.6:0/3566766155 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f80e40866d0 con 0x7f80f415b230 2023-11-28T13:01:24.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.378+0000 7f80f2ffd700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80e00682f0 0x7f80e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.379+0000 7f80f2ffd700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80e00682f0 0x7f80e006a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80e8001cf0 tx=0x7f80e8007980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:24.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.383+0000 7f80db7fe700 1 -- 172.21.15.6:0/3566766155 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f80e4052c60 con 0x7f80f415b230 2023-11-28T13:01:24.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.574+0000 7f80f9651700 1 -- 172.21.15.6:0/3566766155 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80f40739f0 con 0x7f80f415b230 2023-11-28T13:01:24.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.575+0000 7f80db7fe700 1 -- 172.21.15.6:0/3566766155 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80e4022020 con 0x7f80f415b230 2023-11-28T13:01:24.589 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:24.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.576+0000 7f80f9651700 1 -- 172.21.15.6:0/3566766155 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80e00682f0 msgr2=0x7f80e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.576+0000 7f80f9651700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80e00682f0 0x7f80e006a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80e8001cf0 tx=0x7f80e8007980 comp rx=0 tx=0).stop 2023-11-28T13:01:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.576+0000 7f80f9651700 1 -- 172.21.15.6:0/3566766155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80f415b230 msgr2=0x7f80f40c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.576+0000 7f80f9651700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80f415b230 0x7f80f40c8410 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f80e4011aa0 tx=0x7f80e4011db0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.577+0000 7f80f9651700 1 -- 172.21.15.6:0/3566766155 shutdown_connections 2023-11-28T13:01:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.577+0000 7f80f9651700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80e00682f0 0x7f80e006a7a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.577+0000 7f80f9651700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80f406bf00 0x7f80f406c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.577+0000 7f80f9651700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80f415ae60 0x7f80f40c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.577+0000 7f80f9651700 1 --2- 172.21.15.6:0/3566766155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80f415b230 0x7f80f40c8410 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.577+0000 7f80f9651700 1 -- 172.21.15.6:0/3566766155 >> 172.21.15.6:0/3566766155 conn(0x7f80f40cc630 msgr2=0x7f80f415c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:24.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.577+0000 7f80f9651700 1 -- 172.21.15.6:0/3566766155 shutdown_connections 2023-11-28T13:01:24.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.577+0000 7f80f9651700 1 -- 172.21.15.6:0/3566766155 wait complete. 2023-11-28T13:01:24.592 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:24.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.720+0000 7f3120096700 1 Processor -- start 2023-11-28T13:01:24.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.720+0000 7f3120096700 1 -- start start 2023-11-28T13:01:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f3120096700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f311815fc00 0x7f311806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f3120096700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f31180b2d90 0x7f31180cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f3120096700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311806f940 con 0x7f311806f3e0 2023-11-28T13:01:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f3120096700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311806fc20 con 0x7f31180b2d90 2023-11-28T13:01:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f3120096700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31180cb3f0 con 0x7f311815fc00 2023-11-28T13:01:24.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f311e633700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f31180b2d90 0x7f31180cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f311d631700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f311806f3e0 0x7f31180b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47124/0 (socket says 172.21.15.6:47124) 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f311d631700 1 -- 172.21.15.6:0/1846796397 learned_addr learned my addr 172.21.15.6:0/1846796397 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f311de32700 1 --2- 172.21.15.6:0/1846796397 >> v2:172.21.15.61:3300/0 conn(0x7f311815fc00 0x7f311806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f311e633700 1 -- 172.21.15.6:0/1846796397 >> v2:172.21.15.61:3300/0 conn(0x7f311815fc00 msgr2=0x7f311806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f311e633700 1 --2- 172.21.15.6:0/1846796397 >> v2:172.21.15.61:3300/0 conn(0x7f311815fc00 0x7f311806eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.721+0000 7f311e633700 1 -- 172.21.15.6:0/1846796397 >> v1:172.21.15.6:6789/0 conn(0x7f311806f3e0 legacy=0x7f31180b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f311e633700 1 -- 172.21.15.6:0/1846796397 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31180cb6d0 con 0x7f31180b2d90 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f311ce30700 1 -- 172.21.15.6:0/1846796397 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861748813 0 0) 0x7f311806f940 con 0x7f311806f3e0 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f311de32700 1 --2- 172.21.15.6:0/1846796397 >> v2:172.21.15.61:3300/0 conn(0x7f311815fc00 0x7f311806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f311e633700 1 --2- 172.21.15.6:0/1846796397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31180b2d90 0x7f31180cbbf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f31180d0b60 tx=0x7f311000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f931f34c9807c8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:24.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f311ce30700 1 -- 172.21.15.6:0/1846796397 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311000e070 con 0x7f31180b2d90 2023-11-28T13:01:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f311ce30700 1 -- 172.21.15.6:0/1846796397 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311000bbd0 con 0x7f31180b2d90 2023-11-28T13:01:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f311ce30700 1 -- 172.21.15.6:0/1846796397 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3110012430 con 0x7f31180b2d90 2023-11-28T13:01:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f3120096700 1 -- 172.21.15.6:0/1846796397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31180b2d90 msgr2=0x7f31180cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f3120096700 1 --2- 172.21.15.6:0/1846796397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31180b2d90 0x7f31180cbbf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f31180d0b60 tx=0x7f311000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f3120096700 1 -- 172.21.15.6:0/1846796397 shutdown_connections 2023-11-28T13:01:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f3120096700 1 --2- 172.21.15.6:0/1846796397 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31180b2d90 0x7f31180cbbf0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f3120096700 1 --2- 172.21.15.6:0/1846796397 >> v2:172.21.15.61:3300/0 conn(0x7f311815fc00 0x7f311806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.722+0000 7f3120096700 1 -- 172.21.15.6:0/1846796397 >> 172.21.15.6:0/1846796397 conn(0x7f31180cd680 msgr2=0x7f31180cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.723+0000 7f3120096700 1 -- 172.21.15.6:0/1846796397 shutdown_connections 2023-11-28T13:01:24.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.723+0000 7f3120096700 1 -- 172.21.15.6:0/1846796397 wait complete. 2023-11-28T13:01:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.724+0000 7f3120096700 1 Processor -- start 2023-11-28T13:01:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.724+0000 7f3120096700 1 -- start start 2023-11-28T13:01:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.724+0000 7f3120096700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f311806f3e0 0x7f31180c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.724+0000 7f3120096700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31180b2d90 0x7f31180c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.724+0000 7f3120096700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f311815fc00 0x7f3118158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.724+0000 7f3120096700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31180b41e0 con 0x7f311815fc00 2023-11-28T13:01:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.724+0000 7f3120096700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31180b3bb0 con 0x7f311806f3e0 2023-11-28T13:01:24.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.724+0000 7f3120096700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31180c56e0 con 0x7f31180b2d90 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311e633700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f311815fc00 0x7f3118158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311e633700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f311815fc00 0x7f3118158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59814/0 (socket says 172.21.15.6:59814) 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311e633700 1 -- 172.21.15.6:0/2255243442 learned_addr learned my addr 172.21.15.6:0/2255243442 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311de32700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f311806f3e0 0x7f31180c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311d631700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31180b2d90 0x7f31180c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311e633700 1 -- 172.21.15.6:0/2255243442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31180b2d90 msgr2=0x7f31180c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311e633700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31180b2d90 0x7f31180c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311e633700 1 -- 172.21.15.6:0/2255243442 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f311806f3e0 msgr2=0x7f31180c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311e633700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f311806f3e0 0x7f31180c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311e633700 1 -- 172.21.15.6:0/2255243442 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3110009a70 con 0x7f311815fc00 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311de32700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f311806f3e0 0x7f31180c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:24.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f311e633700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f311815fc00 0x7f3118158aa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3118066a20 tx=0x7f3110004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f310e7fc700 1 -- 172.21.15.6:0/2255243442 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3110012430 con 0x7f311815fc00 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f310e7fc700 1 -- 172.21.15.6:0/2255243442 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3110020070 con 0x7f311815fc00 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f3120096700 1 -- 172.21.15.6:0/2255243442 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31180ce340 con 0x7f311815fc00 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f310e7fc700 1 -- 172.21.15.6:0/2255243442 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3110004280 con 0x7f311815fc00 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.725+0000 7f3120096700 1 -- 172.21.15.6:0/2255243442 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311815ffd0 con 0x7f311815fc00 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.726+0000 7f3120096700 1 -- 172.21.15.6:0/2255243442 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31180cc130 con 0x7f311815fc00 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.727+0000 7f310e7fc700 1 -- 172.21.15.6:0/2255243442 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f311002e070 con 0x7f311815fc00 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.727+0000 7f310e7fc700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30fc0682f0 0x7f30fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.727+0000 7f310e7fc700 1 -- 172.21.15.6:0/2255243442 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3110022070 con 0x7f311815fc00 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.727+0000 7f311de32700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30fc0682f0 0x7f30fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:24.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.727+0000 7f311de32700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30fc0682f0 0x7f30fc06a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f310800aa20 tx=0x7f3108008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:24.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.731+0000 7f310e7fc700 1 -- 172.21.15.6:0/2255243442 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f31100543a0 con 0x7f311815fc00 2023-11-28T13:01:24.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.922+0000 7f3120096700 1 -- 172.21.15.6:0/2255243442 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31180739f0 con 0x7f311815fc00 2023-11-28T13:01:24.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.923+0000 7f310e7fc700 1 -- 172.21.15.6:0/2255243442 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31100579c0 con 0x7f311815fc00 2023-11-28T13:01:24.924 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:24.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.924+0000 7f3120096700 1 -- 172.21.15.6:0/2255243442 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30fc0682f0 msgr2=0x7f30fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.924+0000 7f3120096700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30fc0682f0 0x7f30fc06a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f310800aa20 tx=0x7f3108008040 comp rx=0 tx=0).stop 2023-11-28T13:01:24.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.924+0000 7f3120096700 1 -- 172.21.15.6:0/2255243442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f311815fc00 msgr2=0x7f3118158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:24.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.924+0000 7f3120096700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f311815fc00 0x7f3118158aa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3118066a20 tx=0x7f3110004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.925+0000 7f3120096700 1 -- 172.21.15.6:0/2255243442 shutdown_connections 2023-11-28T13:01:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.925+0000 7f3120096700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f311815fc00 0x7f3118158aa0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.925+0000 7f3120096700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f311806f3e0 0x7f31180c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.925+0000 7f3120096700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30fc0682f0 0x7f30fc06a7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.925+0000 7f3120096700 1 --2- 172.21.15.6:0/2255243442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f31180b2d90 0x7f31180c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.925+0000 7f3120096700 1 -- 172.21.15.6:0/2255243442 >> 172.21.15.6:0/2255243442 conn(0x7f31180cd680 msgr2=0x7f311815bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.925+0000 7f3120096700 1 -- 172.21.15.6:0/2255243442 shutdown_connections 2023-11-28T13:01:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:24.925+0000 7f3120096700 1 -- 172.21.15.6:0/2255243442 wait complete. 2023-11-28T13:01:24.938 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:25.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.066+0000 7fc5ade23700 1 Processor -- start 2023-11-28T13:01:25.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.066+0000 7fc5ade23700 1 -- start start 2023-11-28T13:01:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5ade23700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cf5f0 0x7fc5a80caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5ade23700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc5a80cb3e0 0x7fc5a806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5ade23700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80ce550 con 0x7fc5a80b1630 2023-11-28T13:01:25.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5ade23700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80cfa60 con 0x7fc5a80cb3e0 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5ade23700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80ce830 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a7fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc5a80cb3e0 0x7fc5a806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a77fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fc5a80b1630 0x7fc5a80b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47138/0 (socket says 172.21.15.6:47138) 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a77fe700 1 -- 172.21.15.6:0/1889550232 learned_addr learned my addr 172.21.15.6:0/1889550232 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a7fff700 1 --2- 172.21.15.6:0/1889550232 >> v2:172.21.15.6:3300/0 conn(0x7fc5a80cb3e0 0x7fc5a806afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59822/0 (socket says 172.21.15.6:59822) 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a6ffd700 1 --2- 172.21.15.6:0/1889550232 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cf5f0 0x7fc5a80caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a7fff700 1 -- 172.21.15.6:0/1889550232 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cf5f0 msgr2=0x7fc5a80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a7fff700 1 --2- 172.21.15.6:0/1889550232 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cf5f0 0x7fc5a80caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a7fff700 1 -- 172.21.15.6:0/1889550232 >> v1:172.21.15.6:6789/0 conn(0x7fc5a80b1630 legacy=0x7fc5a80b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a67fc700 1 -- 172.21.15.6:0/1889550232 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430764231 0 0) 0x7fc5a80ce550 con 0x7fc5a80b1630 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a7fff700 1 -- 172.21.15.6:0/1889550232 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a80ceb10 con 0x7fc5a80cb3e0 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.067+0000 7fc5a6ffd700 1 --2- 172.21.15.6:0/1889550232 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cf5f0 0x7fc5a80caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:25.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5a7fff700 1 --2- 172.21.15.6:0/1889550232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cb3e0 0x7fc5a806afc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009d90 tx=0x7fc5a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6c20a349f7ba2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5a67fc700 1 -- 172.21.15.6:0/1889550232 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000e070 con 0x7fc5a80cb3e0 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5a67fc700 1 -- 172.21.15.6:0/1889550232 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a000bbd0 con 0x7fc5a80cb3e0 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5a67fc700 1 -- 172.21.15.6:0/1889550232 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0012430 con 0x7fc5a80cb3e0 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5ade23700 1 -- 172.21.15.6:0/1889550232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cb3e0 msgr2=0x7fc5a806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5ade23700 1 --2- 172.21.15.6:0/1889550232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cb3e0 0x7fc5a806afc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009d90 tx=0x7fc5a000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5ade23700 1 -- 172.21.15.6:0/1889550232 shutdown_connections 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5ade23700 1 --2- 172.21.15.6:0/1889550232 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cb3e0 0x7fc5a806afc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5ade23700 1 --2- 172.21.15.6:0/1889550232 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cf5f0 0x7fc5a80caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5ade23700 1 -- 172.21.15.6:0/1889550232 >> 172.21.15.6:0/1889550232 conn(0x7fc5a80cc630 msgr2=0x7fc5a80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5ade23700 1 -- 172.21.15.6:0/1889550232 shutdown_connections 2023-11-28T13:01:25.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.068+0000 7fc5ade23700 1 -- 172.21.15.6:0/1889550232 wait complete. 2023-11-28T13:01:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.070+0000 7fc5ade23700 1 Processor -- start 2023-11-28T13:01:25.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.070+0000 7fc5ade23700 1 -- start start 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5ade23700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a80b1630 0x7fc5a81f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5ade23700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cb3e0 0x7fc5a80c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5ade23700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cf5f0 0x7fc5a815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5ade23700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a815b530 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5ade23700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80c9fa0 con 0x7fc5a80b1630 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5ade23700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80c9ce0 con 0x7fc5a80cb3e0 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cf5f0 0x7fc5a815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cf5f0 0x7fc5a815a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59828/0 (socket says 172.21.15.6:59828) 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a7fff700 1 -- 172.21.15.6:0/639964220 learned_addr learned my addr 172.21.15.6:0/639964220 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a6ffd700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cb3e0 0x7fc5a80c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a77fe700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a80b1630 0x7fc5a81f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a7fff700 1 -- 172.21.15.6:0/639964220 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cb3e0 msgr2=0x7fc5a80c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a7fff700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cb3e0 0x7fc5a80c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a7fff700 1 -- 172.21.15.6:0/639964220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a80b1630 msgr2=0x7fc5a81f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a7fff700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a80b1630 0x7fc5a81f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a7fff700 1 -- 172.21.15.6:0/639964220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0009a70 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5a7fff700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cf5f0 0x7fc5a815a510 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0014040 tx=0x7fc5a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc59ffff700 1 -- 172.21.15.6:0/639964220 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0012430 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc59ffff700 1 -- 172.21.15.6:0/639964220 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0020070 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc59ffff700 1 -- 172.21.15.6:0/639964220 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0004280 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.071+0000 7fc5ade23700 1 -- 172.21.15.6:0/639964220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a80cf0d0 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.072+0000 7fc5ade23700 1 -- 172.21.15.6:0/639964220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a80cedf0 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.072+0000 7fc5ade23700 1 -- 172.21.15.6:0/639964220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a80ce140 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.073+0000 7fc59ffff700 1 -- 172.21.15.6:0/639964220 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc5a002e070 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.073+0000 7fc59ffff700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5880682f0 0x7fc58806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.073+0000 7fc59ffff700 1 -- 172.21.15.6:0/639964220 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc5a0022070 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.073+0000 7fc5a77fe700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5880682f0 0x7fc58806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.074+0000 7fc5a77fe700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5880682f0 0x7fc58806a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc598007fd0 tx=0x7fc598007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:25.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.078+0000 7fc59ffff700 1 -- 172.21.15.6:0/639964220 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc5a0054330 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.274+0000 7fc5ade23700 1 -- 172.21.15.6:0/639964220 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5a80c3380 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.274+0000 7fc59ffff700 1 -- 172.21.15.6:0/639964220 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5a002c0a0 con 0x7fc5a80cf5f0 2023-11-28T13:01:25.276 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:25.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.276+0000 7fc5ade23700 1 -- 172.21.15.6:0/639964220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5880682f0 msgr2=0x7fc58806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.276+0000 7fc5ade23700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5880682f0 0x7fc58806a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc598007fd0 tx=0x7fc598007de0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.276+0000 7fc5ade23700 1 -- 172.21.15.6:0/639964220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cf5f0 msgr2=0x7fc5a815a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.276+0000 7fc5ade23700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cf5f0 0x7fc5a815a510 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0014040 tx=0x7fc5a0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.276+0000 7fc5ade23700 1 -- 172.21.15.6:0/639964220 shutdown_connections 2023-11-28T13:01:25.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.276+0000 7fc5ade23700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc5a80cf5f0 0x7fc5a815a510 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.276+0000 7fc5ade23700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc5a80b1630 0x7fc5a81f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.276+0000 7fc5ade23700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc5880682f0 0x7fc58806a7a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.276+0000 7fc5ade23700 1 --2- 172.21.15.6:0/639964220 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc5a80cb3e0 0x7fc5a80c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.276+0000 7fc5ade23700 1 -- 172.21.15.6:0/639964220 >> 172.21.15.6:0/639964220 conn(0x7fc5a80cc630 msgr2=0x7fc5a80b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.277+0000 7fc5ade23700 1 -- 172.21.15.6:0/639964220 shutdown_connections 2023-11-28T13:01:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.277+0000 7fc5ade23700 1 -- 172.21.15.6:0/639964220 wait complete. 2023-11-28T13:01:25.290 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:25.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.416+0000 7fec2c53c700 1 Processor -- start 2023-11-28T13:01:25.418 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.416+0000 7fec2c53c700 1 -- start start 2023-11-28T13:01:25.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.417+0000 7fec2c53c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec240b62d0 0x7fec240cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.417+0000 7fec2c53c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fec2406f8b0 0x7fec240b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.417+0000 7fec2c53c700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec240b3020 con 0x7fec240b2bc0 2023-11-28T13:01:25.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.417+0000 7fec2c53c700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240cd2a0 con 0x7fec2406f8b0 2023-11-28T13:01:25.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.417+0000 7fec2c53c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec240d0400 con 0x7fec240b62d0 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.417+0000 7fec29ad7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fec2406f8b0 0x7fec240b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.417+0000 7fec2aad9700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fec240b2bc0 0x7fec240ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47156/0 (socket says 172.21.15.6:47156) 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.417+0000 7fec2aad9700 1 -- 172.21.15.6:0/3005799056 learned_addr learned my addr 172.21.15.6:0/3005799056 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.418+0000 7fec2a2d8700 1 --2- 172.21.15.6:0/3005799056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec240b62d0 0x7fec240cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.418+0000 7fec29ad7700 1 -- 172.21.15.6:0/3005799056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec240b62d0 msgr2=0x7fec240cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.418+0000 7fec29ad7700 1 --2- 172.21.15.6:0/3005799056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec240b62d0 0x7fec240cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.418+0000 7fec29ad7700 1 -- 172.21.15.6:0/3005799056 >> v1:172.21.15.6:6789/0 conn(0x7fec240b2bc0 legacy=0x7fec240ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.418+0000 7fec292d6700 1 -- 172.21.15.6:0/3005799056 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386805010 0 0) 0x7fec240b3020 con 0x7fec240b2bc0 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.418+0000 7fec29ad7700 1 -- 172.21.15.6:0/3005799056 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240d06e0 con 0x7fec2406f8b0 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.418+0000 7fec29ad7700 1 --2- 172.21.15.6:0/3005799056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec2406f8b0 0x7fec240b2680 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec14009840 tx=0x7fec14009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a9cb07effa2136 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.418+0000 7fec292d6700 1 -- 172.21.15.6:0/3005799056 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14011830 con 0x7fec2406f8b0 2023-11-28T13:01:25.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.418+0000 7fec292d6700 1 -- 172.21.15.6:0/3005799056 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec14011990 con 0x7fec2406f8b0 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.418+0000 7fec292d6700 1 -- 172.21.15.6:0/3005799056 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14011d20 con 0x7fec2406f8b0 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.419+0000 7fec2c53c700 1 -- 172.21.15.6:0/3005799056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec2406f8b0 msgr2=0x7fec240b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.419+0000 7fec2c53c700 1 --2- 172.21.15.6:0/3005799056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec2406f8b0 0x7fec240b2680 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec14009840 tx=0x7fec14009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.419+0000 7fec2c53c700 1 -- 172.21.15.6:0/3005799056 shutdown_connections 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.419+0000 7fec2c53c700 1 --2- 172.21.15.6:0/3005799056 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec240b62d0 0x7fec240cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.419+0000 7fec2c53c700 1 --2- 172.21.15.6:0/3005799056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec2406f8b0 0x7fec240b2680 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.419+0000 7fec2c53c700 1 -- 172.21.15.6:0/3005799056 >> 172.21.15.6:0/3005799056 conn(0x7fec240cd680 msgr2=0x7fec240cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.419+0000 7fec2c53c700 1 -- 172.21.15.6:0/3005799056 shutdown_connections 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.419+0000 7fec2c53c700 1 -- 172.21.15.6:0/3005799056 wait complete. 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.421+0000 7fec2c53c700 1 Processor -- start 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.421+0000 7fec2c53c700 1 -- start start 2023-11-28T13:01:25.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.421+0000 7fec2c53c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec2406f8b0 0x7fec24156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec2c53c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec240b2bc0 0x7fec240c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec2c53c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec240b62d0 0x7fec240c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec2c53c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec240cad00 con 0x7fec240b2bc0 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec2c53c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec240c9cc0 con 0x7fec240b62d0 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec2c53c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec240c9f40 con 0x7fec2406f8b0 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec29ad7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec240b2bc0 0x7fec240c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec29ad7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec240b2bc0 0x7fec240c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59840/0 (socket says 172.21.15.6:59840) 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec29ad7700 1 -- 172.21.15.6:0/1253984878 learned_addr learned my addr 172.21.15.6:0/1253984878 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec2a2d8700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec2406f8b0 0x7fec24156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec2aad9700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec240b62d0 0x7fec240c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec29ad7700 1 -- 172.21.15.6:0/1253984878 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec2406f8b0 msgr2=0x7fec24156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec29ad7700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec2406f8b0 0x7fec24156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec29ad7700 1 -- 172.21.15.6:0/1253984878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec240b62d0 msgr2=0x7fec240c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec29ad7700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec240b62d0 0x7fec240c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec29ad7700 1 -- 172.21.15.6:0/1253984878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec140094a0 con 0x7fec240b2bc0 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec2aad9700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec240b62d0 0x7fec240c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec29ad7700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec240b2bc0 0x7fec240c7cd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fec140090b0 tx=0x7fec14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec1affd700 1 -- 172.21.15.6:0/1253984878 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec140036a0 con 0x7fec240b2bc0 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec1affd700 1 -- 172.21.15.6:0/1253984878 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1401f070 con 0x7fec240b2bc0 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec2c53c700 1 -- 172.21.15.6:0/1253984878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec240b66a0 con 0x7fec240b2bc0 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec1affd700 1 -- 172.21.15.6:0/1253984878 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14003a00 con 0x7fec240b2bc0 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.422+0000 7fec2c53c700 1 -- 172.21.15.6:0/1253984878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec240cffb0 con 0x7fec240b2bc0 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.424+0000 7fec2c53c700 1 -- 172.21.15.6:0/1253984878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2406fd20 con 0x7fec240b2bc0 2023-11-28T13:01:25.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.424+0000 7fec1affd700 1 -- 172.21.15.6:0/1253984878 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec1400f050 con 0x7fec240b2bc0 2023-11-28T13:01:25.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.424+0000 7fec1affd700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fec080682f0 0x7fec0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.424+0000 7fec1affd700 1 -- 172.21.15.6:0/1253984878 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fec14021070 con 0x7fec240b2bc0 2023-11-28T13:01:25.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.424+0000 7fec2a2d8700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fec080682f0 0x7fec0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.425+0000 7fec2a2d8700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fec080682f0 0x7fec0806a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec20006b20 tx=0x7fec20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:25.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.429+0000 7fec1affd700 1 -- 172.21.15.6:0/1253984878 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fec14053f10 con 0x7fec240b2bc0 2023-11-28T13:01:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.622+0000 7fec2c53c700 1 -- 172.21.15.6:0/1253984878 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fec240739f0 con 0x7fec240b2bc0 2023-11-28T13:01:25.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.623+0000 7fec1affd700 1 -- 172.21.15.6:0/1253984878 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fec14024e90 con 0x7fec240b2bc0 2023-11-28T13:01:25.624 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:25.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 -- 172.21.15.6:0/1253984878 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fec080682f0 msgr2=0x7fec0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fec080682f0 0x7fec0806a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec20006b20 tx=0x7fec20008040 comp rx=0 tx=0).stop 2023-11-28T13:01:25.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 -- 172.21.15.6:0/1253984878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec240b2bc0 msgr2=0x7fec240c7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec240b2bc0 0x7fec240c7cd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fec140090b0 tx=0x7fec14004750 comp rx=0 tx=0).stop 2023-11-28T13:01:25.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 -- 172.21.15.6:0/1253984878 shutdown_connections 2023-11-28T13:01:25.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec240b62d0 0x7fec240c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fec080682f0 0x7fec0806a7a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fec2406f8b0 0x7fec24156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 --2- 172.21.15.6:0/1253984878 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fec240b2bc0 0x7fec240c7cd0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 -- 172.21.15.6:0/1253984878 >> 172.21.15.6:0/1253984878 conn(0x7fec240cd680 msgr2=0x7fec240b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:25.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 -- 172.21.15.6:0/1253984878 shutdown_connections 2023-11-28T13:01:25.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.625+0000 7fec2c53c700 1 -- 172.21.15.6:0/1253984878 wait complete. 2023-11-28T13:01:25.639 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:25.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.764+0000 7f67c4d49700 1 Processor -- start 2023-11-28T13:01:25.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.764+0000 7f67c4d49700 1 -- start start 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67c4d49700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f67c015a6c0 0x7f67c00cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67c4d49700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f67c00d0020 0x7f67c00ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67c4d49700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c00cfb90 con 0x7f67c00b2660 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67c4d49700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c00cb930 con 0x7f67c015a6c0 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67c4d49700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c00cf260 con 0x7f67c00d0020 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67be59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f67c015a6c0 0x7f67c00cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67bed9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f67c00b2660 0x7f67c00b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47170/0 (socket says 172.21.15.6:47170) 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67be59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f67c015a6c0 0x7f67c00cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59846/0 (socket says 172.21.15.6:59846) 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67bed9d700 1 -- 172.21.15.6:0/1941783609 learned_addr learned my addr 172.21.15.6:0/1941783609 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67bdd9b700 1 --2- 172.21.15.6:0/1941783609 >> v2:172.21.15.61:3300/0 conn(0x7f67c00d0020 0x7f67c00ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67be59c700 1 -- 172.21.15.6:0/1941783609 >> v2:172.21.15.61:3300/0 conn(0x7f67c00d0020 msgr2=0x7f67c00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67be59c700 1 --2- 172.21.15.6:0/1941783609 >> v2:172.21.15.61:3300/0 conn(0x7f67c00d0020 0x7f67c00ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67be59c700 1 -- 172.21.15.6:0/1941783609 >> v1:172.21.15.6:6789/0 conn(0x7f67c00b2660 legacy=0x7f67c00b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67be59c700 1 -- 172.21.15.6:0/1941783609 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c00cf540 con 0x7f67c015a6c0 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67bdd9b700 1 --2- 172.21.15.6:0/1941783609 >> v2:172.21.15.61:3300/0 conn(0x7f67c00d0020 0x7f67c00ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.765+0000 7f67be59c700 1 --2- 172.21.15.6:0/1941783609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c015a6c0 0x7f67c00cb230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f67c015dff0 tx=0x7f67b0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50deaa8451a0715a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67bd59a700 1 -- 172.21.15.6:0/1941783609 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0011830 con 0x7f67c015a6c0 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67bd59a700 1 -- 172.21.15.6:0/1941783609 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b0011990 con 0x7f67c015a6c0 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67bd59a700 1 -- 172.21.15.6:0/1941783609 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0011d20 con 0x7f67c015a6c0 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67c4d49700 1 -- 172.21.15.6:0/1941783609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c015a6c0 msgr2=0x7f67c00cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67c4d49700 1 --2- 172.21.15.6:0/1941783609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c015a6c0 0x7f67c00cb230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f67c015dff0 tx=0x7f67b0009b20 comp rx=0 tx=0).stop 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67c4d49700 1 -- 172.21.15.6:0/1941783609 shutdown_connections 2023-11-28T13:01:25.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67c4d49700 1 --2- 172.21.15.6:0/1941783609 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c015a6c0 0x7f67c00cb230 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67c4d49700 1 --2- 172.21.15.6:0/1941783609 >> v2:172.21.15.61:3300/0 conn(0x7f67c00d0020 0x7f67c00ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67c4d49700 1 -- 172.21.15.6:0/1941783609 >> 172.21.15.6:0/1941783609 conn(0x7f67c00cd660 msgr2=0x7f67c00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:25.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67c4d49700 1 -- 172.21.15.6:0/1941783609 shutdown_connections 2023-11-28T13:01:25.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.766+0000 7f67c4d49700 1 -- 172.21.15.6:0/1941783609 wait complete. 2023-11-28T13:01:25.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.768+0000 7f67c4d49700 1 Processor -- start 2023-11-28T13:01:25.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.768+0000 7f67c4d49700 1 -- start start 2023-11-28T13:01:25.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.768+0000 7f67c4d49700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67c00b2660 0x7f67c015d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.768+0000 7f67c4d49700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c00d0020 0x7f67c015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.768+0000 7f67c4d49700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67c015a6c0 0x7f67c015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.768+0000 7f67c4d49700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00c9a40 con 0x7f67c00d0020 2023-11-28T13:01:25.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.768+0000 7f67c4d49700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00c9d20 con 0x7f67c015a6c0 2023-11-28T13:01:25.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.768+0000 7f67c4d49700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00ca000 con 0x7f67c00b2660 2023-11-28T13:01:25.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67bdd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c00d0020 0x7f67c015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67bdd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c00d0020 0x7f67c015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59854/0 (socket says 172.21.15.6:59854) 2023-11-28T13:01:25.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67bdd9b700 1 -- 172.21.15.6:0/429112080 learned_addr learned my addr 172.21.15.6:0/429112080 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:25.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67be59c700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67c00b2660 0x7f67c015d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67bed9d700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67c015a6c0 0x7f67c015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67bdd9b700 1 -- 172.21.15.6:0/429112080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67c00b2660 msgr2=0x7f67c015d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67bdd9b700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67c00b2660 0x7f67c015d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67bdd9b700 1 -- 172.21.15.6:0/429112080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67c015a6c0 msgr2=0x7f67c015b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67bdd9b700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67c015a6c0 0x7f67c015b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67bdd9b700 1 -- 172.21.15.6:0/429112080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b00094a0 con 0x7f67c00d0020 2023-11-28T13:01:25.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67bdd9b700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c00d0020 0x7f67c015b380 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f67b400c750 tx=0x7f67b400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:25.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67aeffd700 1 -- 172.21.15.6:0/429112080 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4011800 con 0x7f67c00d0020 2023-11-28T13:01:25.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67aeffd700 1 -- 172.21.15.6:0/429112080 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b4011960 con 0x7f67c00d0020 2023-11-28T13:01:25.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67aeffd700 1 -- 172.21.15.6:0/429112080 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4011cd0 con 0x7f67c00d0020 2023-11-28T13:01:25.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67c4d49700 1 -- 172.21.15.6:0/429112080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c00cd3e0 con 0x7f67c00d0020 2023-11-28T13:01:25.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.769+0000 7f67c4d49700 1 -- 172.21.15.6:0/429112080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c00cf820 con 0x7f67c00d0020 2023-11-28T13:01:25.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.770+0000 7f67c4d49700 1 -- 172.21.15.6:0/429112080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c00d0470 con 0x7f67c00d0020 2023-11-28T13:01:25.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.771+0000 7f67aeffd700 1 -- 172.21.15.6:0/429112080 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f67b4020070 con 0x7f67c00d0020 2023-11-28T13:01:25.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.771+0000 7f67aeffd700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f67a0068320 0x7f67a006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:25.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.771+0000 7f67aeffd700 1 -- 172.21.15.6:0/429112080 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f67b4085e70 con 0x7f67c00d0020 2023-11-28T13:01:25.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.771+0000 7f67be59c700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f67a0068320 0x7f67a006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:25.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.771+0000 7f67be59c700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f67a0068320 0x7f67a006a7d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f67b0000c00 tx=0x7f67b001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:25.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.775+0000 7f67aeffd700 1 -- 172.21.15.6:0/429112080 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f67b4052310 con 0x7f67c00d0020 2023-11-28T13:01:25.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.983+0000 7f67c4d49700 1 -- 172.21.15.6:0/429112080 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f67c00675a0 con 0x7f67c00d0020 2023-11-28T13:01:25.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.984+0000 7f67aeffd700 1 -- 172.21.15.6:0/429112080 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f67b4055930 con 0x7f67c00d0020 2023-11-28T13:01:25.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 -- 172.21.15.6:0/429112080 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f67a0068320 msgr2=0x7f67a006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f67a0068320 0x7f67a006a7d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f67b0000c00 tx=0x7f67b001c040 comp rx=0 tx=0).stop 2023-11-28T13:01:25.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 -- 172.21.15.6:0/429112080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c00d0020 msgr2=0x7f67c015b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:25.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c00d0020 0x7f67c015b380 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f67b400c750 tx=0x7f67b400cb10 comp rx=0 tx=0).stop 2023-11-28T13:01:25.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 -- 172.21.15.6:0/429112080 shutdown_connections 2023-11-28T13:01:25.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67c015a6c0 0x7f67c015b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f67a0068320 0x7f67a006a7d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f67c00b2660 0x7f67c015d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 --2- 172.21.15.6:0/429112080 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f67c00d0020 0x7f67c015b380 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:25.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 -- 172.21.15.6:0/429112080 >> 172.21.15.6:0/429112080 conn(0x7f67c00cd660 msgr2=0x7f67c00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:25.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 -- 172.21.15.6:0/429112080 shutdown_connections 2023-11-28T13:01:25.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:25.986+0000 7f67c4d49700 1 -- 172.21.15.6:0/429112080 wait complete. 2023-11-28T13:01:25.990 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:01:25.999 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi061.front.sepia.ceph.com... 2023-11-28T13:01:25.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:26.000 DEBUG:teuthology.orchestra.run.smithi061:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-28T13:01:26.043 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:26.043 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:01:26.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:26.044 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T13:01:26.104 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T13:01:26.105 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:26.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.229+0000 7fd66bf3d700 1 Processor -- start 2023-11-28T13:01:26.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.229+0000 7fd66bf3d700 1 -- start start 2023-11-28T13:01:26.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66bf3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd66415a6c0 0x7fd6640cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66bf3d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6640b2660 0x7fd6640b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66bf3d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6640cfb90 con 0x7fd6640d0020 2023-11-28T13:01:26.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66bf3d700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6640b3020 con 0x7fd6640b2660 2023-11-28T13:01:26.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66bf3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640cb930 con 0x7fd66415a6c0 2023-11-28T13:01:26.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66a4da700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6640b2660 0x7fd6640b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66a4da700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6640b2660 0x7fd6640b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59864/0 (socket says 172.21.15.6:59864) 2023-11-28T13:01:26.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66a4da700 1 -- 172.21.15.6:0/1701840435 learned_addr learned my addr 172.21.15.6:0/1701840435 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:26.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd669cd9700 1 --2- 172.21.15.6:0/1701840435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd66415a6c0 0x7fd6640cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66a4da700 1 -- 172.21.15.6:0/1701840435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd66415a6c0 msgr2=0x7fd6640cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:26.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66a4da700 1 --2- 172.21.15.6:0/1701840435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd66415a6c0 0x7fd6640cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66a4da700 1 -- 172.21.15.6:0/1701840435 >> v1:172.21.15.61:6789/0 conn(0x7fd6640d0020 legacy=0x7fd6640ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66a4da700 1 -- 172.21.15.6:0/1701840435 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6640cf260 con 0x7fd6640b2660 2023-11-28T13:01:26.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.230+0000 7fd66a4da700 1 --2- 172.21.15.6:0/1701840435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640b2660 0x7fd6640b2ae0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009e50 tx=0x7fd65c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d68cb798084753 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:26.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd668cd7700 1 -- 172.21.15.6:0/1701840435 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c00e070 con 0x7fd6640b2660 2023-11-28T13:01:26.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd668cd7700 1 -- 172.21.15.6:0/1701840435 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c00b9c0 con 0x7fd6640b2660 2023-11-28T13:01:26.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd668cd7700 1 -- 172.21.15.6:0/1701840435 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c00bd70 con 0x7fd6640b2660 2023-11-28T13:01:26.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1701840435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640b2660 msgr2=0x7fd6640b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd66bf3d700 1 --2- 172.21.15.6:0/1701840435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640b2660 0x7fd6640b2ae0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009e50 tx=0x7fd65c00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:26.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1701840435 shutdown_connections 2023-11-28T13:01:26.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd66bf3d700 1 --2- 172.21.15.6:0/1701840435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640b2660 0x7fd6640b2ae0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd66bf3d700 1 --2- 172.21.15.6:0/1701840435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd66415a6c0 0x7fd6640cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1701840435 >> 172.21.15.6:0/1701840435 conn(0x7fd6640cd660 msgr2=0x7fd6640cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:26.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1701840435 shutdown_connections 2023-11-28T13:01:26.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.231+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1701840435 wait complete. 2023-11-28T13:01:26.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.232+0000 7fd66bf3d700 1 Processor -- start 2023-11-28T13:01:26.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.232+0000 7fd66bf3d700 1 -- start start 2023-11-28T13:01:26.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd66bf3d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6640b2660 0x7fd66406bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd66bf3d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640d0020 0x7fd66406d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd66bf3d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd66415a6c0 0x7fd66415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd66bf3d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6640b5030 con 0x7fd6640d0020 2023-11-28T13:01:26.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd66bf3d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66406ce90 con 0x7fd66415a6c0 2023-11-28T13:01:26.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd66bf3d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66406c280 con 0x7fd6640b2660 2023-11-28T13:01:26.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd6694d8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640d0020 0x7fd66406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd6694d8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640d0020 0x7fd66406d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59866/0 (socket says 172.21.15.6:59866) 2023-11-28T13:01:26.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd6694d8700 1 -- 172.21.15.6:0/1410160776 learned_addr learned my addr 172.21.15.6:0/1410160776 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:26.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd669cd9700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6640b2660 0x7fd66406bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd66a4da700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd66415a6c0 0x7fd66415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd6694d8700 1 -- 172.21.15.6:0/1410160776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6640b2660 msgr2=0x7fd66406bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd6694d8700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6640b2660 0x7fd66406bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.233+0000 7fd6694d8700 1 -- 172.21.15.6:0/1410160776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd66415a6c0 msgr2=0x7fd66415aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.234+0000 7fd6694d8700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd66415a6c0 0x7fd66415aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.234+0000 7fd6694d8700 1 -- 172.21.15.6:0/1410160776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c009b30 con 0x7fd6640d0020 2023-11-28T13:01:26.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.234+0000 7fd669cd9700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6640b2660 0x7fd66406bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:26.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.234+0000 7fd6694d8700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640d0020 0x7fd66406d320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c4e0 tx=0x7fd66000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:26.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.234+0000 7fd65a7fc700 1 -- 172.21.15.6:0/1410160776 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600052d0 con 0x7fd6640d0020 2023-11-28T13:01:26.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.234+0000 7fd65a7fc700 1 -- 172.21.15.6:0/1410160776 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd660005430 con 0x7fd6640d0020 2023-11-28T13:01:26.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.234+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1410160776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6640cf820 con 0x7fd6640d0020 2023-11-28T13:01:26.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.234+0000 7fd65a7fc700 1 -- 172.21.15.6:0/1410160776 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600038c0 con 0x7fd6640d0020 2023-11-28T13:01:26.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.234+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1410160776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6640cf540 con 0x7fd6640d0020 2023-11-28T13:01:26.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.235+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1410160776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6640d0470 con 0x7fd6640d0020 2023-11-28T13:01:26.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.235+0000 7fd65a7fc700 1 -- 172.21.15.6:0/1410160776 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd660019030 con 0x7fd6640d0020 2023-11-28T13:01:26.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.236+0000 7fd65a7fc700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6480682f0 0x7fd64806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.236+0000 7fd65a7fc700 1 -- 172.21.15.6:0/1410160776 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd660085350 con 0x7fd6640d0020 2023-11-28T13:01:26.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.236+0000 7fd669cd9700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6480682f0 0x7fd64806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.236+0000 7fd669cd9700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6480682f0 0x7fd64806a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd664151e30 tx=0x7fd654008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:26.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.240+0000 7fd65a7fc700 1 -- 172.21.15.6:0/1410160776 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd66004b5e0 con 0x7fd6640d0020 2023-11-28T13:01:26.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.478+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1410160776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd66406d000 con 0x7fd6640d0020 2023-11-28T13:01:26.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.479+0000 7fd65a7fc700 1 -- 172.21.15.6:0/1410160776 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4638 (secure 0 0 0) 0x7fd6600517c0 con 0x7fd6640d0020 2023-11-28T13:01:26.481 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:26.481 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6834/1568686201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":1568686201},{"type":"v1","addr":"172.21.15.61:6834","nonce":1568686201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/3333487635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6833","nonce":3333487635},{"type":"v1","addr":"172.21.15.61:6835","nonce":3333487635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3917075730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3917075730},{"type":"v1","addr":"172.21.15.6:6835","nonce":3917075730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:21.404170+0000","modified":"2023-11-28T13:00:37.945716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/1668558371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1668558371},{"type":"v1","addr":"172.21.15.6:6837","nonce":1668558371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:01:26.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1410160776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6480682f0 msgr2=0x7fd64806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6480682f0 0x7fd64806a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd664151e30 tx=0x7fd654008040 comp rx=0 tx=0).stop 2023-11-28T13:01:26.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1410160776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640d0020 msgr2=0x7fd66406d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640d0020 0x7fd66406d320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c4e0 tx=0x7fd66000c8a0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1410160776 shutdown_connections 2023-11-28T13:01:26.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd66415a6c0 0x7fd66415aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6480682f0 0x7fd64806a7a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6640b2660 0x7fd66406bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 --2- 172.21.15.6:0/1410160776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6640d0020 0x7fd66406d320 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1410160776 >> 172.21.15.6:0/1410160776 conn(0x7fd6640cd660 msgr2=0x7fd6640ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:26.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1410160776 shutdown_connections 2023-11-28T13:01:26.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.481+0000 7fd66bf3d700 1 -- 172.21.15.6:0/1410160776 wait complete. 2023-11-28T13:01:26.484 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 9 2023-11-28T13:01:26.495 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:26.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.618+0000 7f4555657700 1 Processor -- start 2023-11-28T13:01:26.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.618+0000 7f4555657700 1 -- start start 2023-11-28T13:01:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f4555657700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f455015a230 0x7f455015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f4555657700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455006abb0 con 0x7f45500cd4a0 2023-11-28T13:01:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f4555657700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455006a280 con 0x7f45500b4fe0 2023-11-28T13:01:26.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f4555657700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455015a600 con 0x7f455015a230 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f454effd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f455015a230 0x7f455015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f454e7fc700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f45500cd4a0 0x7f45500b48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47196/0 (socket says 172.21.15.6:47196) 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f454e7fc700 1 -- 172.21.15.6:0/2018233249 learned_addr learned my addr 172.21.15.6:0/2018233249 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f454effd700 1 -- 172.21.15.6:0/2018233249 >> v1:172.21.15.61:6789/0 conn(0x7f45500b4fe0 legacy=0x7f45500d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f454effd700 1 -- 172.21.15.6:0/2018233249 >> v1:172.21.15.6:6789/0 conn(0x7f45500cd4a0 legacy=0x7f45500b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f454effd700 1 -- 172.21.15.6:0/2018233249 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455006a560 con 0x7f455015a230 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f454effd700 1 --2- 172.21.15.6:0/2018233249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 0x7f455015d6b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4544009840 tx=0x7f4544009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ccef85f5d23b8e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.619+0000 7f454dffb700 1 -- 172.21.15.6:0/2018233249 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544004230 con 0x7f455015a230 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.620+0000 7f454dffb700 1 -- 172.21.15.6:0/2018233249 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4544004390 con 0x7f455015a230 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.620+0000 7f454dffb700 1 -- 172.21.15.6:0/2018233249 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440036a0 con 0x7f455015a230 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.620+0000 7f4555657700 1 -- 172.21.15.6:0/2018233249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 msgr2=0x7f455015d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.620+0000 7f4555657700 1 --2- 172.21.15.6:0/2018233249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 0x7f455015d6b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4544009840 tx=0x7f4544009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.620+0000 7f4555657700 1 -- 172.21.15.6:0/2018233249 shutdown_connections 2023-11-28T13:01:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.620+0000 7f4555657700 1 --2- 172.21.15.6:0/2018233249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 0x7f455015d6b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.620+0000 7f4555657700 1 -- 172.21.15.6:0/2018233249 >> 172.21.15.6:0/2018233249 conn(0x7f45500cf8d0 msgr2=0x7f45500cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.620+0000 7f4555657700 1 -- 172.21.15.6:0/2018233249 shutdown_connections 2023-11-28T13:01:26.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.620+0000 7f4555657700 1 -- 172.21.15.6:0/2018233249 wait complete. 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.621+0000 7f4555657700 1 Processor -- start 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.622+0000 7f4555657700 1 -- start start 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.622+0000 7f4555657700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45500b4fe0 0x7f45500c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.622+0000 7f4555657700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500cd4a0 0x7f4550158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.622+0000 7f4555657700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 0x7f45500c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.622+0000 7f4555657700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45500c6e00 con 0x7f455015a230 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.622+0000 7f4555657700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45500b7ae0 con 0x7f45500b4fe0 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.622+0000 7f4555657700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45500b7dc0 con 0x7f45500cd4a0 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.622+0000 7f454f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 0x7f45500c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.622+0000 7f454f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 0x7f45500c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59882/0 (socket says 172.21.15.6:59882) 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f454f7fe700 1 -- 172.21.15.6:0/1711328035 learned_addr learned my addr 172.21.15.6:0/1711328035 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:26.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f454effd700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45500b4fe0 0x7f45500c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f454e7fc700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500cd4a0 0x7f4550158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f454f7fe700 1 -- 172.21.15.6:0/1711328035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500cd4a0 msgr2=0x7f4550158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f454f7fe700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500cd4a0 0x7f4550158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f454f7fe700 1 -- 172.21.15.6:0/1711328035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45500b4fe0 msgr2=0x7f45500c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f454f7fe700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45500b4fe0 0x7f45500c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f454f7fe700 1 -- 172.21.15.6:0/1711328035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440094a0 con 0x7f455015a230 2023-11-28T13:01:26.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f454f7fe700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 0x7f45500c8720 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f454800c780 tx=0x7f454800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:26.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f45377fe700 1 -- 172.21.15.6:0/1711328035 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548003e10 con 0x7f455015a230 2023-11-28T13:01:26.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f45377fe700 1 -- 172.21.15.6:0/1711328035 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4548005070 con 0x7f455015a230 2023-11-28T13:01:26.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f4555657700 1 -- 172.21.15.6:0/1711328035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45500cd920 con 0x7f455015a230 2023-11-28T13:01:26.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f45377fe700 1 -- 172.21.15.6:0/1711328035 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548005420 con 0x7f455015a230 2023-11-28T13:01:26.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.623+0000 7f4555657700 1 -- 172.21.15.6:0/1711328035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455006a840 con 0x7f455015a230 2023-11-28T13:01:26.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.624+0000 7f4555657700 1 -- 172.21.15.6:0/1711328035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455006b020 con 0x7f455015a230 2023-11-28T13:01:26.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.625+0000 7f45377fe700 1 -- 172.21.15.6:0/1711328035 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4548019030 con 0x7f455015a230 2023-11-28T13:01:26.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.625+0000 7f45377fe700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f453c0682f0 0x7f453c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.625+0000 7f45377fe700 1 -- 172.21.15.6:0/1711328035 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4548085590 con 0x7f455015a230 2023-11-28T13:01:26.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.625+0000 7f454effd700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f453c0682f0 0x7f453c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.626+0000 7f454effd700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f453c0682f0 0x7f453c06a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f45440090b0 tx=0x7f4544008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:26.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.629+0000 7f45377fe700 1 -- 172.21.15.6:0/1711328035 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4548051ac0 con 0x7f455015a230 2023-11-28T13:01:26.640 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:01:26.642 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:01:26.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.826+0000 7f4555657700 1 -- 172.21.15.6:0/1711328035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45500675a0 con 0x7f455015a230 2023-11-28T13:01:26.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.827+0000 7f45377fe700 1 -- 172.21.15.6:0/1711328035 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19690 (secure 0 0 0) 0x7f4548017090 con 0x7f455015a230 2023-11-28T13:01:26.829 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:26.829 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:10.053012+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:26.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.829+0000 7f4555657700 1 -- 172.21.15.6:0/1711328035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f453c0682f0 msgr2=0x7f453c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.829+0000 7f4555657700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f453c0682f0 0x7f453c06a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f45440090b0 tx=0x7f4544008030 comp rx=0 tx=0).stop 2023-11-28T13:01:26.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.829+0000 7f4555657700 1 -- 172.21.15.6:0/1711328035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 msgr2=0x7f45500c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.829+0000 7f4555657700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 0x7f45500c8720 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f454800c780 tx=0x7f454800cb40 comp rx=0 tx=0).stop 2023-11-28T13:01:26.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.830+0000 7f4555657700 1 -- 172.21.15.6:0/1711328035 shutdown_connections 2023-11-28T13:01:26.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.830+0000 7f4555657700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f455015a230 0x7f45500c8720 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.830+0000 7f4555657700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45500b4fe0 0x7f45500c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.830+0000 7f4555657700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f453c0682f0 0x7f453c06a7a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.830+0000 7f4555657700 1 --2- 172.21.15.6:0/1711328035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f45500cd4a0 0x7f4550158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.830+0000 7f4555657700 1 -- 172.21.15.6:0/1711328035 >> 172.21.15.6:0/1711328035 conn(0x7f45500cf8d0 msgr2=0x7f45500d1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:26.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.830+0000 7f4555657700 1 -- 172.21.15.6:0/1711328035 shutdown_connections 2023-11-28T13:01:26.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.830+0000 7f4555657700 1 -- 172.21.15.6:0/1711328035 wait complete. 2023-11-28T13:01:26.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:00:19.218196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:00:20.374831+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T13:00:23.460601+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T13:01:26.844 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:26.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.970+0000 7fa0a08fc700 1 Processor -- start 2023-11-28T13:01:26.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.970+0000 7fa0a08fc700 1 -- start start 2023-11-28T13:01:26.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.971+0000 7fa0a08fc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa09c15c880 0x7fa09c15d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.971+0000 7fa0a08fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c0c9950 0x7fa09c0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.971+0000 7fa0a08fc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c0cae80 con 0x7fa09c15c4b0 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.971+0000 7fa0a08fc700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c15dba0 con 0x7fa09c15c880 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.971+0000 7fa0a08fc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0c9de0 con 0x7fa09c0c9950 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa099d9b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa09c15c880 0x7fa09c15d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09ad9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c0c9950 0x7fa09c0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09a59c700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fa09c15c4b0 0x7fa09c06a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:37236/0 (socket says 172.21.15.6:37236) 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09a59c700 1 -- 172.21.15.6:0/3115197205 learned_addr learned my addr 172.21.15.6:0/3115197205 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09ad9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c0c9950 0x7fa09c0cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:49594/0 (socket says 172.21.15.6:49594) 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa099d9b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa09c15c880 0x7fa09c15d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:35374/0 (socket says 172.21.15.6:35374) 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09959a700 1 -- 172.21.15.6:0/3115197205 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807968270 0 0) 0x7fa09c0cae80 con 0x7fa09c15c4b0 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09959a700 1 -- 172.21.15.6:0/3115197205 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0840035b0 con 0x7fa09c15c4b0 2023-11-28T13:01:26.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09ad9d700 1 -- 172.21.15.6:0/3115197205 >> v2:172.21.15.61:3300/0 conn(0x7fa09c15c880 msgr2=0x7fa09c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:26.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09ad9d700 1 --2- 172.21.15.6:0/3115197205 >> v2:172.21.15.61:3300/0 conn(0x7fa09c15c880 0x7fa09c15d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09ad9d700 1 -- 172.21.15.6:0/3115197205 >> v1:172.21.15.61:6789/0 conn(0x7fa09c15c4b0 legacy=0x7fa09c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09ad9d700 1 -- 172.21.15.6:0/3115197205 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c0ca0c0 con 0x7fa09c0c9950 2023-11-28T13:01:26.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa099d9b700 1 --2- 172.21.15.6:0/3115197205 >> v2:172.21.15.61:3300/0 conn(0x7fa09c15c880 0x7fa09c15d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:26.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09ad9d700 1 --2- 172.21.15.6:0/3115197205 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c0c9950 0x7fa09c0cced0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa094009d90 tx=0x7fa09400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b897598e4e96993a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:26.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.972+0000 7fa09959a700 1 -- 172.21.15.6:0/3115197205 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09400e070 con 0x7fa09c0c9950 2023-11-28T13:01:26.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.973+0000 7fa09959a700 1 -- 172.21.15.6:0/3115197205 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09400bb20 con 0x7fa09c0c9950 2023-11-28T13:01:26.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.973+0000 7fa09959a700 1 -- 172.21.15.6:0/3115197205 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09400be90 con 0x7fa09c0c9950 2023-11-28T13:01:26.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.973+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3115197205 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c0c9950 msgr2=0x7fa09c0cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.973+0000 7fa0a08fc700 1 --2- 172.21.15.6:0/3115197205 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c0c9950 0x7fa09c0cced0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa094009d90 tx=0x7fa09400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:26.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.973+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3115197205 shutdown_connections 2023-11-28T13:01:26.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.973+0000 7fa0a08fc700 1 --2- 172.21.15.6:0/3115197205 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c0c9950 0x7fa09c0cced0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.973+0000 7fa0a08fc700 1 --2- 172.21.15.6:0/3115197205 >> v2:172.21.15.61:3300/0 conn(0x7fa09c15c880 0x7fa09c15d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.973+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3115197205 >> 172.21.15.6:0/3115197205 conn(0x7fa09c0b0df0 msgr2=0x7fa09c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:26.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.973+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3115197205 shutdown_connections 2023-11-28T13:01:26.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.973+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3115197205 wait complete. 2023-11-28T13:01:26.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.975+0000 7fa0a08fc700 1 Processor -- start 2023-11-28T13:01:26.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.975+0000 7fa0a08fc700 1 -- start start 2023-11-28T13:01:26.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa0a08fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa09c0c9950 0x7fa09c150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa0a08fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa09c15c4b0 0x7fa09c0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa0a08fc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c15c880 0x7fa09c0ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa0a08fc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c150d60 con 0x7fa09c15c4b0 2023-11-28T13:01:26.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa0a08fc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0c6d90 con 0x7fa09c0c9950 2023-11-28T13:01:26.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa0a08fc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0b3b40 con 0x7fa09c15c880 2023-11-28T13:01:26.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa099d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa09c15c4b0 0x7fa09c0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa099d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa09c15c4b0 0x7fa09c0b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59894/0 (socket says 172.21.15.6:59894) 2023-11-28T13:01:26.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa099d9b700 1 -- 172.21.15.6:0/3785306483 learned_addr learned my addr 172.21.15.6:0/3785306483 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa09a59c700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa09c0c9950 0x7fa09c150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa09ad9d700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c15c880 0x7fa09c0ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa099d9b700 1 -- 172.21.15.6:0/3785306483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c15c880 msgr2=0x7fa09c0ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa099d9b700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c15c880 0x7fa09c0ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa099d9b700 1 -- 172.21.15.6:0/3785306483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa09c0c9950 msgr2=0x7fa09c150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa099d9b700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa09c0c9950 0x7fa09c150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.976+0000 7fa099d9b700 1 -- 172.21.15.6:0/3785306483 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900094a0 con 0x7fa09c15c4b0 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.977+0000 7fa099d9b700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa09c15c4b0 0x7fa09c0b3600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa09000ebe0 tx=0x7fa09000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.977+0000 7fa08affd700 1 -- 172.21.15.6:0/3785306483 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090009760 con 0x7fa09c15c4b0 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.977+0000 7fa08affd700 1 -- 172.21.15.6:0/3785306483 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09000c850 con 0x7fa09c15c4b0 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.977+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3785306483 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa094009a70 con 0x7fa09c15c4b0 2023-11-28T13:01:26.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.977+0000 7fa08affd700 1 -- 172.21.15.6:0/3785306483 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09000cbe0 con 0x7fa09c15c4b0 2023-11-28T13:01:26.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.977+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3785306483 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c0ca680 con 0x7fa09c15c4b0 2023-11-28T13:01:26.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.978+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3785306483 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c0ca3a0 con 0x7fa09c15c4b0 2023-11-28T13:01:26.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.978+0000 7fa08affd700 1 -- 172.21.15.6:0/3785306483 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa090019030 con 0x7fa09c15c4b0 2023-11-28T13:01:26.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.979+0000 7fa08affd700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa07c0682f0 0x7fa07c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:26.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.979+0000 7fa08affd700 1 -- 172.21.15.6:0/3785306483 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa090085e80 con 0x7fa09c15c4b0 2023-11-28T13:01:26.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.979+0000 7fa09a59c700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa07c0682f0 0x7fa07c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:26.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.979+0000 7fa09a59c700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa07c0682f0 0x7fa07c06a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa08c007fd0 tx=0x7fa08c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:26.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:26.983+0000 7fa08affd700 1 -- 172.21.15.6:0/3785306483 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa0900523b0 con 0x7fa09c15c4b0 2023-11-28T13:01:27.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.179+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3785306483 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa09c0ca960 con 0x7fa09c15c4b0 2023-11-28T13:01:27.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.179+0000 7fa08affd700 1 -- 172.21.15.6:0/3785306483 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa0900559d0 con 0x7fa09c15c4b0 2023-11-28T13:01:27.181 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:27.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.181+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3785306483 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa07c0682f0 msgr2=0x7fa07c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.181+0000 7fa0a08fc700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa07c0682f0 0x7fa07c06a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa08c007fd0 tx=0x7fa08c009040 comp rx=0 tx=0).stop 2023-11-28T13:01:27.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.181+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3785306483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa09c15c4b0 msgr2=0x7fa09c0b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.181+0000 7fa0a08fc700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa09c15c4b0 0x7fa09c0b3600 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa09000ebe0 tx=0x7fa09000eef0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.181+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3785306483 shutdown_connections 2023-11-28T13:01:27.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.181+0000 7fa0a08fc700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa07c0682f0 0x7fa07c06a7a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.181+0000 7fa0a08fc700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa09c15c880 0x7fa09c0ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.181+0000 7fa0a08fc700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa09c0c9950 0x7fa09c150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.181+0000 7fa0a08fc700 1 --2- 172.21.15.6:0/3785306483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa09c15c4b0 0x7fa09c0b3600 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.181+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3785306483 >> 172.21.15.6:0/3785306483 conn(0x7fa09c0b0df0 msgr2=0x7fa09c06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.182+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3785306483 shutdown_connections 2023-11-28T13:01:27.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.182+0000 7fa0a08fc700 1 -- 172.21.15.6:0/3785306483 wait complete. 2023-11-28T13:01:27.195 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:27.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.321+0000 7fd4b163a700 1 Processor -- start 2023-11-28T13:01:27.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.321+0000 7fd4b163a700 1 -- start start 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.322+0000 7fd4b163a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac1589c0 0x7fd4ac0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.322+0000 7fd4b163a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd4ac158d90 0x7fd4ac06aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.322+0000 7fd4b163a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac06e470 con 0x7fd4ac06e900 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.322+0000 7fd4b163a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac06f490 con 0x7fd4ac158d90 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.322+0000 7fd4b163a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac06de60 con 0x7fd4ac1589c0 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.322+0000 7fd4aa7fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd4ac158d90 0x7fd4ac06aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.322+0000 7fd4aa7fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd4ac158d90 0x7fd4ac06aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59902/0 (socket says 172.21.15.6:59902) 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.322+0000 7fd4aa7fc700 1 -- 172.21.15.6:0/2446051445 learned_addr learned my addr 172.21.15.6:0/2446051445 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4aaffd700 1 --2- 172.21.15.6:0/2446051445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac1589c0 0x7fd4ac0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4aa7fc700 1 -- 172.21.15.6:0/2446051445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac1589c0 msgr2=0x7fd4ac0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:27.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4aa7fc700 1 --2- 172.21.15.6:0/2446051445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac1589c0 0x7fd4ac0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4aa7fc700 1 -- 172.21.15.6:0/2446051445 >> v1:172.21.15.61:6789/0 conn(0x7fd4ac06e900 legacy=0x7fd4ac06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4aa7fc700 1 -- 172.21.15.6:0/2446051445 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac0ceb40 con 0x7fd4ac158d90 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4aa7fc700 1 --2- 172.21.15.6:0/2446051445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac158d90 0x7fd4ac06aa70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000bb90 tx=0x7fd4a000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53277015b7c30441 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4a9ffb700 1 -- 172.21.15.6:0/2446051445 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000dc40 con 0x7fd4ac158d90 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4a9ffb700 1 -- 172.21.15.6:0/2446051445 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a000dda0 con 0x7fd4ac158d90 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4a9ffb700 1 -- 172.21.15.6:0/2446051445 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0017630 con 0x7fd4ac158d90 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4b163a700 1 -- 172.21.15.6:0/2446051445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac158d90 msgr2=0x7fd4ac06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.323+0000 7fd4b163a700 1 --2- 172.21.15.6:0/2446051445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac158d90 0x7fd4ac06aa70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000bb90 tx=0x7fd4a000bea0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.324+0000 7fd4b163a700 1 -- 172.21.15.6:0/2446051445 shutdown_connections 2023-11-28T13:01:27.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.324+0000 7fd4b163a700 1 --2- 172.21.15.6:0/2446051445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac1589c0 0x7fd4ac0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.324+0000 7fd4b163a700 1 --2- 172.21.15.6:0/2446051445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac158d90 0x7fd4ac06aa70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.324+0000 7fd4b163a700 1 -- 172.21.15.6:0/2446051445 >> 172.21.15.6:0/2446051445 conn(0x7fd4ac0b0df0 msgr2=0x7fd4ac0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.324+0000 7fd4b163a700 1 -- 172.21.15.6:0/2446051445 shutdown_connections 2023-11-28T13:01:27.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.324+0000 7fd4b163a700 1 -- 172.21.15.6:0/2446051445 wait complete. 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.325+0000 7fd4b163a700 1 Processor -- start 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.325+0000 7fd4b163a700 1 -- start start 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4b163a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac06e900 0x7fd4ac0ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4b163a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac1589c0 0x7fd4ac155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4b163a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ac158d90 0x7fd4ac1f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4b163a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac156160 con 0x7fd4ac06e900 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4b163a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0c2e30 con 0x7fd4ac158d90 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4b163a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0c30d0 con 0x7fd4ac1589c0 2023-11-28T13:01:27.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4aaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac06e900 0x7fd4ac0ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4aaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac06e900 0x7fd4ac0ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59906/0 (socket says 172.21.15.6:59906) 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4aaffd700 1 -- 172.21.15.6:0/1643351081 learned_addr learned my addr 172.21.15.6:0/1643351081 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4aa7fc700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac1589c0 0x7fd4ac155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4ab7fe700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ac158d90 0x7fd4ac1f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4aaffd700 1 -- 172.21.15.6:0/1643351081 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac1589c0 msgr2=0x7fd4ac155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4aaffd700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac1589c0 0x7fd4ac155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4aaffd700 1 -- 172.21.15.6:0/1643351081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ac158d90 msgr2=0x7fd4ac1f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4aaffd700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ac158d90 0x7fd4ac1f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4aaffd700 1 -- 172.21.15.6:0/1643351081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a000b7f0 con 0x7fd4ac06e900 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4aaffd700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac06e900 0x7fd4ac0ae020 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00e810 tx=0x7fd49c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4937fe700 1 -- 172.21.15.6:0/1643351081 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c013610 con 0x7fd4ac06e900 2023-11-28T13:01:27.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4937fe700 1 -- 172.21.15.6:0/1643351081 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c013770 con 0x7fd4ac06e900 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4b163a700 1 -- 172.21.15.6:0/1643351081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac06e100 con 0x7fd4ac06e900 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.326+0000 7fd4937fe700 1 -- 172.21.15.6:0/1643351081 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c013b20 con 0x7fd4ac06e900 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.327+0000 7fd4b163a700 1 -- 172.21.15.6:0/1643351081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac0cee20 con 0x7fd4ac06e900 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.327+0000 7fd4b163a700 1 -- 172.21.15.6:0/1643351081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac06afb0 con 0x7fd4ac06e900 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.328+0000 7fd4937fe700 1 -- 172.21.15.6:0/1643351081 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd49c023070 con 0x7fd4ac06e900 2023-11-28T13:01:27.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.328+0000 7fd4937fe700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4980682f0 0x7fd49806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.328+0000 7fd4937fe700 1 -- 172.21.15.6:0/1643351081 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd49c085cd0 con 0x7fd4ac06e900 2023-11-28T13:01:27.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.328+0000 7fd4aa7fc700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4980682f0 0x7fd49806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.329+0000 7fd4aa7fc700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4980682f0 0x7fd49806a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac1591e0 tx=0x7fd4a0014000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.333+0000 7fd4937fe700 1 -- 172.21.15.6:0/1643351081 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd49c052180 con 0x7fd4ac06e900 2023-11-28T13:01:27.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.526+0000 7fd4b163a700 1 -- 172.21.15.6:0/1643351081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4ac0739f0 con 0x7fd4ac06e900 2023-11-28T13:01:27.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.526+0000 7fd4937fe700 1 -- 172.21.15.6:0/1643351081 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd49c0557a0 con 0x7fd4ac06e900 2023-11-28T13:01:27.527 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:27.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 -- 172.21.15.6:0/1643351081 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4980682f0 msgr2=0x7fd49806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4980682f0 0x7fd49806a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac1591e0 tx=0x7fd4a0014000 comp rx=0 tx=0).stop 2023-11-28T13:01:27.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 -- 172.21.15.6:0/1643351081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac06e900 msgr2=0x7fd4ac0ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac06e900 0x7fd4ac0ae020 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00e810 tx=0x7fd49c00ebd0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 -- 172.21.15.6:0/1643351081 shutdown_connections 2023-11-28T13:01:27.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd4ac158d90 0x7fd4ac1f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4ac06e900 0x7fd4ac0ae020 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd4980682f0 0x7fd49806a7a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 --2- 172.21.15.6:0/1643351081 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4ac1589c0 0x7fd4ac155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 -- 172.21.15.6:0/1643351081 >> 172.21.15.6:0/1643351081 conn(0x7fd4ac0b0df0 msgr2=0x7fd4ac06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 -- 172.21.15.6:0/1643351081 shutdown_connections 2023-11-28T13:01:27.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.528+0000 7fd4b163a700 1 -- 172.21.15.6:0/1643351081 wait complete. 2023-11-28T13:01:27.542 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.668+0000 7f2069442700 1 Processor -- start 2023-11-28T13:01:27.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.668+0000 7f2069442700 1 -- start start 2023-11-28T13:01:27.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2069442700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f20640ceec0 0x7f20640b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2069442700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206415d2d0 con 0x7f206415c890 2023-11-28T13:01:27.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2069442700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20641611d0 con 0x7f2064160d30 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2069442700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20640b2ba0 con 0x7f20640ceec0 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2062ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f206415c890 0x7f20640ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47234/0 (socket says 172.21.15.6:47234) 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2062ffd700 1 -- 172.21.15.6:0/1959220434 learned_addr learned my addr 172.21.15.6:0/1959220434 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f20627fc700 1 --2- 172.21.15.6:0/1959220434 >> v2:172.21.15.61:3300/0 conn(0x7f20640ceec0 0x7f20640b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683719717 0 0) 0x7f206415d2d0 con 0x7f206415c890 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f204c0035b0 con 0x7f206415c890 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 189361567 0 0) 0x7f204c0035b0 con 0x7f206415c890 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 >> v2:172.21.15.61:3300/0 conn(0x7f20640ceec0 msgr2=0x7f20640b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2061ffb700 1 --2- 172.21.15.6:0/1959220434 >> v2:172.21.15.61:3300/0 conn(0x7f20640ceec0 0x7f20640b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 >> v1:172.21.15.61:6789/0 conn(0x7f2064160d30 legacy=0x7f20640cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20640b2d90 con 0x7f206415c890 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f2058001670 con 0x7f206415c890 2023-11-28T13:01:27.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.669+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 >> v1:172.21.15.6:6789/0 conn(0x7f206415c890 legacy=0x7f20640ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f2061ffb700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f204c003a00 0x7f204c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f2061ffb700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f204c004850 0x7f204c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f2061ffb700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f204c005200 0x7f204c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c00c050 con 0x7f204c004850 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c00c1c0 con 0x7f204c003a00 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c00c380 con 0x7f204c005200 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f20627fc700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f204c004850 0x7f204c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f2062ffd700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f204c003a00 0x7f204c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f20637fe700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f204c005200 0x7f204c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f20627fc700 1 -- 172.21.15.6:0/1959220434 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f204c005200 msgr2=0x7f204c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:27.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f20627fc700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f204c005200 0x7f204c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f20627fc700 1 -- 172.21.15.6:0/1959220434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f204c003a00 msgr2=0x7f204c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f20627fc700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f204c003a00 0x7f204c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f20627fc700 1 -- 172.21.15.6:0/1959220434 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0035b0 con 0x7f204c004850 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f20637fe700 1 -- 172.21.15.6:0/1959220434 reap_dead start 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f20627fc700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f204c004850 0x7f204c004cc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2054009b30 tx=0x7f204c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be56f69685551b6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205400e6c0 con 0x7f204c004850 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205400e820 con 0x7f204c004850 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.670+0000 7f2061ffb700 1 -- 172.21.15.6:0/1959220434 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205400eb90 con 0x7f204c004850 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.671+0000 7f2069442700 1 -- 172.21.15.6:0/1959220434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f204c004850 msgr2=0x7f204c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.671+0000 7f2069442700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f204c004850 0x7f204c004cc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2054009b30 tx=0x7f204c005880 comp rx=0 tx=0).stop 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.671+0000 7f2069442700 1 -- 172.21.15.6:0/1959220434 shutdown_connections 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.671+0000 7f2069442700 1 --2- 172.21.15.6:0/1959220434 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f204c004850 0x7f204c004cc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.671+0000 7f2069442700 1 -- 172.21.15.6:0/1959220434 >> 172.21.15.6:0/1959220434 conn(0x7f206406e9b0 msgr2=0x7f20640cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.671+0000 7f2069442700 1 -- 172.21.15.6:0/1959220434 shutdown_connections 2023-11-28T13:01:27.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.671+0000 7f2069442700 1 -- 172.21.15.6:0/1959220434 wait complete. 2023-11-28T13:01:27.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.672+0000 7f2069442700 1 Processor -- start 2023-11-28T13:01:27.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.672+0000 7f2069442700 1 -- start start 2023-11-28T13:01:27.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f2069442700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f204c004850 0x7f20641f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f2069442700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2064156c90 0x7f20640ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f2069442700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20640caaf0 0x7f20640c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f2069442700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20640caf60 con 0x7f2064156c90 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f2069442700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20640c92f0 con 0x7f204c004850 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f2069442700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206415d940 con 0x7f20640caaf0 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f20627fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2064156c90 0x7f20640ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f20627fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2064156c90 0x7f20640ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59928/0 (socket says 172.21.15.6:59928) 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f20627fc700 1 -- 172.21.15.6:0/3806312103 learned_addr learned my addr 172.21.15.6:0/3806312103 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f2062ffd700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f204c004850 0x7f20641f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f20627fc700 1 -- 172.21.15.6:0/3806312103 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20640caaf0 msgr2=0x7f20640c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f20627fc700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20640caaf0 0x7f20640c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f20627fc700 1 -- 172.21.15.6:0/3806312103 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f204c004850 msgr2=0x7f20641f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f20627fc700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f204c004850 0x7f20641f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f20627fc700 1 -- 172.21.15.6:0/3806312103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206415cf60 con 0x7f2064156c90 2023-11-28T13:01:27.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.673+0000 7f2062ffd700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f204c004850 0x7f20641f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.674+0000 7f20627fc700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2064156c90 0x7f20640ca5b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f204c00c190 tx=0x7f20540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.674+0000 7f204b7fe700 1 -- 172.21.15.6:0/3806312103 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205400e6c0 con 0x7f2064156c90 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.674+0000 7f204b7fe700 1 -- 172.21.15.6:0/3806312103 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2054003af0 con 0x7f2064156c90 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.674+0000 7f2069442700 1 -- 172.21.15.6:0/3806312103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20640b3070 con 0x7f2064156c90 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.674+0000 7f204b7fe700 1 -- 172.21.15.6:0/3806312103 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054003ea0 con 0x7f2064156c90 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.674+0000 7f2069442700 1 -- 172.21.15.6:0/3806312103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206415cc60 con 0x7f2064156c90 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.675+0000 7f2069442700 1 -- 172.21.15.6:0/3806312103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20640caf00 con 0x7f2064156c90 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.675+0000 7f204b7fe700 1 -- 172.21.15.6:0/3806312103 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2054019030 con 0x7f2064156c90 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.676+0000 7f204b7fe700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2050068320 0x7f205006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.676+0000 7f204b7fe700 1 -- 172.21.15.6:0/3806312103 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2054087600 con 0x7f2064156c90 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.676+0000 7f2062ffd700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2050068320 0x7f205006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:27.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.676+0000 7f2062ffd700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2050068320 0x7f205006a7d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2058007cf0 tx=0x7f2058009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:27.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.680+0000 7f204b7fe700 1 -- 172.21.15.6:0/3806312103 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20640caf00 con 0x7f2064156c90 2023-11-28T13:01:27.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.874+0000 7f2069442700 1 -- 172.21.15.6:0/3806312103 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20640caf00 con 0x7f2064156c90 2023-11-28T13:01:27.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.875+0000 7f204b7fe700 1 -- 172.21.15.6:0/3806312103 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f20640caf00 con 0x7f2064156c90 2023-11-28T13:01:27.876 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:27.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.876+0000 7f2069442700 1 -- 172.21.15.6:0/3806312103 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2050068320 msgr2=0x7f205006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.876+0000 7f2069442700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2050068320 0x7f205006a7d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2058007cf0 tx=0x7f2058009040 comp rx=0 tx=0).stop 2023-11-28T13:01:27.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.877+0000 7f2069442700 1 -- 172.21.15.6:0/3806312103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2064156c90 msgr2=0x7f20640ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:27.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.877+0000 7f2069442700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2064156c90 0x7f20640ca5b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f204c00c190 tx=0x7f20540047c0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.877+0000 7f2069442700 1 -- 172.21.15.6:0/3806312103 shutdown_connections 2023-11-28T13:01:27.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.877+0000 7f2069442700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2050068320 0x7f205006a7d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.877+0000 7f2069442700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20640caaf0 0x7f20640c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.877+0000 7f2069442700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f204c004850 0x7f20641f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.877+0000 7f2069442700 1 --2- 172.21.15.6:0/3806312103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2064156c90 0x7f20640ca5b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:27.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.877+0000 7f2069442700 1 -- 172.21.15.6:0/3806312103 >> 172.21.15.6:0/3806312103 conn(0x7f206406e9b0 msgr2=0x7f20640cb210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:27.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.877+0000 7f2069442700 1 -- 172.21.15.6:0/3806312103 shutdown_connections 2023-11-28T13:01:27.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:27.877+0000 7f2069442700 1 -- 172.21.15.6:0/3806312103 wait complete. 2023-11-28T13:01:27.890 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:28.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.018+0000 7f8278805700 1 Processor -- start 2023-11-28T13:01:28.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.018+0000 7f8278805700 1 -- start start 2023-11-28T13:01:28.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f8278805700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274165670 0x7f8274167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f8278805700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82740ce550 con 0x7f82740cf5f0 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f8278805700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82740ce830 con 0x7f82740b1630 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f8278805700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82740ceb10 con 0x7f8274165670 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f8271d9b700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f82740cf5f0 0x7f82740caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47254/0 (socket says 172.21.15.6:47254) 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f8271d9b700 1 -- 172.21.15.6:0/3946783288 learned_addr learned my addr 172.21.15.6:0/3946783288 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230211820 0 0) 0x7f82740ce550 con 0x7f82740cf5f0 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f8272d9d700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274165670 0x7f8274167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f825c0035b0 con 0x7f82740cf5f0 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2530948509 0 0) 0x7f825c0035b0 con 0x7f82740cf5f0 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.019+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274165670 msgr2=0x7f8274167a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274165670 0x7f8274167a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 >> v1:172.21.15.61:6789/0 conn(0x7f82740b1630 legacy=0x7f82740b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82740cedf0 con 0x7f82740cf5f0 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f8264001670 con 0x7f82740cf5f0 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 >> v1:172.21.15.6:6789/0 conn(0x7f82740cf5f0 legacy=0x7f82740caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f825c003a30 0x7f825c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f825c004850 0x7f825c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274165670 0x7f825c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82740ceb10 con 0x7f825c003a30 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825c00bac0 con 0x7f825c004850 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825c00bc80 con 0x7f8274165670 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827259c700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f825c003a30 0x7f825c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f8272d9d700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274165670 0x7f825c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f8271d9b700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f825c004850 0x7f825c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827259c700 1 -- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274165670 msgr2=0x7f825c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827259c700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274165670 0x7f825c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827259c700 1 -- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f825c004850 msgr2=0x7f825c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827259c700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f825c004850 0x7f825c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827259c700 1 -- 172.21.15.6:0/3946783288 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c0035b0 con 0x7f825c003a30 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f8271d9b700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f825c004850 0x7f825c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827259c700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f825c003a30 0x7f825c003e40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f82680017c0 tx=0x7f825c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed230670f86f750b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82680052d0 con 0x7f825c003a30 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8268005430 con 0x7f825c003a30 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.020+0000 7f827159a700 1 -- 172.21.15.6:0/3946783288 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82680038c0 con 0x7f825c003a30 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.021+0000 7f8278805700 1 -- 172.21.15.6:0/3946783288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f825c003a30 msgr2=0x7f825c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.021+0000 7f8278805700 1 --2- 172.21.15.6:0/3946783288 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f825c003a30 0x7f825c003e40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f82680017c0 tx=0x7f825c005880 comp rx=0 tx=0).stop 2023-11-28T13:01:28.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.021+0000 7f8272d9d700 1 -- 172.21.15.6:0/3946783288 reap_dead start 2023-11-28T13:01:28.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.021+0000 7f8278805700 1 -- 172.21.15.6:0/3946783288 shutdown_connections 2023-11-28T13:01:28.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.021+0000 7f8278805700 1 -- 172.21.15.6:0/3946783288 >> 172.21.15.6:0/3946783288 conn(0x7f82740cc630 msgr2=0x7f82740cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.021+0000 7f8278805700 1 -- 172.21.15.6:0/3946783288 shutdown_connections 2023-11-28T13:01:28.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.021+0000 7f8278805700 1 -- 172.21.15.6:0/3946783288 wait complete. 2023-11-28T13:01:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.022+0000 7f8278805700 1 Processor -- start 2023-11-28T13:01:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.022+0000 7f8278805700 1 -- start start 2023-11-28T13:01:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8278805700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f825c003a30 0x7f8274152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8278805700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274153720 0x7f827415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8278805700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f827415d1c0 0x7f827415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8278805700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82740b4430 con 0x7f827415d1c0 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8278805700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8274154d00 con 0x7f825c003a30 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8278805700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8274152960 con 0x7f8274153720 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8272d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f827415d1c0 0x7f827415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8272d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f827415d1c0 0x7f827415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59948/0 (socket says 172.21.15.6:59948) 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8272d9d700 1 -- 172.21.15.6:0/4023369600 learned_addr learned my addr 172.21.15.6:0/4023369600 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f827259c700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f825c003a30 0x7f8274152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8271d9b700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274153720 0x7f827415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8272d9d700 1 -- 172.21.15.6:0/4023369600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274153720 msgr2=0x7f827415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8272d9d700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274153720 0x7f827415cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8272d9d700 1 -- 172.21.15.6:0/4023369600 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f825c003a30 msgr2=0x7f8274152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8272d9d700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f825c003a30 0x7f8274152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.023+0000 7f8272d9d700 1 -- 172.21.15.6:0/4023369600 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82740cb5b0 con 0x7f827415d1c0 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.024+0000 7f8271d9b700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274153720 0x7f827415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.024+0000 7f8272d9d700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f827415d1c0 0x7f827415d630 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f827406eb30 tx=0x7f826c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.024+0000 7f8262ffd700 1 -- 172.21.15.6:0/4023369600 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c00e040 con 0x7f827415d1c0 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.024+0000 7f8262ffd700 1 -- 172.21.15.6:0/4023369600 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c00f040 con 0x7f827415d1c0 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.024+0000 7f8278805700 1 -- 172.21.15.6:0/4023369600 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82740cf0d0 con 0x7f827415d1c0 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.024+0000 7f8262ffd700 1 -- 172.21.15.6:0/4023369600 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c015430 con 0x7f827415d1c0 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.024+0000 7f8278805700 1 -- 172.21.15.6:0/4023369600 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82740ce140 con 0x7f827415d1c0 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.025+0000 7f8278805700 1 -- 172.21.15.6:0/4023369600 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82740739f0 con 0x7f827415d1c0 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.025+0000 7f8262ffd700 1 -- 172.21.15.6:0/4023369600 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f826c015590 con 0x7f827415d1c0 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.026+0000 7f8262ffd700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82540682f0 0x7f825406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.026+0000 7f8262ffd700 1 -- 172.21.15.6:0/4023369600 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f826c0865a0 con 0x7f827415d1c0 2023-11-28T13:01:28.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.026+0000 7f827259c700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82540682f0 0x7f825406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.026+0000 7f827259c700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82540682f0 0x7f825406a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8268000c00 tx=0x7f8268009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.030+0000 7f8262ffd700 1 -- 172.21.15.6:0/4023369600 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f826c052a70 con 0x7f827415d1c0 2023-11-28T13:01:28.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.225+0000 7f8278805700 1 -- 172.21.15.6:0/4023369600 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82740c3480 con 0x7f827415d1c0 2023-11-28T13:01:28.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.226+0000 7f8262ffd700 1 -- 172.21.15.6:0/4023369600 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f826c056090 con 0x7f827415d1c0 2023-11-28T13:01:28.227 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:28.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 -- 172.21.15.6:0/4023369600 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82540682f0 msgr2=0x7f825406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82540682f0 0x7f825406a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8268000c00 tx=0x7f8268009030 comp rx=0 tx=0).stop 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 -- 172.21.15.6:0/4023369600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f827415d1c0 msgr2=0x7f827415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f827415d1c0 0x7f827415d630 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f827406eb30 tx=0x7f826c011f60 comp rx=0 tx=0).stop 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 -- 172.21.15.6:0/4023369600 shutdown_connections 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f827415d1c0 0x7f827415d630 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f825c003a30 0x7f8274152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f82540682f0 0x7f825406a7a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 --2- 172.21.15.6:0/4023369600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8274153720 0x7f827415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 -- 172.21.15.6:0/4023369600 >> 172.21.15.6:0/4023369600 conn(0x7f82740cc630 msgr2=0x7f82740cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 -- 172.21.15.6:0/4023369600 shutdown_connections 2023-11-28T13:01:28.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.228+0000 7f8278805700 1 -- 172.21.15.6:0/4023369600 wait complete. 2023-11-28T13:01:28.242 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T13:01:28.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.373+0000 7fb802cca700 1 Processor -- start 2023-11-28T13:01:28.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.373+0000 7fb802cca700 1 -- start start 2023-11-28T13:01:28.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.373+0000 7fb802cca700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 0x7fb7fc0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.373+0000 7fb802cca700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb7fc0b2bc0 0x7fb7fc0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.373+0000 7fb802cca700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7fc0cd2a0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.373+0000 7fb802cca700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7fc0cba80 con 0x7fb7fc0b2bc0 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.373+0000 7fb802cca700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc0b3040 con 0x7fb7fc0cb630 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb7fbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 0x7fb7fc0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb801267700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb7fc0b2bc0 0x7fb7fc0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb801267700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb7fc0b2bc0 0x7fb7fc0ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:35442/0 (socket says 172.21.15.6:35442) 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb800a66700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fb7fc0b62d0 0x7fb7fc06f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:37302/0 (socket says 172.21.15.6:37302) 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb7fbfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 0x7fb7fc0b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:49658/0 (socket says 172.21.15.6:49658) 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb801267700 1 -- 172.21.15.6:0/2942800766 learned_addr learned my addr 172.21.15.6:0/2942800766 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:28.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb7fbfff700 1 -- 172.21.15.6:0/2942800766 >> v2:172.21.15.61:3300/0 conn(0x7fb7fc0b2bc0 msgr2=0x7fb7fc0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb7fbfff700 1 --2- 172.21.15.6:0/2942800766 >> v2:172.21.15.61:3300/0 conn(0x7fb7fc0b2bc0 0x7fb7fc0ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb7fbfff700 1 -- 172.21.15.6:0/2942800766 >> v1:172.21.15.61:6789/0 conn(0x7fb7fc0b62d0 legacy=0x7fb7fc06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb7fbfff700 1 -- 172.21.15.6:0/2942800766 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc0d0400 con 0x7fb7fc0cb630 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb7fb7fe700 1 -- 172.21.15.6:0/2942800766 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542817484 0 0) 0x7fb7fc0cd2a0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.374+0000 7fb801267700 1 --2- 172.21.15.6:0/2942800766 >> v2:172.21.15.61:3300/0 conn(0x7fb7fc0b2bc0 0x7fb7fc0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb7fbfff700 1 --2- 172.21.15.6:0/2942800766 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 0x7fb7fc0b2680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009840 tx=0x7fb7e8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baa6e2ea5a5e63ef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb7fb7fe700 1 -- 172.21.15.6:0/2942800766 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8011830 con 0x7fb7fc0cb630 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb7fb7fe700 1 -- 172.21.15.6:0/2942800766 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e8011990 con 0x7fb7fc0cb630 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb7fb7fe700 1 -- 172.21.15.6:0/2942800766 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8011d20 con 0x7fb7fc0cb630 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb802cca700 1 -- 172.21.15.6:0/2942800766 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 msgr2=0x7fb7fc0b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb802cca700 1 --2- 172.21.15.6:0/2942800766 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 0x7fb7fc0b2680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009840 tx=0x7fb7e8009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb802cca700 1 -- 172.21.15.6:0/2942800766 shutdown_connections 2023-11-28T13:01:28.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb802cca700 1 --2- 172.21.15.6:0/2942800766 >> v2:172.21.15.61:3300/0 conn(0x7fb7fc0b2bc0 0x7fb7fc0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb802cca700 1 --2- 172.21.15.6:0/2942800766 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 0x7fb7fc0b2680 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb802cca700 1 -- 172.21.15.6:0/2942800766 >> 172.21.15.6:0/2942800766 conn(0x7fb7fc0cd680 msgr2=0x7fb7fc0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb802cca700 1 -- 172.21.15.6:0/2942800766 shutdown_connections 2023-11-28T13:01:28.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.375+0000 7fb802cca700 1 -- 172.21.15.6:0/2942800766 wait complete. 2023-11-28T13:01:28.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.376+0000 7fb802cca700 1 Processor -- start 2023-11-28T13:01:28.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.377+0000 7fb802cca700 1 -- start start 2023-11-28T13:01:28.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.377+0000 7fb802cca700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7fc0b2bc0 0x7fb7fc06d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.377+0000 7fb802cca700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7fc0b62d0 0x7fb7fc0c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.377+0000 7fb802cca700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 0x7fb7fc0b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.377+0000 7fb802cca700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc157be0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.377+0000 7fb802cca700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc06dc10 con 0x7fb7fc0b2bc0 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.377+0000 7fb802cca700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc0c7cd0 con 0x7fb7fc0cb630 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7fbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7fc0b62d0 0x7fb7fc0c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7fbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7fc0b62d0 0x7fb7fc0c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59960/0 (socket says 172.21.15.6:59960) 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7fbfff700 1 -- 172.21.15.6:0/202072940 learned_addr learned my addr 172.21.15.6:0/202072940 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb800a66700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7fc0b2bc0 0x7fb7fc06d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb801267700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 0x7fb7fc0b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7fbfff700 1 -- 172.21.15.6:0/202072940 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 msgr2=0x7fb7fc0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7fbfff700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 0x7fb7fc0b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7fbfff700 1 -- 172.21.15.6:0/202072940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7fc0b2bc0 msgr2=0x7fb7fc06d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7fbfff700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7fc0b2bc0 0x7fb7fc06d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7fbfff700 1 -- 172.21.15.6:0/202072940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f4009b30 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7fbfff700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7fc0b62d0 0x7fb7fc0c9810 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009fd0 tx=0x7fb7e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7f97fa700 1 -- 172.21.15.6:0/202072940 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80036a0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7f97fa700 1 -- 172.21.15.6:0/202072940 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e801e070 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb7f97fa700 1 -- 172.21.15.6:0/202072940 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8003a40 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb802cca700 1 -- 172.21.15.6:0/202072940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e80094a0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.378+0000 7fb802cca700 1 -- 172.21.15.6:0/202072940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc0cffb0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.379+0000 7fb802cca700 1 -- 172.21.15.6:0/202072940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc0d06e0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.380+0000 7fb7f97fa700 1 -- 172.21.15.6:0/202072940 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb7e800f050 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.380+0000 7fb7f97fa700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7e00682f0 0x7fb7e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.380+0000 7fb7f97fa700 1 -- 172.21.15.6:0/202072940 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb7e80870b0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.380+0000 7fb800a66700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7e00682f0 0x7fb7e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.381+0000 7fb800a66700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7e00682f0 0x7fb7e006a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0000f60 tx=0x7fb7f0007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.384+0000 7fb7f97fa700 1 -- 172.21.15.6:0/202072940 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb7e80535e0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.577+0000 7fb802cca700 1 -- 172.21.15.6:0/202072940 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb7fc0b66a0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.577+0000 7fb7f97fa700 1 -- 172.21.15.6:0/202072940 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb7e80200a0 con 0x7fb7fc0b62d0 2023-11-28T13:01:28.579 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 -- 172.21.15.6:0/202072940 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7e00682f0 msgr2=0x7fb7e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7e00682f0 0x7fb7e006a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0000f60 tx=0x7fb7f0007c00 comp rx=0 tx=0).stop 2023-11-28T13:01:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 -- 172.21.15.6:0/202072940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7fc0b62d0 msgr2=0x7fb7fc0c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7fc0b62d0 0x7fb7fc0c9810 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009fd0 tx=0x7fb7e8004750 comp rx=0 tx=0).stop 2023-11-28T13:01:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 -- 172.21.15.6:0/202072940 shutdown_connections 2023-11-28T13:01:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb7e00682f0 0x7fb7e006a7a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb7fc0cb630 0x7fb7fc0b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7fc0b2bc0 0x7fb7fc06d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 --2- 172.21.15.6:0/202072940 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb7fc0b62d0 0x7fb7fc0c9810 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 -- 172.21.15.6:0/202072940 >> 172.21.15.6:0/202072940 conn(0x7fb7fc0cd680 msgr2=0x7fb7fc0b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 -- 172.21.15.6:0/202072940 shutdown_connections 2023-11-28T13:01:28.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.579+0000 7fb802cca700 1 -- 172.21.15.6:0/202072940 wait complete. 2023-11-28T13:01:28.593 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:28.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.717+0000 7f5d17f4b700 1 Processor -- start 2023-11-28T13:01:28.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.717+0000 7f5d17f4b700 1 -- start start 2023-11-28T13:01:28.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.718+0000 7f5d17f4b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5d100cc310 0x7f5d100cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.718+0000 7f5d17f4b700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d100ce340 con 0x7f5d1015c990 2023-11-28T13:01:28.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.718+0000 7f5d17f4b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d100cebe0 con 0x7f5d10161090 2023-11-28T13:01:28.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.718+0000 7f5d17f4b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d100cf780 con 0x7f5d100cc310 2023-11-28T13:01:28.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.718+0000 7f5d15ce7700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f5d1015c990 0x7f5d100cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47286/0 (socket says 172.21.15.6:47286) 2023-11-28T13:01:28.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.718+0000 7f5d15ce7700 1 -- 172.21.15.6:0/1499654925 learned_addr learned my addr 172.21.15.6:0/1499654925 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:28.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.718+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718384823 0 0) 0x7f5d100ce340 con 0x7f5d1015c990 2023-11-28T13:01:28.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.718+0000 7f5d164e8700 1 --2- 172.21.15.6:0/1499654925 >> v2:172.21.15.61:3300/0 conn(0x7f5d100cc310 0x7f5d100cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.718+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cfc0035b0 con 0x7f5d1015c990 2023-11-28T13:01:28.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3904288047 0 0) 0x7f5cfc0035b0 con 0x7f5d1015c990 2023-11-28T13:01:28.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 >> v2:172.21.15.61:3300/0 conn(0x7f5d100cc310 msgr2=0x7f5d100cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 --2- 172.21.15.6:0/1499654925 >> v2:172.21.15.61:3300/0 conn(0x7f5d100cc310 0x7f5d100cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 >> v1:172.21.15.61:6789/0 conn(0x7f5d10161090 legacy=0x7f5d100ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d100cccb0 con 0x7f5d1015c990 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f5d00001670 con 0x7f5d1015c990 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 >> v1:172.21.15.6:6789/0 conn(0x7f5d1015c990 legacy=0x7f5d100cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cfc003a00 0x7f5cfc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cfc004850 0x7f5cfc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5cfc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100cf780 con 0x7f5cfc005320 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc00c250 con 0x7f5cfc003a00 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc00c410 con 0x7f5cfc004850 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d164e8700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5cfc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d15ce7700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cfc003a00 0x7f5cfc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d154e6700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cfc004850 0x7f5cfc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d164e8700 1 -- 172.21.15.6:0/1499654925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cfc004850 msgr2=0x7f5cfc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d164e8700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cfc004850 0x7f5cfc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d164e8700 1 -- 172.21.15.6:0/1499654925 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cfc003a00 msgr2=0x7f5cfc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d164e8700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cfc003a00 0x7f5cfc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d164e8700 1 -- 172.21.15.6:0/1499654925 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0035b0 con 0x7f5cfc005320 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d154e6700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cfc004850 0x7f5cfc004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d164e8700 1 -- 172.21.15.6:0/1499654925 reap_dead start 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.719+0000 7f5d164e8700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5cfc00ba10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009fc0 tx=0x7f5cfc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7acb55918e34ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.720+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08003a20 con 0x7f5cfc005320 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.720+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0800f040 con 0x7f5cfc005320 2023-11-28T13:01:28.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.720+0000 7f5d14ce5700 1 -- 172.21.15.6:0/1499654925 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08003de0 con 0x7f5cfc005320 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.720+0000 7f5d17f4b700 1 -- 172.21.15.6:0/1499654925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 msgr2=0x7f5cfc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.720+0000 7f5d17f4b700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5cfc00ba10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009fc0 tx=0x7f5cfc005880 comp rx=0 tx=0).stop 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.720+0000 7f5d17f4b700 1 -- 172.21.15.6:0/1499654925 shutdown_connections 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.720+0000 7f5d17f4b700 1 --2- 172.21.15.6:0/1499654925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5cfc00ba10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.720+0000 7f5d17f4b700 1 -- 172.21.15.6:0/1499654925 >> 172.21.15.6:0/1499654925 conn(0x7f5d100b1650 msgr2=0x7f5d100b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.720+0000 7f5d17f4b700 1 -- 172.21.15.6:0/1499654925 shutdown_connections 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.720+0000 7f5d17f4b700 1 -- 172.21.15.6:0/1499654925 wait complete. 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.721+0000 7f5d17f4b700 1 Processor -- start 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.722+0000 7f5d17f4b700 1 -- start start 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.722+0000 7f5d17f4b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5d101538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.722+0000 7f5d17f4b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d10153e00 0x7f5d100c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.722+0000 7f5d17f4b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d100c5710 0x7f5d1015ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.722+0000 7f5d17f4b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100b2950 con 0x7f5cfc005320 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.722+0000 7f5d17f4b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100b4470 con 0x7f5d100c5710 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.722+0000 7f5d17f4b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100b4750 con 0x7f5d10153e00 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.722+0000 7f5d15ce7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5d101538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.722+0000 7f5d15ce7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5d101538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59976/0 (socket says 172.21.15.6:59976) 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.722+0000 7f5d15ce7700 1 -- 172.21.15.6:0/3262408689 learned_addr learned my addr 172.21.15.6:0/3262408689 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d164e8700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d100c5710 0x7f5d1015ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d15ce7700 1 -- 172.21.15.6:0/3262408689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d10153e00 msgr2=0x7f5d100c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d154e6700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d10153e00 0x7f5d100c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d15ce7700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d10153e00 0x7f5d100c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d15ce7700 1 -- 172.21.15.6:0/3262408689 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d100c5710 msgr2=0x7f5d1015ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d15ce7700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d100c5710 0x7f5d1015ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d15ce7700 1 -- 172.21.15.6:0/3262408689 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d100ce590 con 0x7f5cfc005320 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d15ce7700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5d101538c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5d000072c0 tx=0x7f5d00007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d067fc700 1 -- 172.21.15.6:0/3262408689 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000077a0 con 0x7f5cfc005320 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d067fc700 1 -- 172.21.15.6:0/3262408689 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0000f8d0 con 0x7f5cfc005320 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d17f4b700 1 -- 172.21.15.6:0/3262408689 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d100cb3f0 con 0x7f5cfc005320 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d067fc700 1 -- 172.21.15.6:0/3262408689 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000fc80 con 0x7f5cfc005320 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.723+0000 7f5d17f4b700 1 -- 172.21.15.6:0/3262408689 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d100ce870 con 0x7f5cfc005320 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.724+0000 7f5d17f4b700 1 -- 172.21.15.6:0/3262408689 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d100675a0 con 0x7f5cfc005320 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.724+0000 7f5d067fc700 1 -- 172.21.15.6:0/3262408689 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5d0001d460 con 0x7f5cfc005320 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.725+0000 7f5d067fc700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5cf40682f0 0x7f5cf406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.725+0000 7f5d067fc700 1 -- 172.21.15.6:0/3262408689 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5d00086040 con 0x7f5cfc005320 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.725+0000 7f5d154e6700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5cf40682f0 0x7f5cf406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:28.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.725+0000 7f5d154e6700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5cf40682f0 0x7f5cf406a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d100b40d0 tx=0x7f5d0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:28.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.729+0000 7f5d067fc700 1 -- 172.21.15.6:0/3262408689 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5d00052510 con 0x7f5cfc005320 2023-11-28T13:01:28.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.967+0000 7f5d17f4b700 1 -- 172.21.15.6:0/3262408689 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d100c6d10 con 0x7f5cfc005320 2023-11-28T13:01:28.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.968+0000 7f5d067fc700 1 -- 172.21.15.6:0/3262408689 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4638 (secure 0 0 0) 0x7f5d00055b30 con 0x7f5cfc005320 2023-11-28T13:01:28.969 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:28.969 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6834/1568686201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":1568686201},{"type":"v1","addr":"172.21.15.61:6834","nonce":1568686201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/3333487635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6833","nonce":3333487635},{"type":"v1","addr":"172.21.15.61:6835","nonce":3333487635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3917075730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3917075730},{"type":"v1","addr":"172.21.15.6:6835","nonce":3917075730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:21.404170+0000","modified":"2023-11-28T13:00:37.945716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/1668558371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1668558371},{"type":"v1","addr":"172.21.15.6:6837","nonce":1668558371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:01:28.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 -- 172.21.15.6:0/3262408689 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5cf40682f0 msgr2=0x7f5cf406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5cf40682f0 0x7f5cf406a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d100b40d0 tx=0x7f5d0c009040 comp rx=0 tx=0).stop 2023-11-28T13:01:28.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 -- 172.21.15.6:0/3262408689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 msgr2=0x7f5d101538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:28.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5d101538c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5d000072c0 tx=0x7f5d00007680 comp rx=0 tx=0).stop 2023-11-28T13:01:28.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 -- 172.21.15.6:0/3262408689 shutdown_connections 2023-11-28T13:01:28.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d100c5710 0x7f5d1015ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cfc005320 0x7f5d101538c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5cf40682f0 0x7f5cf406a7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 --2- 172.21.15.6:0/3262408689 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5d10153e00 0x7f5d100c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:28.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 -- 172.21.15.6:0/3262408689 >> 172.21.15.6:0/3262408689 conn(0x7f5d100b1650 msgr2=0x7f5d100c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:28.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 -- 172.21.15.6:0/3262408689 shutdown_connections 2023-11-28T13:01:28.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:28.970+0000 7f5d17f4b700 1 -- 172.21.15.6:0/3262408689 wait complete. 2023-11-28T13:01:28.973 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 9 2023-11-28T13:01:28.983 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:29.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.107+0000 7fdeb31a0700 1 Processor -- start 2023-11-28T13:01:29.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.107+0000 7fdeb31a0700 1 -- start start 2023-11-28T13:01:29.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.108+0000 7fdeb31a0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdeac161090 0x7fdeac0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.108+0000 7fdeb31a0700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac0ce340 con 0x7fdeac0caeb0 2023-11-28T13:01:29.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.108+0000 7fdeb31a0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac0cebe0 con 0x7fdeac15c990 2023-11-28T13:01:29.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.108+0000 7fdeb31a0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac0cf780 con 0x7fdeac161090 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.108+0000 7fdeb173d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fdeac0caeb0 0x7fdeac0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47308/0 (socket says 172.21.15.6:47308) 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.108+0000 7fdeb173d700 1 -- 172.21.15.6:0/383931929 learned_addr learned my addr 172.21.15.6:0/383931929 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.109+0000 7fdeabfff700 1 --2- 172.21.15.6:0/383931929 >> v2:172.21.15.61:3300/0 conn(0x7fdeac161090 0x7fdeac0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.109+0000 7fdeab7fe700 1 -- 172.21.15.6:0/383931929 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086921473 0 0) 0x7fdeac0ce340 con 0x7fdeac0caeb0 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.109+0000 7fdeab7fe700 1 -- 172.21.15.6:0/383931929 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde9c0035b0 con 0x7fdeac0caeb0 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.109+0000 7fdeabfff700 1 -- 172.21.15.6:0/383931929 >> v1:172.21.15.61:6789/0 conn(0x7fdeac15c990 legacy=0x7fdeac0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.109+0000 7fdeabfff700 1 -- 172.21.15.6:0/383931929 >> v1:172.21.15.6:6789/0 conn(0x7fdeac0caeb0 legacy=0x7fdeac0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.109+0000 7fdeabfff700 1 -- 172.21.15.6:0/383931929 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0cb3f0 con 0x7fdeac161090 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.109+0000 7fdeabfff700 1 --2- 172.21.15.6:0/383931929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdeac161090 0x7fdeac0ca970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde98009840 tx=0x7fde98009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20b1d3c472e36b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.109+0000 7fdeab7fe700 1 -- 172.21.15.6:0/383931929 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98011830 con 0x7fdeac161090 2023-11-28T13:01:29.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.109+0000 7fdeab7fe700 1 -- 172.21.15.6:0/383931929 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde98011990 con 0x7fdeac161090 2023-11-28T13:01:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.109+0000 7fdeab7fe700 1 -- 172.21.15.6:0/383931929 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98011d20 con 0x7fdeac161090 2023-11-28T13:01:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.110+0000 7fdeb31a0700 1 -- 172.21.15.6:0/383931929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdeac161090 msgr2=0x7fdeac0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.110+0000 7fdeb31a0700 1 --2- 172.21.15.6:0/383931929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdeac161090 0x7fdeac0ca970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde98009840 tx=0x7fde98009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.110+0000 7fdeb31a0700 1 -- 172.21.15.6:0/383931929 shutdown_connections 2023-11-28T13:01:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.110+0000 7fdeb31a0700 1 --2- 172.21.15.6:0/383931929 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdeac161090 0x7fdeac0ca970 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.110+0000 7fdeb31a0700 1 -- 172.21.15.6:0/383931929 >> 172.21.15.6:0/383931929 conn(0x7fdeac0b1650 msgr2=0x7fdeac0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.110+0000 7fdeb31a0700 1 -- 172.21.15.6:0/383931929 shutdown_connections 2023-11-28T13:01:29.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.110+0000 7fdeb31a0700 1 -- 172.21.15.6:0/383931929 wait complete. 2023-11-28T13:01:29.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.111+0000 7fdeb31a0700 1 Processor -- start 2023-11-28T13:01:29.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.111+0000 7fdeb31a0700 1 -- start start 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeb31a0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdeac0caeb0 0x7fdeac0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeb31a0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdeac15c990 0x7fdeac14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeb31a0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdeac161090 0x7fdeac0adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeb31a0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0c7580 con 0x7fdeac15c990 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeb31a0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0c8100 con 0x7fdeac161090 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeb31a0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0c77d0 con 0x7fdeac0caeb0 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeabfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdeac15c990 0x7fdeac14baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeabfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdeac15c990 0x7fdeac14baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:59992/0 (socket says 172.21.15.6:59992) 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeabfff700 1 -- 172.21.15.6:0/241715880 learned_addr learned my addr 172.21.15.6:0/241715880 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeb173d700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdeac161090 0x7fdeac0adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeb0f3c700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdeac0caeb0 0x7fdeac0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeabfff700 1 -- 172.21.15.6:0/241715880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdeac0caeb0 msgr2=0x7fdeac0c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeabfff700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdeac0caeb0 0x7fdeac0c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeabfff700 1 -- 172.21.15.6:0/241715880 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdeac161090 msgr2=0x7fdeac0adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeabfff700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdeac161090 0x7fdeac0adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.112+0000 7fdeabfff700 1 -- 172.21.15.6:0/241715880 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde980094a0 con 0x7fdeac15c990 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.113+0000 7fdeabfff700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdeac15c990 0x7fdeac14baa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fde98009fd0 tx=0x7fde98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.113+0000 7fdea97fa700 1 -- 172.21.15.6:0/241715880 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde980036a0 con 0x7fdeac15c990 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.113+0000 7fdea97fa700 1 -- 172.21.15.6:0/241715880 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9801f070 con 0x7fdeac15c990 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.113+0000 7fdeb31a0700 1 -- 172.21.15.6:0/241715880 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac0ce870 con 0x7fdeac15c990 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.113+0000 7fdea97fa700 1 -- 172.21.15.6:0/241715880 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98003a60 con 0x7fdeac15c990 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.113+0000 7fdeb31a0700 1 -- 172.21.15.6:0/241715880 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac0ce590 con 0x7fdeac15c990 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.114+0000 7fdeb31a0700 1 -- 172.21.15.6:0/241715880 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac0cca20 con 0x7fdeac15c990 2023-11-28T13:01:29.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.114+0000 7fdea97fa700 1 -- 172.21.15.6:0/241715880 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fde9800f050 con 0x7fdeac15c990 2023-11-28T13:01:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.115+0000 7fdea97fa700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fde900682f0 0x7fde9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.115+0000 7fdea97fa700 1 -- 172.21.15.6:0/241715880 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fde98021070 con 0x7fdeac15c990 2023-11-28T13:01:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.115+0000 7fdeb0f3c700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fde900682f0 0x7fde9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.115+0000 7fdeb0f3c700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fde900682f0 0x7fde9006a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdea0007fd0 tx=0x7fdea0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.121+0000 7fdea97fa700 1 -- 172.21.15.6:0/241715880 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fde98053b90 con 0x7fdeac15c990 2023-11-28T13:01:29.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.316+0000 7fdeb31a0700 1 -- 172.21.15.6:0/241715880 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdeac0675a0 con 0x7fdeac15c990 2023-11-28T13:01:29.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.318+0000 7fdea97fa700 1 -- 172.21.15.6:0/241715880 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19690 (secure 0 0 0) 0x7fde980571b0 con 0x7fdeac15c990 2023-11-28T13:01:29.319 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:29.319 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:10.053012+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:29.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 -- 172.21.15.6:0/241715880 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fde900682f0 msgr2=0x7fde9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fde900682f0 0x7fde9006a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdea0007fd0 tx=0x7fdea0009040 comp rx=0 tx=0).stop 2023-11-28T13:01:29.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 -- 172.21.15.6:0/241715880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdeac15c990 msgr2=0x7fdeac14baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdeac15c990 0x7fdeac14baa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fde98009fd0 tx=0x7fde98004750 comp rx=0 tx=0).stop 2023-11-28T13:01:29.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 -- 172.21.15.6:0/241715880 shutdown_connections 2023-11-28T13:01:29.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdeac161090 0x7fdeac0adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fde900682f0 0x7fde9006a7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdeac0caeb0 0x7fdeac0c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 --2- 172.21.15.6:0/241715880 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdeac15c990 0x7fdeac14baa0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 -- 172.21.15.6:0/241715880 >> 172.21.15.6:0/241715880 conn(0x7fdeac0b1650 msgr2=0x7fdeac06be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:29.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 -- 172.21.15.6:0/241715880 shutdown_connections 2023-11-28T13:01:29.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.320+0000 7fdeb31a0700 1 -- 172.21.15.6:0/241715880 wait complete. 2023-11-28T13:01:29.337 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:01:29.337 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:29.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.461+0000 7f73b2567700 1 Processor -- start 2023-11-28T13:01:29.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.461+0000 7f73b2567700 1 -- start start 2023-11-28T13:01:29.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.462+0000 7f73b2567700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 0x7f73ac0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.462+0000 7f73b2567700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f73ac0cecf0 0x7f73ac06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.462+0000 7f73b2567700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73ac0b30b0 con 0x7f73ac15c560 2023-11-28T13:01:29.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.462+0000 7f73b2567700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73ac06c020 con 0x7f73ac0cecf0 2023-11-28T13:01:29.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.462+0000 7f73b2567700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac0cf130 con 0x7f73ac0b2660 2023-11-28T13:01:29.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.462+0000 7f73b0b04700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f73ac0cecf0 0x7f73ac06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.462+0000 7f73b0b04700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f73ac0cecf0 0x7f73ac06c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60002/0 (socket says 172.21.15.6:60002) 2023-11-28T13:01:29.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.462+0000 7f73b0b04700 1 -- 172.21.15.6:0/3460185562 learned_addr learned my addr 172.21.15.6:0/3460185562 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:29.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.462+0000 7f73ab7fe700 1 --2- 172.21.15.6:0/3460185562 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 0x7f73ac0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.463+0000 7f73b0b04700 1 -- 172.21.15.6:0/3460185562 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 msgr2=0x7f73ac0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:29.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.463+0000 7f73b0b04700 1 --2- 172.21.15.6:0/3460185562 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 0x7f73ac0ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.463+0000 7f73b0b04700 1 -- 172.21.15.6:0/3460185562 >> v1:172.21.15.61:6789/0 conn(0x7f73ac15c560 legacy=0x7f73ac06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:29.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.463+0000 7f73b0b04700 1 -- 172.21.15.6:0/3460185562 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac0d09c0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.463+0000 7f73ab7fe700 1 --2- 172.21.15.6:0/3460185562 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 0x7f73ac0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:29.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.463+0000 7f73b0b04700 1 --2- 172.21.15.6:0/3460185562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 0x7f73ac06c4b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009e50 tx=0x7f73a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17970108972dffce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.463+0000 7f73aaffd700 1 -- 172.21.15.6:0/3460185562 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a400e070 con 0x7f73ac0cecf0 2023-11-28T13:01:29.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.463+0000 7f73aaffd700 1 -- 172.21.15.6:0/3460185562 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a4003a20 con 0x7f73ac0cecf0 2023-11-28T13:01:29.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.463+0000 7f73aaffd700 1 -- 172.21.15.6:0/3460185562 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4003dd0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.464+0000 7f73b2567700 1 -- 172.21.15.6:0/3460185562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 msgr2=0x7f73ac06c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.464+0000 7f73b2567700 1 --2- 172.21.15.6:0/3460185562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 0x7f73ac06c4b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009e50 tx=0x7f73a400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:29.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.464+0000 7f73b2567700 1 -- 172.21.15.6:0/3460185562 shutdown_connections 2023-11-28T13:01:29.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.464+0000 7f73b2567700 1 --2- 172.21.15.6:0/3460185562 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 0x7f73ac06c4b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.464+0000 7f73b2567700 1 --2- 172.21.15.6:0/3460185562 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 0x7f73ac0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.464+0000 7f73b2567700 1 -- 172.21.15.6:0/3460185562 >> 172.21.15.6:0/3460185562 conn(0x7f73ac0cd660 msgr2=0x7f73ac0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:29.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.464+0000 7f73b2567700 1 -- 172.21.15.6:0/3460185562 shutdown_connections 2023-11-28T13:01:29.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.464+0000 7f73b2567700 1 -- 172.21.15.6:0/3460185562 wait complete. 2023-11-28T13:01:29.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.465+0000 7f73b2567700 1 Processor -- start 2023-11-28T13:01:29.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.465+0000 7f73b2567700 1 -- start start 2023-11-28T13:01:29.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73b2567700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 0x7f73ac0af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73b2567700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 0x7f73ac15c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73b2567700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f73ac15c560 0x7f73ac1f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73b2567700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac1f31e0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73b2567700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac15cff0 con 0x7f73ac15c560 2023-11-28T13:01:29.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73b2567700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac15d2d0 con 0x7f73ac0b2660 2023-11-28T13:01:29.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73ab7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 0x7f73ac15c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73ab7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 0x7f73ac15c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60006/0 (socket says 172.21.15.6:60006) 2023-11-28T13:01:29.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73ab7fe700 1 -- 172.21.15.6:0/2448936378 learned_addr learned my addr 172.21.15.6:0/2448936378 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:29.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73b0b04700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f73ac15c560 0x7f73ac1f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.466+0000 7f73abfff700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 0x7f73ac0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73ab7fe700 1 -- 172.21.15.6:0/2448936378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 msgr2=0x7f73ac0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73ab7fe700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 0x7f73ac0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73ab7fe700 1 -- 172.21.15.6:0/2448936378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f73ac15c560 msgr2=0x7f73ac1f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73ab7fe700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f73ac15c560 0x7f73ac1f2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73ab7fe700 1 -- 172.21.15.6:0/2448936378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a4009b30 con 0x7f73ac0cecf0 2023-11-28T13:01:29.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73ab7fe700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 0x7f73ac15c930 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f73a000e810 tx=0x7f73a000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73a8ff9700 1 -- 172.21.15.6:0/2448936378 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0004230 con 0x7f73ac0cecf0 2023-11-28T13:01:29.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73a8ff9700 1 -- 172.21.15.6:0/2448936378 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a00043b0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73b2567700 1 -- 172.21.15.6:0/2448936378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac0b2dd0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73a8ff9700 1 -- 172.21.15.6:0/2448936378 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a00038e0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.467+0000 7f73b2567700 1 -- 172.21.15.6:0/2448936378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac0b2af0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.469+0000 7f73b2567700 1 -- 172.21.15.6:0/2448936378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac06bcb0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.469+0000 7f73a8ff9700 1 -- 172.21.15.6:0/2448936378 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f73a0022070 con 0x7f73ac0cecf0 2023-11-28T13:01:29.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.470+0000 7f73a8ff9700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f73980682f0 0x7f739806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.470+0000 7f73a8ff9700 1 -- 172.21.15.6:0/2448936378 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f73a00859c0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.470+0000 7f73abfff700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f73980682f0 0x7f739806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.470+0000 7f73abfff700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f73980682f0 0x7f739806a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f739c009870 tx=0x7f739c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.474+0000 7f73a8ff9700 1 -- 172.21.15.6:0/2448936378 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f73a0051ef0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.712+0000 7f73b2567700 1 -- 172.21.15.6:0/2448936378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73ac0675a0 con 0x7f73ac0cecf0 2023-11-28T13:01:29.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.713+0000 7f73a8ff9700 1 -- 172.21.15.6:0/2448936378 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4638 (secure 0 0 0) 0x7f73a000f040 con 0x7f73ac0cecf0 2023-11-28T13:01:29.714 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:29.714 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6834/1568686201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":1568686201},{"type":"v1","addr":"172.21.15.61:6834","nonce":1568686201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/3333487635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6833","nonce":3333487635},{"type":"v1","addr":"172.21.15.61:6835","nonce":3333487635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3917075730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3917075730},{"type":"v1","addr":"172.21.15.6:6835","nonce":3917075730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:21.404170+0000","modified":"2023-11-28T13:00:37.945716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/1668558371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1668558371},{"type":"v1","addr":"172.21.15.6:6837","nonce":1668558371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:01:29.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 -- 172.21.15.6:0/2448936378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f73980682f0 msgr2=0x7f739806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f73980682f0 0x7f739806a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f739c009870 tx=0x7f739c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:29.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 -- 172.21.15.6:0/2448936378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 msgr2=0x7f73ac15c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 0x7f73ac15c930 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f73a000e810 tx=0x7f73a000eb20 comp rx=0 tx=0).stop 2023-11-28T13:01:29.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 -- 172.21.15.6:0/2448936378 shutdown_connections 2023-11-28T13:01:29.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f73ac15c560 0x7f73ac1f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f73980682f0 0x7f739806a7a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f73ac0b2660 0x7f73ac0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 --2- 172.21.15.6:0/2448936378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f73ac0cecf0 0x7f73ac15c930 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 -- 172.21.15.6:0/2448936378 >> 172.21.15.6:0/2448936378 conn(0x7f73ac0cd660 msgr2=0x7f73ac0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:29.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 -- 172.21.15.6:0/2448936378 shutdown_connections 2023-11-28T13:01:29.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.715+0000 7f73b2567700 1 -- 172.21.15.6:0/2448936378 wait complete. 2023-11-28T13:01:29.718 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 9 2023-11-28T13:01:29.731 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:29.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.854+0000 7f8d9ae7c700 1 Processor -- start 2023-11-28T13:01:29.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.854+0000 7f8d9ae7c700 1 -- start start 2023-11-28T13:01:29.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.855+0000 7f8d9ae7c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8d9406d9d0 0x7f8d9415b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.855+0000 7f8d9ae7c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d9406dda0 0x7f8d9415e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.855+0000 7f8d9ae7c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9406c3a0 con 0x7f8d9406bc30 2023-11-28T13:01:29.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.855+0000 7f8d9ae7c700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9415ee90 con 0x7f8d9406d9d0 2023-11-28T13:01:29.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.855+0000 7f8d9ae7c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9406c0c0 con 0x7f8d9406dda0 2023-11-28T13:01:29.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.855+0000 7f8d98c18700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8d9406d9d0 0x7f8d9415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.855+0000 7f8d98c18700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8d9406d9d0 0x7f8d9415b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60012/0 (socket says 172.21.15.6:60012) 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.855+0000 7f8d98c18700 1 -- 172.21.15.6:0/771312001 learned_addr learned my addr 172.21.15.6:0/771312001 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.855+0000 7f8d99419700 1 --2- 172.21.15.6:0/771312001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d9406dda0 0x7f8d9415e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.856+0000 7f8d98c18700 1 -- 172.21.15.6:0/771312001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d9406dda0 msgr2=0x7f8d9415e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.856+0000 7f8d98c18700 1 --2- 172.21.15.6:0/771312001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d9406dda0 0x7f8d9415e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.856+0000 7f8d98c18700 1 -- 172.21.15.6:0/771312001 >> v1:172.21.15.61:6789/0 conn(0x7f8d9406bc30 legacy=0x7f8d94150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.856+0000 7f8d98c18700 1 -- 172.21.15.6:0/771312001 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9415ab60 con 0x7f8d9406d9d0 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.856+0000 7f8d98c18700 1 --2- 172.21.15.6:0/771312001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406d9d0 0x7f8d9415b640 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8d880098f0 tx=0x7f8d88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8ca22bc3a6e5cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.856+0000 7f8d937fe700 1 -- 172.21.15.6:0/771312001 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88011830 con 0x7f8d9406d9d0 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.856+0000 7f8d937fe700 1 -- 172.21.15.6:0/771312001 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d88011990 con 0x7f8d9406d9d0 2023-11-28T13:01:29.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.856+0000 7f8d937fe700 1 -- 172.21.15.6:0/771312001 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88011d20 con 0x7f8d9406d9d0 2023-11-28T13:01:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.857+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/771312001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406d9d0 msgr2=0x7f8d9415b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.857+0000 7f8d9ae7c700 1 --2- 172.21.15.6:0/771312001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406d9d0 0x7f8d9415b640 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8d880098f0 tx=0x7f8d88009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.857+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/771312001 shutdown_connections 2023-11-28T13:01:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.857+0000 7f8d9ae7c700 1 --2- 172.21.15.6:0/771312001 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d9406dda0 0x7f8d9415e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.857+0000 7f8d9ae7c700 1 --2- 172.21.15.6:0/771312001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406d9d0 0x7f8d9415b640 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.857+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/771312001 >> 172.21.15.6:0/771312001 conn(0x7f8d940b0dd0 msgr2=0x7f8d940b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.857+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/771312001 shutdown_connections 2023-11-28T13:01:29.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.857+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/771312001 wait complete. 2023-11-28T13:01:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.858+0000 7f8d9ae7c700 1 Processor -- start 2023-11-28T13:01:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.858+0000 7f8d9ae7c700 1 -- start start 2023-11-28T13:01:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d9ae7c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d9406bc30 0x7f8d9414c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d9ae7c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d9406d9d0 0x7f8d940b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d9ae7c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406dda0 0x7f8d941559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d9ae7c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d94156840 con 0x7f8d9406dda0 2023-11-28T13:01:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d9ae7c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d94156b20 con 0x7f8d9406bc30 2023-11-28T13:01:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d9ae7c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d941551b0 con 0x7f8d9406d9d0 2023-11-28T13:01:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d99419700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406dda0 0x7f8d941559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d99419700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406dda0 0x7f8d941559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60020/0 (socket says 172.21.15.6:60020) 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d99419700 1 -- 172.21.15.6:0/1014573437 learned_addr learned my addr 172.21.15.6:0/1014573437 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d98c18700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d9406bc30 0x7f8d9414c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d93fff700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d9406d9d0 0x7f8d940b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d99419700 1 -- 172.21.15.6:0/1014573437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d9406d9d0 msgr2=0x7f8d940b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d99419700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d9406d9d0 0x7f8d940b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d99419700 1 -- 172.21.15.6:0/1014573437 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d9406bc30 msgr2=0x7f8d9414c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d99419700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d9406bc30 0x7f8d9414c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d99419700 1 -- 172.21.15.6:0/1014573437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d88009550 con 0x7f8d9406dda0 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.859+0000 7f8d99419700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406dda0 0x7f8d941559b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00efc0 tx=0x7f8d8c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.860+0000 7f8d917fa700 1 -- 172.21.15.6:0/1014573437 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00cad0 con 0x7f8d9406dda0 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.860+0000 7f8d917fa700 1 -- 172.21.15.6:0/1014573437 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c00f040 con 0x7f8d9406dda0 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.860+0000 7f8d917fa700 1 -- 172.21.15.6:0/1014573437 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00ce70 con 0x7f8d9406dda0 2023-11-28T13:01:29.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.860+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/1014573437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9415b120 con 0x7f8d9406dda0 2023-11-28T13:01:29.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.860+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/1014573437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9415ae40 con 0x7f8d9406dda0 2023-11-28T13:01:29.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.861+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/1014573437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9406fac0 con 0x7f8d9406dda0 2023-11-28T13:01:29.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.861+0000 7f8d917fa700 1 -- 172.21.15.6:0/1014573437 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8d8c014480 con 0x7f8d9406dda0 2023-11-28T13:01:29.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.861+0000 7f8d917fa700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d780682f0 0x7f8d7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:29.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.861+0000 7f8d917fa700 1 -- 172.21.15.6:0/1014573437 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8d8c086b20 con 0x7f8d9406dda0 2023-11-28T13:01:29.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.862+0000 7f8d98c18700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d780682f0 0x7f8d7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:29.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.862+0000 7f8d98c18700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d780682f0 0x7f8d7806a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009c40 tx=0x7f8d88008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:29.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:29.866+0000 7f8d917fa700 1 -- 172.21.15.6:0/1014573437 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8d8c052ff0 con 0x7f8d9406dda0 2023-11-28T13:01:30.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.102+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/1014573437 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d941556a0 con 0x7f8d9406dda0 2023-11-28T13:01:30.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.102+0000 7f8d917fa700 1 -- 172.21.15.6:0/1014573437 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4638 (secure 0 0 0) 0x7f8d8c056610 con 0x7f8d9406dda0 2023-11-28T13:01:30.104 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:30.104 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6834/1568686201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":1568686201},{"type":"v1","addr":"172.21.15.61:6834","nonce":1568686201}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/3333487635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6833","nonce":3333487635},{"type":"v1","addr":"172.21.15.61:6835","nonce":3333487635}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3917075730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3917075730},{"type":"v1","addr":"172.21.15.6:6835","nonce":3917075730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:00:21.404170+0000","modified":"2023-11-28T13:00:37.945716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/1668558371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6836","nonce":1668558371},{"type":"v1","addr":"172.21.15.6:6837","nonce":1668558371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T13:01:30.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/1014573437 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d780682f0 msgr2=0x7f8d7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d780682f0 0x7f8d7806a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009c40 tx=0x7f8d88008030 comp rx=0 tx=0).stop 2023-11-28T13:01:30.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/1014573437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406dda0 msgr2=0x7f8d941559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406dda0 0x7f8d941559b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00efc0 tx=0x7f8d8c00c0c0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/1014573437 shutdown_connections 2023-11-28T13:01:30.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8d9406dda0 0x7f8d941559b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d9406bc30 0x7f8d9414c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8d780682f0 0x7f8d7806a7a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 --2- 172.21.15.6:0/1014573437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8d9406d9d0 0x7f8d940b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/1014573437 >> 172.21.15.6:0/1014573437 conn(0x7f8d940b0dd0 msgr2=0x7f8d94158690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:30.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.104+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/1014573437 shutdown_connections 2023-11-28T13:01:30.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.105+0000 7f8d9ae7c700 1 -- 172.21.15.6:0/1014573437 wait complete. 2023-11-28T13:01:30.107 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 9 2023-11-28T13:01:30.118 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:30.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.242+0000 7fcfba870700 1 Processor -- start 2023-11-28T13:01:30.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.242+0000 7fcfba870700 1 -- start start 2023-11-28T13:01:30.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.242+0000 7fcfba870700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcfb415c990 0x7fcfb40cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.242+0000 7fcfba870700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb40ce340 con 0x7fcfb4161090 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfba870700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb40cebe0 con 0x7fcfb40cc310 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfba870700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb40cb3f0 con 0x7fcfb415c990 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb3fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcfb415c990 0x7fcfb40cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb3fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcfb415c990 0x7fcfb40cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60024/0 (socket says 172.21.15.6:60024) 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb3fff700 1 -- 172.21.15.6:0/307843899 learned_addr learned my addr 172.21.15.6:0/307843899 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb37fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fcfb4161090 0x7fcfb40ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47346/0 (socket says 172.21.15.6:47346) 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb3fff700 1 -- 172.21.15.6:0/307843899 >> v1:172.21.15.61:6789/0 conn(0x7fcfb40cc310 legacy=0x7fcfb40cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb3fff700 1 -- 172.21.15.6:0/307843899 >> v1:172.21.15.6:6789/0 conn(0x7fcfb4161090 legacy=0x7fcfb40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb3fff700 1 -- 172.21.15.6:0/307843899 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40ce590 con 0x7fcfb415c990 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb2ffd700 1 -- 172.21.15.6:0/307843899 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365064713 0 0) 0x7fcfb40ce340 con 0x7fcfb4161090 2023-11-28T13:01:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb3fff700 1 --2- 172.21.15.6:0/307843899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 0x7fcfb40cf070 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40098f0 tx=0x7fcfa4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de32c5ae35fd0cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb2ffd700 1 -- 172.21.15.6:0/307843899 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4004230 con 0x7fcfb415c990 2023-11-28T13:01:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb2ffd700 1 -- 172.21.15.6:0/307843899 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa4004390 con 0x7fcfb415c990 2023-11-28T13:01:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.243+0000 7fcfb2ffd700 1 -- 172.21.15.6:0/307843899 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa40036a0 con 0x7fcfb415c990 2023-11-28T13:01:30.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.244+0000 7fcfba870700 1 -- 172.21.15.6:0/307843899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 msgr2=0x7fcfb40cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.244+0000 7fcfba870700 1 --2- 172.21.15.6:0/307843899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 0x7fcfb40cf070 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40098f0 tx=0x7fcfa4009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.244+0000 7fcfba870700 1 -- 172.21.15.6:0/307843899 shutdown_connections 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.244+0000 7fcfba870700 1 --2- 172.21.15.6:0/307843899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 0x7fcfb40cf070 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.244+0000 7fcfba870700 1 -- 172.21.15.6:0/307843899 >> 172.21.15.6:0/307843899 conn(0x7fcfb40b1650 msgr2=0x7fcfb40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.244+0000 7fcfba870700 1 -- 172.21.15.6:0/307843899 shutdown_connections 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.244+0000 7fcfba870700 1 -- 172.21.15.6:0/307843899 wait complete. 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.245+0000 7fcfba870700 1 Processor -- start 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.245+0000 7fcfba870700 1 -- start start 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfba870700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfb40cc310 0x7fcfb415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfba870700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 0x7fcfb40c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfba870700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfb4161090 0x7fcfb415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfba870700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb41604c0 con 0x7fcfb415c990 2023-11-28T13:01:30.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfba870700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb415eee0 con 0x7fcfb40cc310 2023-11-28T13:01:30.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfba870700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb415f1c0 con 0x7fcfb4161090 2023-11-28T13:01:30.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 0x7fcfb40c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 0x7fcfb40c59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60032/0 (socket says 172.21.15.6:60032) 2023-11-28T13:01:30.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb37fe700 1 -- 172.21.15.6:0/3579428074 learned_addr learned my addr 172.21.15.6:0/3579428074 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:30.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb3fff700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfb40cc310 0x7fcfb415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb8e0d700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfb4161090 0x7fcfb415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb37fe700 1 -- 172.21.15.6:0/3579428074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfb4161090 msgr2=0x7fcfb415e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb37fe700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfb4161090 0x7fcfb415e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb37fe700 1 -- 172.21.15.6:0/3579428074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfb40cc310 msgr2=0x7fcfb415ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb37fe700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfb40cc310 0x7fcfb415ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb37fe700 1 -- 172.21.15.6:0/3579428074 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa4009550 con 0x7fcfb415c990 2023-11-28T13:01:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.246+0000 7fcfb37fe700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 0x7fcfb40c59a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80011e0 tx=0x7fcfa800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.247+0000 7fcfb0ff9700 1 -- 172.21.15.6:0/3579428074 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8004c80 con 0x7fcfb415c990 2023-11-28T13:01:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.247+0000 7fcfb0ff9700 1 -- 172.21.15.6:0/3579428074 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8004de0 con 0x7fcfb415c990 2023-11-28T13:01:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.247+0000 7fcfb0ff9700 1 -- 172.21.15.6:0/3579428074 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8007310 con 0x7fcfb415c990 2023-11-28T13:01:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.247+0000 7fcfba870700 1 -- 172.21.15.6:0/3579428074 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb40cb080 con 0x7fcfb415c990 2023-11-28T13:01:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.247+0000 7fcfba870700 1 -- 172.21.15.6:0/3579428074 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb40ce870 con 0x7fcfb415c990 2023-11-28T13:01:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.248+0000 7fcfba870700 1 -- 172.21.15.6:0/3579428074 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb40cf5b0 con 0x7fcfb415c990 2023-11-28T13:01:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.248+0000 7fcfb0ff9700 1 -- 172.21.15.6:0/3579428074 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcfa801a020 con 0x7fcfb415c990 2023-11-28T13:01:30.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.248+0000 7fcfb0ff9700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcfa0068320 0x7fcfa006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.249+0000 7fcfb0ff9700 1 -- 172.21.15.6:0/3579428074 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcfa8085b70 con 0x7fcfb415c990 2023-11-28T13:01:30.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.249+0000 7fcfb3fff700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcfa0068320 0x7fcfa006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.249+0000 7fcfb3fff700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcfa0068320 0x7fcfa006a7d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009130 tx=0x7fcfa4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:30.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.253+0000 7fcfb0ff9700 1 -- 172.21.15.6:0/3579428074 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcfa8052150 con 0x7fcfb415c990 2023-11-28T13:01:30.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.447+0000 7fcfba870700 1 -- 172.21.15.6:0/3579428074 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfb4067150 con 0x7fcfb415c990 2023-11-28T13:01:30.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.448+0000 7fcfb0ff9700 1 -- 172.21.15.6:0/3579428074 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19690 (secure 0 0 0) 0x7fcfa8055770 con 0x7fcfb415c990 2023-11-28T13:01:30.450 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:30.450 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":27,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:10.053012+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:00:19.218196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T13:00:20.374831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T13:00:23.460601+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:30.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 -- 172.21.15.6:0/3579428074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcfa0068320 msgr2=0x7fcfa006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcfa0068320 0x7fcfa006a7d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009130 tx=0x7fcfa4008030 comp rx=0 tx=0).stop 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 -- 172.21.15.6:0/3579428074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 msgr2=0x7fcfb40c59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 0x7fcfb40c59a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80011e0 tx=0x7fcfa800ca30 comp rx=0 tx=0).stop 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 -- 172.21.15.6:0/3579428074 shutdown_connections 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcfa0068320 0x7fcfa006a7d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcfb4161090 0x7fcfb415e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcfb40cc310 0x7fcfb415ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 --2- 172.21.15.6:0/3579428074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcfb415c990 0x7fcfb40c59a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 -- 172.21.15.6:0/3579428074 >> 172.21.15.6:0/3579428074 conn(0x7fcfb40b1650 msgr2=0x7fcfb40cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 -- 172.21.15.6:0/3579428074 shutdown_connections 2023-11-28T13:01:30.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.450+0000 7fcfba870700 1 -- 172.21.15.6:0/3579428074 wait complete. 2023-11-28T13:01:30.464 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:01:30.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.588+0000 7fce9d080700 1 Processor -- start 2023-11-28T13:01:30.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.588+0000 7fce9d080700 1 -- start start 2023-11-28T13:01:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9d080700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fce98161090 0x7fce980ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9d080700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fce980caeb0 0x7fce980cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9d080700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce980ce340 con 0x7fce9815c990 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9d080700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce980cebe0 con 0x7fce980caeb0 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9d080700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce980cf780 con 0x7fce98161090 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9759e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fce980caeb0 0x7fce980cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9759e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fce980caeb0 0x7fce980cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60042/0 (socket says 172.21.15.6:60042) 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9759e700 1 -- 172.21.15.6:0/4171282147 learned_addr learned my addr 172.21.15.6:0/4171282147 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9659c700 1 --2- 172.21.15.6:0/4171282147 >> v2:172.21.15.61:3300/0 conn(0x7fce98161090 0x7fce980ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9759e700 1 -- 172.21.15.6:0/4171282147 >> v2:172.21.15.61:3300/0 conn(0x7fce98161090 msgr2=0x7fce980ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9759e700 1 --2- 172.21.15.6:0/4171282147 >> v2:172.21.15.61:3300/0 conn(0x7fce98161090 0x7fce980ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9759e700 1 -- 172.21.15.6:0/4171282147 >> v1:172.21.15.61:6789/0 conn(0x7fce9815c990 legacy=0x7fce980cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.589+0000 7fce9759e700 1 -- 172.21.15.6:0/4171282147 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce980cb3f0 con 0x7fce980caeb0 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce9759e700 1 --2- 172.21.15.6:0/4171282147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce980caeb0 0x7fce980cc310 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fce9806ab20 tx=0x7fce9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf8640de502d5b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce95d9b700 1 -- 172.21.15.6:0/4171282147 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9000e070 con 0x7fce980caeb0 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce95d9b700 1 -- 172.21.15.6:0/4171282147 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9000b910 con 0x7fce980caeb0 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce95d9b700 1 -- 172.21.15.6:0/4171282147 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9000bcc0 con 0x7fce980caeb0 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce9d080700 1 -- 172.21.15.6:0/4171282147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce980caeb0 msgr2=0x7fce980cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce9d080700 1 --2- 172.21.15.6:0/4171282147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce980caeb0 0x7fce980cc310 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fce9806ab20 tx=0x7fce9000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce9d080700 1 -- 172.21.15.6:0/4171282147 shutdown_connections 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce9d080700 1 --2- 172.21.15.6:0/4171282147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce980caeb0 0x7fce980cc310 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce9d080700 1 --2- 172.21.15.6:0/4171282147 >> v2:172.21.15.61:3300/0 conn(0x7fce98161090 0x7fce980ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce9d080700 1 -- 172.21.15.6:0/4171282147 >> 172.21.15.6:0/4171282147 conn(0x7fce980b1650 msgr2=0x7fce980b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce9d080700 1 -- 172.21.15.6:0/4171282147 shutdown_connections 2023-11-28T13:01:30.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.590+0000 7fce9d080700 1 -- 172.21.15.6:0/4171282147 wait complete. 2023-11-28T13:01:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.592+0000 7fce9d080700 1 Processor -- start 2023-11-28T13:01:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.592+0000 7fce9d080700 1 -- start start 2023-11-28T13:01:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.592+0000 7fce9d080700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce980caeb0 0x7fce980b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.592+0000 7fce9d080700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce9815c990 0x7fce9814e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.592+0000 7fce9d080700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98161090 0x7fce9815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9d080700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce980c3770 con 0x7fce98161090 2023-11-28T13:01:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9d080700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce980c31f0 con 0x7fce9815c990 2023-11-28T13:01:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9d080700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce980c4bd0 con 0x7fce980caeb0 2023-11-28T13:01:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9759e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98161090 0x7fce9815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9759e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98161090 0x7fce9815ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60048/0 (socket says 172.21.15.6:60048) 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9759e700 1 -- 172.21.15.6:0/727280583 learned_addr learned my addr 172.21.15.6:0/727280583 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9659c700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce9815c990 0x7fce9814e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce96d9d700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce980caeb0 0x7fce980b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9759e700 1 -- 172.21.15.6:0/727280583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce980caeb0 msgr2=0x7fce980b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9759e700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce980caeb0 0x7fce980b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9759e700 1 -- 172.21.15.6:0/727280583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce9815c990 msgr2=0x7fce9814e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9759e700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce9815c990 0x7fce9814e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9759e700 1 -- 172.21.15.6:0/727280583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90009b30 con 0x7fce98161090 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9659c700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce9815c990 0x7fce9814e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9759e700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98161090 0x7fce9815ddb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fce9000b370 tx=0x7fce90004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce7f7fe700 1 -- 172.21.15.6:0/727280583 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce900047d0 con 0x7fce98161090 2023-11-28T13:01:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce7f7fe700 1 -- 172.21.15.6:0/727280583 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce90004930 con 0x7fce98161090 2023-11-28T13:01:30.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce7f7fe700 1 -- 172.21.15.6:0/727280583 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9001c680 con 0x7fce98161090 2023-11-28T13:01:30.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9d080700 1 -- 172.21.15.6:0/727280583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce980ce870 con 0x7fce98161090 2023-11-28T13:01:30.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.593+0000 7fce9d080700 1 -- 172.21.15.6:0/727280583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce980ce590 con 0x7fce98161090 2023-11-28T13:01:30.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.594+0000 7fce9d080700 1 -- 172.21.15.6:0/727280583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce980cc850 con 0x7fce98161090 2023-11-28T13:01:30.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.595+0000 7fce7f7fe700 1 -- 172.21.15.6:0/727280583 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fce90022030 con 0x7fce98161090 2023-11-28T13:01:30.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.595+0000 7fce7f7fe700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce840682f0 0x7fce8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:30.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.595+0000 7fce7f7fe700 1 -- 172.21.15.6:0/727280583 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fce90087850 con 0x7fce98161090 2023-11-28T13:01:30.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.595+0000 7fce96d9d700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce840682f0 0x7fce8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:30.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.596+0000 7fce96d9d700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce840682f0 0x7fce8406a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fce8c000f60 tx=0x7fce8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:30.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.599+0000 7fce7f7fe700 1 -- 172.21.15.6:0/727280583 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fce90053d00 con 0x7fce98161090 2023-11-28T13:01:30.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:30.839+0000 7fce9d080700 1 -- 172.21.15.6:0/727280583 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fce980675a0 con 0x7fce98161090 2023-11-28T13:01:31.124 INFO:tasks.ceph.mon.a.smithi006.stderr:2023-11-28T13:01:31.123+0000 7fa6275a1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:01:31.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.135+0000 7fce7f7fe700 1 -- 172.21.15.6:0/727280583 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fce90057320 con 0x7fce98161090 2023-11-28T13:01:31.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 -- 172.21.15.6:0/727280583 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce840682f0 msgr2=0x7fce8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce840682f0 0x7fce8406a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fce8c000f60 tx=0x7fce8c009040 comp rx=0 tx=0).stop 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 -- 172.21.15.6:0/727280583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98161090 msgr2=0x7fce9815ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98161090 0x7fce9815ddb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fce9000b370 tx=0x7fce90004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 -- 172.21.15.6:0/727280583 shutdown_connections 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98161090 0x7fce9815ddb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce840682f0 0x7fce8406a7a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce980caeb0 0x7fce980b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 --2- 172.21.15.6:0/727280583 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce9815c990 0x7fce9814e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 -- 172.21.15.6:0/727280583 >> 172.21.15.6:0/727280583 conn(0x7fce980b1650 msgr2=0x7fce9806d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 -- 172.21.15.6:0/727280583 shutdown_connections 2023-11-28T13:01:31.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.138+0000 7fce9d080700 1 -- 172.21.15.6:0/727280583 wait complete. 2023-11-28T13:01:31.141 INFO:tasks.ceph.mds.c.smithi006.stderr: -5056> 2023-11-28T13:00:26.079+0000 7fe43c474700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:01:31.141 INFO:teuthology.orchestra.run.smithi006.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:01:31.153 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:01:31.172 INFO:tasks.ceph.mds.c.smithi006.stdout:starting mds.c at 2023-11-28T13:01:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.287+0000 7fcbdaf89700 1 Processor -- start 2023-11-28T13:01:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.287+0000 7fcbdaf89700 1 -- start start 2023-11-28T13:01:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbdaf89700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcbd40b1630 0x7fcbd40cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbdaf89700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcbd40b1a00 0x7fcbd40caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbdaf89700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd40ce550 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbdaf89700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd40ce830 con 0x7fcbd40b1630 2023-11-28T13:01:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbdaf89700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd40ceb10 con 0x7fcbd40b1a00 2023-11-28T13:01:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbd8d25700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcbd40b1630 0x7fcbd40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbd9526700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fcbd40cb3e0 0x7fcbd4160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47388/0 (socket says 172.21.15.6:47388) 2023-11-28T13:01:31.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbd9526700 1 -- 172.21.15.6:0/2181844172 learned_addr learned my addr 172.21.15.6:0/2181844172 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbd8d25700 1 -- 172.21.15.6:0/2181844172 >> v2:172.21.15.61:3300/0 conn(0x7fcbd40b1a00 msgr2=0x7fcbd40caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbd3fff700 1 --2- 172.21.15.6:0/2181844172 >> v2:172.21.15.61:3300/0 conn(0x7fcbd40b1a00 0x7fcbd40caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbd8d25700 1 --2- 172.21.15.6:0/2181844172 >> v2:172.21.15.61:3300/0 conn(0x7fcbd40b1a00 0x7fcbd40caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbd37fe700 1 -- 172.21.15.6:0/2181844172 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128958260 0 0) 0x7fcbd40ce550 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbd8d25700 1 -- 172.21.15.6:0/2181844172 >> v1:172.21.15.6:6789/0 conn(0x7fcbd40cb3e0 legacy=0x7fcbd4160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.288+0000 7fcbd8d25700 1 -- 172.21.15.6:0/2181844172 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd40cedf0 con 0x7fcbd40b1630 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbd8d25700 1 --2- 172.21.15.6:0/2181844172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40b1630 0x7fcbd40cf5f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009840 tx=0x7fcbc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f007a6434d2027 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbd37fe700 1 -- 172.21.15.6:0/2181844172 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8011830 con 0x7fcbd40b1630 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbd37fe700 1 -- 172.21.15.6:0/2181844172 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc8011990 con 0x7fcbd40b1630 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbd37fe700 1 -- 172.21.15.6:0/2181844172 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8011d20 con 0x7fcbd40b1630 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbdaf89700 1 -- 172.21.15.6:0/2181844172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40b1630 msgr2=0x7fcbd40cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:31.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbdaf89700 1 --2- 172.21.15.6:0/2181844172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40b1630 0x7fcbd40cf5f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009840 tx=0x7fcbc8009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbdaf89700 1 -- 172.21.15.6:0/2181844172 shutdown_connections 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbdaf89700 1 --2- 172.21.15.6:0/2181844172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40b1630 0x7fcbd40cf5f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbdaf89700 1 --2- 172.21.15.6:0/2181844172 >> v2:172.21.15.61:3300/0 conn(0x7fcbd40b1a00 0x7fcbd40caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbdaf89700 1 -- 172.21.15.6:0/2181844172 >> 172.21.15.6:0/2181844172 conn(0x7fcbd40cc630 msgr2=0x7fcbd40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbdaf89700 1 -- 172.21.15.6:0/2181844172 shutdown_connections 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.289+0000 7fcbdaf89700 1 -- 172.21.15.6:0/2181844172 wait complete. 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.291+0000 7fcbdaf89700 1 Processor -- start 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.291+0000 7fcbdaf89700 1 -- start start 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.291+0000 7fcbdaf89700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcbd40b1630 0x7fcbd415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.291+0000 7fcbdaf89700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcbd40b1a00 0x7fcbd415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.291+0000 7fcbdaf89700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40cb3e0 0x7fcbd40ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.291+0000 7fcbdaf89700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd41f3350 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.291+0000 7fcbdaf89700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd41f2740 con 0x7fcbd40b1a00 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.291+0000 7fcbdaf89700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd41f2a20 con 0x7fcbd40b1630 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd9526700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40cb3e0 0x7fcbd40ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd9526700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40cb3e0 0x7fcbd40ad6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60074/0 (socket says 172.21.15.6:60074) 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd9526700 1 -- 172.21.15.6:0/112109070 learned_addr learned my addr 172.21.15.6:0/112109070 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd8d25700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcbd40b1630 0x7fcbd415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd3fff700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcbd40b1a00 0x7fcbd415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd9526700 1 -- 172.21.15.6:0/112109070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcbd40b1630 msgr2=0x7fcbd415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd9526700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcbd40b1630 0x7fcbd415bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd9526700 1 -- 172.21.15.6:0/112109070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcbd40b1a00 msgr2=0x7fcbd415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd9526700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcbd40b1a00 0x7fcbd415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd9526700 1 -- 172.21.15.6:0/112109070 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc80094a0 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd8d25700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcbd40b1630 0x7fcbd415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd3fff700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcbd40b1a00 0x7fcbd415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd9526700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40cb3e0 0x7fcbd40ad6c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc002230 tx=0x7fcbcc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd17fa700 1 -- 172.21.15.6:0/112109070 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00ac20 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd17fa700 1 -- 172.21.15.6:0/112109070 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc00ada0 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbdaf89700 1 -- 172.21.15.6:0/112109070 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd40b1ea0 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbd17fa700 1 -- 172.21.15.6:0/112109070 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc007310 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.292+0000 7fcbdaf89700 1 -- 172.21.15.6:0/112109070 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd40cf0d0 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.293+0000 7fcbdaf89700 1 -- 172.21.15.6:0/112109070 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd40ce140 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.294+0000 7fcbd17fa700 1 -- 172.21.15.6:0/112109070 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcbcc01a030 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.294+0000 7fcbd17fa700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcbb80682f0 0x7fcbb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.294+0000 7fcbd17fa700 1 -- 172.21.15.6:0/112109070 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fcbcc085dd0 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.294+0000 7fcbd8d25700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcbb80682f0 0x7fcbb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:31.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.294+0000 7fcbd8d25700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcbb80682f0 0x7fcbb806a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcbd415c770 tx=0x7fcbc8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:31.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.298+0000 7fcbd17fa700 1 -- 172.21.15.6:0/112109070 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcbcc0522b0 con 0x7fcbd40cb3e0 2023-11-28T13:01:31.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:31.538+0000 7fcbdaf89700 1 -- 172.21.15.6:0/112109070 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcbd40739f0 con 0x7fcbd40cb3e0 2023-11-28T13:01:32.139 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.138+0000 7fcbd17fa700 1 -- 172.21.15.6:0/112109070 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcbcc0558d0 con 0x7fcbd40cb3e0 2023-11-28T13:01:32.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 -- 172.21.15.6:0/112109070 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcbb80682f0 msgr2=0x7fcbb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:32.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcbb80682f0 0x7fcbb806a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcbd415c770 tx=0x7fcbc8008030 comp rx=0 tx=0).stop 2023-11-28T13:01:32.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 -- 172.21.15.6:0/112109070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40cb3e0 msgr2=0x7fcbd40ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:32.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40cb3e0 0x7fcbd40ad6c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc002230 tx=0x7fcbcc00a060 comp rx=0 tx=0).stop 2023-11-28T13:01:32.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 -- 172.21.15.6:0/112109070 shutdown_connections 2023-11-28T13:01:32.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcbd40cb3e0 0x7fcbd40ad6c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:32.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcbb80682f0 0x7fcbb806a7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:32.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcbd40b1630 0x7fcbd415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:32.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 --2- 172.21.15.6:0/112109070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcbd40b1a00 0x7fcbd415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:32.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 -- 172.21.15.6:0/112109070 >> 172.21.15.6:0/112109070 conn(0x7fcbd40cc630 msgr2=0x7fcbd40ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:32.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 -- 172.21.15.6:0/112109070 shutdown_connections 2023-11-28T13:01:32.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.141+0000 7fcbdaf89700 1 -- 172.21.15.6:0/112109070 wait complete. 2023-11-28T13:01:32.158 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:01:32.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.335+0000 7f6a310e5700 1 Processor -- start 2023-11-28T13:01:32.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.335+0000 7f6a310e5700 1 -- start start 2023-11-28T13:01:32.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.336+0000 7f6a310e5700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6a2c06bc30 0x7f6a2c15b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:32.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.336+0000 7f6a310e5700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a2c06c3a0 con 0x7f6a2c06d9d0 2023-11-28T13:01:32.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.336+0000 7f6a310e5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a2c06c0c0 con 0x7f6a2c06dda0 2023-11-28T13:01:32.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.336+0000 7f6a310e5700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2c15ab60 con 0x7f6a2c06bc30 2023-11-28T13:01:32.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.336+0000 7f6a2a59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6a2c06bc30 0x7f6a2c15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:32.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.336+0000 7f6a2ad9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6a2c06d9d0 0x7f6a2c150eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47400/0 (socket says 172.21.15.6:47400) 2023-11-28T13:01:32.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.336+0000 7f6a2ad9d700 1 -- 172.21.15.6:0/3479052923 learned_addr learned my addr 172.21.15.6:0/3479052923 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:32.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.336+0000 7f6a2a59c700 1 -- 172.21.15.6:0/3479052923 >> v1:172.21.15.61:6789/0 conn(0x7f6a2c06dda0 legacy=0x7f6a2c0c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:32.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.336+0000 7f6a2a59c700 1 -- 172.21.15.6:0/3479052923 >> v1:172.21.15.6:6789/0 conn(0x7f6a2c06d9d0 legacy=0x7f6a2c150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:32.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a2a59c700 1 -- 172.21.15.6:0/3479052923 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c15ae40 con 0x7f6a2c06bc30 2023-11-28T13:01:32.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a29d9b700 1 -- 172.21.15.6:0/3479052923 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023140247 0 0) 0x7f6a2c06c3a0 con 0x7f6a2c06d9d0 2023-11-28T13:01:32.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a2a59c700 1 --2- 172.21.15.6:0/3479052923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06bc30 0x7f6a2c15b640 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009840 tx=0x7f6a1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b341b66e716f483 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:32.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a29d9b700 1 -- 172.21.15.6:0/3479052923 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c011830 con 0x7f6a2c06bc30 2023-11-28T13:01:32.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a29d9b700 1 -- 172.21.15.6:0/3479052923 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c011990 con 0x7f6a2c06bc30 2023-11-28T13:01:32.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a29d9b700 1 -- 172.21.15.6:0/3479052923 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c011d20 con 0x7f6a2c06bc30 2023-11-28T13:01:32.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a310e5700 1 -- 172.21.15.6:0/3479052923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06bc30 msgr2=0x7f6a2c15b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:32.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a310e5700 1 --2- 172.21.15.6:0/3479052923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06bc30 0x7f6a2c15b640 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009840 tx=0x7f6a1c009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:32.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a310e5700 1 -- 172.21.15.6:0/3479052923 shutdown_connections 2023-11-28T13:01:32.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a310e5700 1 --2- 172.21.15.6:0/3479052923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06bc30 0x7f6a2c15b640 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:32.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a310e5700 1 -- 172.21.15.6:0/3479052923 >> 172.21.15.6:0/3479052923 conn(0x7f6a2c0b0dd0 msgr2=0x7f6a2c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:32.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a310e5700 1 -- 172.21.15.6:0/3479052923 shutdown_connections 2023-11-28T13:01:32.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.337+0000 7f6a310e5700 1 -- 172.21.15.6:0/3479052923 wait complete. 2023-11-28T13:01:32.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.339+0000 7f6a310e5700 1 Processor -- start 2023-11-28T13:01:32.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.339+0000 7f6a310e5700 1 -- start start 2023-11-28T13:01:32.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a310e5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a2c06bc30 0x7f6a2c1579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:32.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a310e5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a2c06d9d0 0x7f6a2c1560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:32.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a310e5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06dda0 0x7f6a2c146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a310e5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c156940 con 0x7f6a2c06dda0 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a310e5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c156c20 con 0x7f6a2c06bc30 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a310e5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c156f00 con 0x7f6a2c06d9d0 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2b59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06dda0 0x7f6a2c146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2b59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06dda0 0x7f6a2c146430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60090/0 (socket says 172.21.15.6:60090) 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2b59e700 1 -- 172.21.15.6:0/560351292 learned_addr learned my addr 172.21.15.6:0/560351292 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2a59c700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a2c06d9d0 0x7f6a2c1560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2ad9d700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a2c06bc30 0x7f6a2c1579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2b59e700 1 -- 172.21.15.6:0/560351292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a2c06d9d0 msgr2=0x7f6a2c1560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2b59e700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a2c06d9d0 0x7f6a2c1560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2b59e700 1 -- 172.21.15.6:0/560351292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a2c06bc30 msgr2=0x7f6a2c1579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:32.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2b59e700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a2c06bc30 0x7f6a2c1579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:32.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2b59e700 1 -- 172.21.15.6:0/560351292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c0094a0 con 0x7f6a2c06dda0 2023-11-28T13:01:32.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2ad9d700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a2c06bc30 0x7f6a2c1579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:32.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.340+0000 7f6a2b59e700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06dda0 0x7f6a2c146430 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400cb50 tx=0x7f6a2400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:32.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.341+0000 7f6a137fe700 1 -- 172.21.15.6:0/560351292 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a24003c00 con 0x7f6a2c06dda0 2023-11-28T13:01:32.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.341+0000 7f6a137fe700 1 -- 172.21.15.6:0/560351292 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a24003d60 con 0x7f6a2c06dda0 2023-11-28T13:01:32.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.341+0000 7f6a310e5700 1 -- 172.21.15.6:0/560351292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2c06fac0 con 0x7f6a2c06dda0 2023-11-28T13:01:32.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.341+0000 7f6a137fe700 1 -- 172.21.15.6:0/560351292 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400aac0 con 0x7f6a2c06dda0 2023-11-28T13:01:32.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.341+0000 7f6a310e5700 1 -- 172.21.15.6:0/560351292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2c15b120 con 0x7f6a2c06dda0 2023-11-28T13:01:32.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.342+0000 7f6a310e5700 1 -- 172.21.15.6:0/560351292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2c0739f0 con 0x7f6a2c06dda0 2023-11-28T13:01:32.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.342+0000 7f6a137fe700 1 -- 172.21.15.6:0/560351292 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a24010050 con 0x7f6a2c06dda0 2023-11-28T13:01:32.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.342+0000 7f6a137fe700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a180682f0 0x7f6a1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:32.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.343+0000 7f6a2ad9d700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a180682f0 0x7f6a1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:32.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.343+0000 7f6a137fe700 1 -- 172.21.15.6:0/560351292 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f6a24085ea0 con 0x7f6a2c06dda0 2023-11-28T13:01:32.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.343+0000 7f6a2ad9d700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a180682f0 0x7f6a1806a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6a20000f60 tx=0x7f6a20007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:32.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.347+0000 7f6a137fe700 1 -- 172.21.15.6:0/560351292 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a24052380 con 0x7f6a2c06dda0 2023-11-28T13:01:32.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:32.539+0000 7f6a310e5700 1 -- 172.21.15.6:0/560351292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6a2c0af4e0 con 0x7f6a2c06dda0 2023-11-28T13:01:33.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.140+0000 7f6a137fe700 1 -- 172.21.15.6:0/560351292 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6a240559a0 con 0x7f6a2c06dda0 2023-11-28T13:01:33.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 -- 172.21.15.6:0/560351292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a180682f0 msgr2=0x7f6a1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:33.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a180682f0 0x7f6a1806a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6a20000f60 tx=0x7f6a20007ce0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 -- 172.21.15.6:0/560351292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06dda0 msgr2=0x7f6a2c146430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:33.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06dda0 0x7f6a2c146430 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400cb50 tx=0x7f6a2400ce60 comp rx=0 tx=0).stop 2023-11-28T13:01:33.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 -- 172.21.15.6:0/560351292 shutdown_connections 2023-11-28T13:01:33.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a2c06dda0 0x7f6a2c146430 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a2c06bc30 0x7f6a2c1579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a180682f0 0x7f6a1806a7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 --2- 172.21.15.6:0/560351292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a2c06d9d0 0x7f6a2c1560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 -- 172.21.15.6:0/560351292 >> 172.21.15.6:0/560351292 conn(0x7f6a2c0b0dd0 msgr2=0x7f6a2c071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:33.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 -- 172.21.15.6:0/560351292 shutdown_connections 2023-11-28T13:01:33.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.147+0000 7f6a310e5700 1 -- 172.21.15.6:0/560351292 wait complete. 2023-11-28T13:01:33.150 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:01:33.164 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:01:33.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.299+0000 7f4e86c9b700 1 Processor -- start 2023-11-28T13:01:33.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.299+0000 7f4e86c9b700 1 -- start start 2023-11-28T13:01:33.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.300+0000 7f4e86c9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4e800af140 0x7f4e8006cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:33.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.300+0000 7f4e86c9b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4e8015e040 0x7f4e8015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:33.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.300+0000 7f4e86c9b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8015ee80 con 0x7f4e8015ea20 2023-11-28T13:01:33.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.300+0000 7f4e86c9b700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8006d4e0 con 0x7f4e800af140 2023-11-28T13:01:33.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.300+0000 7f4e86c9b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8015a8a0 con 0x7f4e8015e040 2023-11-28T13:01:33.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.300+0000 7f4e84a37700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4e800af140 0x7f4e8006cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:33.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.300+0000 7f4e84a37700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4e800af140 0x7f4e8006cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60094/0 (socket says 172.21.15.6:60094) 2023-11-28T13:01:33.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.301+0000 7f4e84a37700 1 -- 172.21.15.6:0/3386210323 learned_addr learned my addr 172.21.15.6:0/3386210323 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:33.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.301+0000 7f4e7ffff700 1 --2- 172.21.15.6:0/3386210323 >> v2:172.21.15.61:3300/0 conn(0x7f4e8015e040 0x7f4e8015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:33.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.301+0000 7f4e84a37700 1 -- 172.21.15.6:0/3386210323 >> v2:172.21.15.61:3300/0 conn(0x7f4e8015e040 msgr2=0x7f4e8015e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:33.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.301+0000 7f4e84a37700 1 --2- 172.21.15.6:0/3386210323 >> v2:172.21.15.61:3300/0 conn(0x7f4e8015e040 0x7f4e8015e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.301+0000 7f4e84a37700 1 -- 172.21.15.6:0/3386210323 >> v1:172.21.15.61:6789/0 conn(0x7f4e8015ea20 legacy=0x7f4e800b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:33.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.301+0000 7f4e84a37700 1 -- 172.21.15.6:0/3386210323 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8006ba30 con 0x7f4e800af140 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.301+0000 7f4e84a37700 1 --2- 172.21.15.6:0/3386210323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e800af140 0x7f4e8006cfa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4e80071570 tx=0x7f4e74009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b46060e37578952 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.301+0000 7f4e7f7fe700 1 -- 172.21.15.6:0/3386210323 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400b870 con 0x7f4e800af140 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.301+0000 7f4e7f7fe700 1 -- 172.21.15.6:0/3386210323 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7400b9d0 con 0x7f4e800af140 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.301+0000 7f4e7f7fe700 1 -- 172.21.15.6:0/3386210323 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400bd60 con 0x7f4e800af140 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.302+0000 7f4e86c9b700 1 -- 172.21.15.6:0/3386210323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e800af140 msgr2=0x7f4e8006cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.302+0000 7f4e86c9b700 1 --2- 172.21.15.6:0/3386210323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e800af140 0x7f4e8006cfa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4e80071570 tx=0x7f4e74009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.302+0000 7f4e86c9b700 1 -- 172.21.15.6:0/3386210323 shutdown_connections 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.302+0000 7f4e86c9b700 1 --2- 172.21.15.6:0/3386210323 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e800af140 0x7f4e8006cfa0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.302+0000 7f4e86c9b700 1 --2- 172.21.15.6:0/3386210323 >> v2:172.21.15.61:3300/0 conn(0x7f4e8015e040 0x7f4e8015e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.302+0000 7f4e86c9b700 1 -- 172.21.15.6:0/3386210323 >> 172.21.15.6:0/3386210323 conn(0x7f4e8006dae0 msgr2=0x7f4e8006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.302+0000 7f4e86c9b700 1 -- 172.21.15.6:0/3386210323 shutdown_connections 2023-11-28T13:01:33.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.302+0000 7f4e86c9b700 1 -- 172.21.15.6:0/3386210323 wait complete. 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.303+0000 7f4e86c9b700 1 Processor -- start 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.303+0000 7f4e86c9b700 1 -- start start 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e86c9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e800af140 0x7f4e800ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e86c9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8015e040 0x7f4e800c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e86c9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e8015ea20 0x7f4e8014e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e86c9b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8014e800 con 0x7f4e8015e040 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e86c9b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8014f210 con 0x7f4e800af140 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e86c9b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800c9a00 con 0x7f4e8015ea20 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8015e040 0x7f4e800c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8015e040 0x7f4e800c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60102/0 (socket says 172.21.15.6:60102) 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7ffff700 1 -- 172.21.15.6:0/1806732286 learned_addr learned my addr 172.21.15.6:0/1806732286 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e84a37700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e800af140 0x7f4e800ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:33.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e85238700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e8015ea20 0x7f4e8014e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7ffff700 1 -- 172.21.15.6:0/1806732286 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e8015ea20 msgr2=0x7f4e8014e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7ffff700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e8015ea20 0x7f4e8014e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7ffff700 1 -- 172.21.15.6:0/1806732286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e800af140 msgr2=0x7f4e800ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7ffff700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e800af140 0x7f4e800ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7ffff700 1 -- 172.21.15.6:0/1806732286 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e74009550 con 0x7f4e8015e040 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7ffff700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8015e040 0x7f4e800c93d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00d800 tx=0x7f4e6c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7d7fa700 1 -- 172.21.15.6:0/1806732286 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0125f0 con 0x7f4e8015e040 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e7d7fa700 1 -- 172.21.15.6:0/1806732286 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c012750 con 0x7f4e8015e040 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.304+0000 7f4e86c9b700 1 -- 172.21.15.6:0/1806732286 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8006b750 con 0x7f4e8015e040 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.305+0000 7f4e7d7fa700 1 -- 172.21.15.6:0/1806732286 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c012b00 con 0x7f4e8015e040 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.305+0000 7f4e86c9b700 1 -- 172.21.15.6:0/1806732286 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8006b470 con 0x7f4e8015e040 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.305+0000 7f4e86c9b700 1 -- 172.21.15.6:0/1806732286 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e800cb2d0 con 0x7f4e8015e040 2023-11-28T13:01:33.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.306+0000 7f4e7d7fa700 1 -- 172.21.15.6:0/1806732286 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4e6c012c60 con 0x7f4e8015e040 2023-11-28T13:01:33.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.306+0000 7f4e7d7fa700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4e640682f0 0x7f4e6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:33.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.306+0000 7f4e7d7fa700 1 -- 172.21.15.6:0/1806732286 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f4e6c0862a0 con 0x7f4e8015e040 2023-11-28T13:01:33.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.306+0000 7f4e84a37700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4e640682f0 0x7f4e6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:33.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.307+0000 7f4e84a37700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4e640682f0 0x7f4e6406a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e74014040 tx=0x7f4e74008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:33.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.311+0000 7f4e7d7fa700 1 -- 172.21.15.6:0/1806732286 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4e6c052930 con 0x7f4e8015e040 2023-11-28T13:01:33.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:33.513+0000 7f4e86c9b700 1 -- 172.21.15.6:0/1806732286 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4e800739f0 con 0x7f4e8015e040 2023-11-28T13:01:34.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.146+0000 7f4e7d7fa700 1 -- 172.21.15.6:0/1806732286 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4e6c055f50 con 0x7f4e8015e040 2023-11-28T13:01:34.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.150+0000 7f4e86c9b700 1 -- 172.21.15.6:0/1806732286 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4e640682f0 msgr2=0x7f4e6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:34.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.150+0000 7f4e86c9b700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4e640682f0 0x7f4e6406a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4e74014040 tx=0x7f4e74008030 comp rx=0 tx=0).stop 2023-11-28T13:01:34.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.150+0000 7f4e86c9b700 1 -- 172.21.15.6:0/1806732286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8015e040 msgr2=0x7f4e800c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:34.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.150+0000 7f4e86c9b700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8015e040 0x7f4e800c93d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00d800 tx=0x7f4e6c00db10 comp rx=0 tx=0).stop 2023-11-28T13:01:34.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.151+0000 7f4e86c9b700 1 -- 172.21.15.6:0/1806732286 shutdown_connections 2023-11-28T13:01:34.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.151+0000 7f4e86c9b700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4e640682f0 0x7f4e6406a7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.151+0000 7f4e86c9b700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4e8015ea20 0x7f4e8014e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.151+0000 7f4e86c9b700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4e800af140 0x7f4e800ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.151+0000 7f4e86c9b700 1 --2- 172.21.15.6:0/1806732286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8015e040 0x7f4e800c93d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.151+0000 7f4e86c9b700 1 -- 172.21.15.6:0/1806732286 >> 172.21.15.6:0/1806732286 conn(0x7f4e8006dae0 msgr2=0x7f4e800cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:34.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.151+0000 7f4e86c9b700 1 -- 172.21.15.6:0/1806732286 shutdown_connections 2023-11-28T13:01:34.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.151+0000 7f4e86c9b700 1 -- 172.21.15.6:0/1806732286 wait complete. 2023-11-28T13:01:34.153 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' removed 2023-11-28T13:01:34.171 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-28T13:01:34.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.305+0000 7f0e3f689700 1 Processor -- start 2023-11-28T13:01:34.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.305+0000 7f0e3f689700 1 -- start start 2023-11-28T13:01:34.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.306+0000 7f0e3f689700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0e38159590 0x7f0e3806e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:34.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.306+0000 7f0e3f689700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e3815db70 0x7f0e380ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:34.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.306+0000 7f0e3f689700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0e380ce890 0x7f0e380ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:34.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.306+0000 7f0e3f689700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e380cf250 con 0x7f0e38159590 2023-11-28T13:01:34.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.306+0000 7f0e3f689700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3815d120 con 0x7f0e380ce890 2023-11-28T13:01:34.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.306+0000 7f0e3f689700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3815d6e0 con 0x7f0e3815db70 2023-11-28T13:01:34.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.306+0000 7f0e3d425700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0e38159590 0x7f0e3806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:34.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.306+0000 7f0e3d425700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0e38159590 0x7f0e3806e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60108/0 (socket says 172.21.15.6:60108) 2023-11-28T13:01:34.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.306+0000 7f0e3d425700 1 -- 172.21.15.6:0/490879088 learned_addr learned my addr 172.21.15.6:0/490879088 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:34.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e3cc24700 1 --2- 172.21.15.6:0/490879088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e3815db70 0x7f0e380ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:34.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e3dc26700 1 --2- 172.21.15.6:0/490879088 >> v2:172.21.15.61:3300/0 conn(0x7f0e380ce890 0x7f0e380ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:34.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e3d425700 1 -- 172.21.15.6:0/490879088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e3815db70 msgr2=0x7f0e380ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:34.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e3d425700 1 --2- 172.21.15.6:0/490879088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e3815db70 0x7f0e380ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e3d425700 1 -- 172.21.15.6:0/490879088 >> v2:172.21.15.61:3300/0 conn(0x7f0e380ce890 msgr2=0x7f0e380ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:34.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e3d425700 1 --2- 172.21.15.6:0/490879088 >> v2:172.21.15.61:3300/0 conn(0x7f0e380ce890 0x7f0e380ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e3d425700 1 -- 172.21.15.6:0/490879088 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380aef60 con 0x7f0e38159590 2023-11-28T13:01:34.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e3dc26700 1 --2- 172.21.15.6:0/490879088 >> v2:172.21.15.61:3300/0 conn(0x7f0e380ce890 0x7f0e380ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:34.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e3d425700 1 --2- 172.21.15.6:0/490879088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e38159590 0x7f0e3806e400 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0e380b15e0 tx=0x7f0e28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13debcb5b8f404a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:34.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e2ffff700 1 -- 172.21.15.6:0/490879088 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800b870 con 0x7f0e38159590 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e2ffff700 1 -- 172.21.15.6:0/490879088 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2800b9d0 con 0x7f0e38159590 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.307+0000 7f0e2ffff700 1 -- 172.21.15.6:0/490879088 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800bd60 con 0x7f0e38159590 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.308+0000 7f0e3f689700 1 -- 172.21.15.6:0/490879088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e38159590 msgr2=0x7f0e3806e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.308+0000 7f0e3f689700 1 --2- 172.21.15.6:0/490879088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e38159590 0x7f0e3806e400 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0e380b15e0 tx=0x7f0e28009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.308+0000 7f0e3f689700 1 -- 172.21.15.6:0/490879088 shutdown_connections 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.308+0000 7f0e3f689700 1 --2- 172.21.15.6:0/490879088 >> v2:172.21.15.61:3300/0 conn(0x7f0e380ce890 0x7f0e380ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.308+0000 7f0e3f689700 1 --2- 172.21.15.6:0/490879088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e38159590 0x7f0e3806e400 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.308+0000 7f0e3f689700 1 --2- 172.21.15.6:0/490879088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e3815db70 0x7f0e380ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.308+0000 7f0e3f689700 1 -- 172.21.15.6:0/490879088 >> 172.21.15.6:0/490879088 conn(0x7f0e380b1650 msgr2=0x7f0e380b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.308+0000 7f0e3f689700 1 -- 172.21.15.6:0/490879088 shutdown_connections 2023-11-28T13:01:34.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.308+0000 7f0e3f689700 1 -- 172.21.15.6:0/490879088 wait complete. 2023-11-28T13:01:34.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.309+0000 7f0e3f689700 1 Processor -- start 2023-11-28T13:01:34.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.309+0000 7f0e3f689700 1 -- start start 2023-11-28T13:01:34.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3f689700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e380ce890 0x7f0e3815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:34.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3f689700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e38159590 0x7f0e38152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3f689700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e3815db70 0x7f0e38158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3f689700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3815aef0 con 0x7f0e380ce890 2023-11-28T13:01:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3f689700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3815b190 con 0x7f0e3815db70 2023-11-28T13:01:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3f689700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3815b470 con 0x7f0e38159590 2023-11-28T13:01:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3d425700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e380ce890 0x7f0e3815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3d425700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e380ce890 0x7f0e3815a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60114/0 (socket says 172.21.15.6:60114) 2023-11-28T13:01:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3d425700 1 -- 172.21.15.6:0/290738201 learned_addr learned my addr 172.21.15.6:0/290738201 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3dc26700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e3815db70 0x7f0e38158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3cc24700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e38159590 0x7f0e38152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:34.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3d425700 1 -- 172.21.15.6:0/290738201 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e38159590 msgr2=0x7f0e38152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3d425700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e38159590 0x7f0e38152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3d425700 1 -- 172.21.15.6:0/290738201 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e3815db70 msgr2=0x7f0e38158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3d425700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e3815db70 0x7f0e38158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3d425700 1 -- 172.21.15.6:0/290738201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e28009550 con 0x7f0e380ce890 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3cc24700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e38159590 0x7f0e38152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3dc26700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e3815db70 0x7f0e38158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3d425700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e380ce890 0x7f0e3815a9b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0e28000c00 tx=0x7f0e28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e2dffb700 1 -- 172.21.15.6:0/290738201 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28004540 con 0x7f0e380ce890 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e2dffb700 1 -- 172.21.15.6:0/290738201 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e280036a0 con 0x7f0e380ce890 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e2dffb700 1 -- 172.21.15.6:0/290738201 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28003a50 con 0x7f0e380ce890 2023-11-28T13:01:34.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.310+0000 7f0e3f689700 1 -- 172.21.15.6:0/290738201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3815d370 con 0x7f0e380ce890 2023-11-28T13:01:34.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.311+0000 7f0e3f689700 1 -- 172.21.15.6:0/290738201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e380af240 con 0x7f0e380ce890 2023-11-28T13:01:34.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.312+0000 7f0e3f689700 1 -- 172.21.15.6:0/290738201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e380aec70 con 0x7f0e380ce890 2023-11-28T13:01:34.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.312+0000 7f0e2dffb700 1 -- 172.21.15.6:0/290738201 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0e2800f050 con 0x7f0e380ce890 2023-11-28T13:01:34.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.312+0000 7f0e2dffb700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0e1c0682f0 0x7f0e1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:34.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.312+0000 7f0e2dffb700 1 -- 172.21.15.6:0/290738201 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f0e280874c0 con 0x7f0e380ce890 2023-11-28T13:01:34.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.312+0000 7f0e3cc24700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0e1c0682f0 0x7f0e1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:34.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.313+0000 7f0e3cc24700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0e1c0682f0 0x7f0e1c06a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e3806b9a0 tx=0x7f0e34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:34.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.317+0000 7f0e2dffb700 1 -- 172.21.15.6:0/290738201 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0e28053d30 con 0x7f0e380ce890 2023-11-28T13:01:34.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:34.519+0000 7f0e3f689700 1 -- 172.21.15.6:0/290738201 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e380739f0 con 0x7f0e380ce890 2023-11-28T13:01:35.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.151+0000 7f0e2dffb700 1 -- 172.21.15.6:0/290738201 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f0e28057350 con 0x7f0e380ce890 2023-11-28T13:01:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.158+0000 7f0e3f689700 1 -- 172.21.15.6:0/290738201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0e1c0682f0 msgr2=0x7f0e1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.158+0000 7f0e3f689700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0e1c0682f0 0x7f0e1c06a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0e3806b9a0 tx=0x7f0e34008040 comp rx=0 tx=0).stop 2023-11-28T13:01:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.158+0000 7f0e3f689700 1 -- 172.21.15.6:0/290738201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e380ce890 msgr2=0x7f0e3815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.158+0000 7f0e3f689700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e380ce890 0x7f0e3815a9b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0e28000c00 tx=0x7f0e28004750 comp rx=0 tx=0).stop 2023-11-28T13:01:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.159+0000 7f0e3f689700 1 -- 172.21.15.6:0/290738201 shutdown_connections 2023-11-28T13:01:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.159+0000 7f0e3f689700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e3815db70 0x7f0e38158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.159+0000 7f0e3f689700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e380ce890 0x7f0e3815a9b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.159+0000 7f0e3f689700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0e1c0682f0 0x7f0e1c06a7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:35.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.159+0000 7f0e3f689700 1 --2- 172.21.15.6:0/290738201 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e38159590 0x7f0e38152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.159+0000 7f0e3f689700 1 -- 172.21.15.6:0/290738201 >> 172.21.15.6:0/290738201 conn(0x7f0e380b1650 msgr2=0x7f0e3806d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.159+0000 7f0e3f689700 1 -- 172.21.15.6:0/290738201 shutdown_connections 2023-11-28T13:01:35.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.159+0000 7f0e3f689700 1 -- 172.21.15.6:0/290738201 wait complete. 2023-11-28T13:01:35.162 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data_ec' removed 2023-11-28T13:01:35.177 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:01:35.177 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T13:01:35.178 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:01:35.178 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:35.178 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:01:35.178 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T13:01:35.178 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:01:35.178 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:35.179 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:01:35.179 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T13:01:35.179 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:01:35.179 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:35.179 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:01:35.179 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T13:01:35.179 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:01:35.179 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:01:35.189 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:01:35.189 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:01:35.190 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:01:35.190 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:01:35.191 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:01:35.191 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:01:35.192 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:01:35.192 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:01:35.231 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:01:35.232 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:35.233 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:01:35.233 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:35.234 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:01:35.234 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:35.234 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:01:35.235 DEBUG:teuthology.parallel:result is None 2023-11-28T13:01:35.235 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T13:01:35.319 INFO:tasks.ceph.mds.a.smithi006.stdout:starting mds.a at 2023-11-28T13:01:35.319 INFO:tasks.ceph.mds.c.smithi006.stdout:starting mds.c at 2023-11-28T13:01:35.330 INFO:tasks.ceph.mds.d.smithi061.stdout:starting mds.d at 2023-11-28T13:01:35.330 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2023-11-28T13:01:35.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.378+0000 7f20f4651700 1 Processor -- start 2023-11-28T13:01:35.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.378+0000 7f20f4651700 1 -- start start 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f4651700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f20ec06bf30 0x7f20ec06c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f4651700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ec0b6350 con 0x7f20ec0d2820 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f4651700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ec0d1470 con 0x7f20ec15c640 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f4651700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec15ca10 con 0x7f20ec06bf30 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f2bee700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f20ec06bf30 0x7f20ec06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f1bec700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f20ec0d2820 0x7f20ec0b5c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47468/0 (socket says 172.21.15.6:47468) 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f1bec700 1 -- 172.21.15.6:0/358049110 learned_addr learned my addr 172.21.15.6:0/358049110 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f2bee700 1 -- 172.21.15.6:0/358049110 >> v1:172.21.15.61:6789/0 conn(0x7f20ec15c640 legacy=0x7f20ec0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f2bee700 1 -- 172.21.15.6:0/358049110 >> v1:172.21.15.6:6789/0 conn(0x7f20ec0d2820 legacy=0x7f20ec0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f13eb700 1 -- 172.21.15.6:0/358049110 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 940641478 0 0) 0x7f20ec0b6350 con 0x7f20ec0d2820 2023-11-28T13:01:35.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.379+0000 7f20f2bee700 1 -- 172.21.15.6:0/358049110 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec06c840 con 0x7f20ec06bf30 2023-11-28T13:01:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f2bee700 1 --2- 172.21.15.6:0/358049110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec06bf30 0x7f20ec06c300 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009840 tx=0x7f20e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180d550dc661df6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f13eb700 1 -- 172.21.15.6:0/358049110 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8011830 con 0x7f20ec06bf30 2023-11-28T13:01:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f13eb700 1 -- 172.21.15.6:0/358049110 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8011990 con 0x7f20ec06bf30 2023-11-28T13:01:35.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f13eb700 1 -- 172.21.15.6:0/358049110 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8011d20 con 0x7f20ec06bf30 2023-11-28T13:01:35.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f4651700 1 -- 172.21.15.6:0/358049110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec06bf30 msgr2=0x7f20ec06c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:35.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f4651700 1 --2- 172.21.15.6:0/358049110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec06bf30 0x7f20ec06c300 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009840 tx=0x7f20e8009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:35.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f4651700 1 -- 172.21.15.6:0/358049110 shutdown_connections 2023-11-28T13:01:35.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f4651700 1 --2- 172.21.15.6:0/358049110 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec06bf30 0x7f20ec06c300 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:35.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f4651700 1 -- 172.21.15.6:0/358049110 >> 172.21.15.6:0/358049110 conn(0x7f20ec0d2df0 msgr2=0x7f20ec0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:35.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f4651700 1 -- 172.21.15.6:0/358049110 shutdown_connections 2023-11-28T13:01:35.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.380+0000 7f20f4651700 1 -- 172.21.15.6:0/358049110 wait complete. 2023-11-28T13:01:35.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f4651700 1 Processor -- start 2023-11-28T13:01:35.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f4651700 1 -- start start 2023-11-28T13:01:35.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f4651700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f20ec06bf30 0x7f20ec0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f4651700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0d2820 0x7f20ec1476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f4651700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20ec15c640 0x7f20ec1f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f4651700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0c7850 con 0x7f20ec0d2820 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f4651700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0c9030 con 0x7f20ec06bf30 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f4651700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec147c20 con 0x7f20ec15c640 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f1bec700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0d2820 0x7f20ec1476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f1bec700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0d2820 0x7f20ec1476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60154/0 (socket says 172.21.15.6:60154) 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.382+0000 7f20f1bec700 1 -- 172.21.15.6:0/1169778120 learned_addr learned my addr 172.21.15.6:0/1169778120 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20f23ed700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f20ec06bf30 0x7f20ec0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20f2bee700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20ec15c640 0x7f20ec1f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20f1bec700 1 -- 172.21.15.6:0/1169778120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20ec15c640 msgr2=0x7f20ec1f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:35.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20f1bec700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20ec15c640 0x7f20ec1f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20f1bec700 1 -- 172.21.15.6:0/1169778120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f20ec06bf30 msgr2=0x7f20ec0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20f1bec700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f20ec06bf30 0x7f20ec0c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20f1bec700 1 -- 172.21.15.6:0/1169778120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e80094a0 con 0x7f20ec0d2820 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20f1bec700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0d2820 0x7f20ec1476e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c520 tx=0x7f20dc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20e2ffd700 1 -- 172.21.15.6:0/1169778120 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc004c80 con 0x7f20ec0d2820 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20e2ffd700 1 -- 172.21.15.6:0/1169778120 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc004e00 con 0x7f20ec0d2820 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20f4651700 1 -- 172.21.15.6:0/1169778120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec0cd7c0 con 0x7f20ec0d2820 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20e2ffd700 1 -- 172.21.15.6:0/1169778120 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc007310 con 0x7f20ec0d2820 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.383+0000 7f20f4651700 1 -- 172.21.15.6:0/1169778120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec0cd4e0 con 0x7f20ec0d2820 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.384+0000 7f20f4651700 1 -- 172.21.15.6:0/1169778120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec0d1710 con 0x7f20ec0d2820 2023-11-28T13:01:35.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.385+0000 7f20e2ffd700 1 -- 172.21.15.6:0/1169778120 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f20dc01a020 con 0x7f20ec0d2820 2023-11-28T13:01:35.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.385+0000 7f20e2ffd700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f20d00682f0 0x7f20d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:35.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.385+0000 7f20e2ffd700 1 -- 172.21.15.6:0/1169778120 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f20dc0855a0 con 0x7f20ec0d2820 2023-11-28T13:01:35.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.385+0000 7f20f23ed700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f20d00682f0 0x7f20d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:35.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.385+0000 7f20f23ed700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f20d00682f0 0x7f20d006a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20e40012e0 tx=0x7f20e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:35.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.389+0000 7f20e2ffd700 1 -- 172.21.15.6:0/1169778120 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f20dc051fc0 con 0x7f20ec0d2820 2023-11-28T13:01:35.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:35.600+0000 7f20f4651700 1 -- 172.21.15.6:0/1169778120 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f20ec068570 con 0x7f20ec0d2820 2023-11-28T13:01:36.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.368+0000 7f20e2ffd700 1 -- 172.21.15.6:0/1169778120 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7f20dc0555e0 con 0x7f20ec0d2820 2023-11-28T13:01:36.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 -- 172.21.15.6:0/1169778120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f20d00682f0 msgr2=0x7f20d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f20d00682f0 0x7f20d006a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f20e40012e0 tx=0x7f20e4009040 comp rx=0 tx=0).stop 2023-11-28T13:01:36.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 -- 172.21.15.6:0/1169778120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0d2820 msgr2=0x7f20ec1476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0d2820 0x7f20ec1476e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c520 tx=0x7f20dc00c830 comp rx=0 tx=0).stop 2023-11-28T13:01:36.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 -- 172.21.15.6:0/1169778120 shutdown_connections 2023-11-28T13:01:36.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f20d00682f0 0x7f20d006a7a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f20ec15c640 0x7f20ec1f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f20ec06bf30 0x7f20ec0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 --2- 172.21.15.6:0/1169778120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f20ec0d2820 0x7f20ec1476e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 -- 172.21.15.6:0/1169778120 >> 172.21.15.6:0/1169778120 conn(0x7f20ec0d2df0 msgr2=0x7f20ec0b8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:36.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 -- 172.21.15.6:0/1169778120 shutdown_connections 2023-11-28T13:01:36.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.371+0000 7f20f4651700 1 -- 172.21.15.6:0/1169778120 wait complete. 2023-11-28T13:01:36.373 INFO:teuthology.orchestra.run.smithi006.stderr: removed all blocklist entries 2023-11-28T13:01:36.387 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T13:01:36.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.516+0000 7effec668700 1 Processor -- start 2023-11-28T13:01:36.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.516+0000 7effec668700 1 -- start start 2023-11-28T13:01:36.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.517+0000 7effec668700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7effe4161090 0x7effe40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.517+0000 7effec668700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effe40caeb0 0x7effe40cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.517+0000 7effec668700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe40ce340 con 0x7effe415c990 2023-11-28T13:01:36.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.517+0000 7effec668700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe40cebe0 con 0x7effe4161090 2023-11-28T13:01:36.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.517+0000 7effec668700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe40cf780 con 0x7effe40caeb0 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.517+0000 7effe9c03700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7effe4161090 0x7effe40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.517+0000 7effe9c03700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7effe4161090 0x7effe40ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60164/0 (socket says 172.21.15.6:60164) 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.517+0000 7effe9c03700 1 -- 172.21.15.6:0/1205885792 learned_addr learned my addr 172.21.15.6:0/1205885792 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.517+0000 7effeac05700 1 --2- 172.21.15.6:0/1205885792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effe40caeb0 0x7effe40cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.518+0000 7effe9c03700 1 -- 172.21.15.6:0/1205885792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effe40caeb0 msgr2=0x7effe40cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.518+0000 7effe9c03700 1 --2- 172.21.15.6:0/1205885792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effe40caeb0 0x7effe40cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.518+0000 7effe9c03700 1 -- 172.21.15.6:0/1205885792 >> v1:172.21.15.61:6789/0 conn(0x7effe415c990 legacy=0x7effe40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.518+0000 7effe9c03700 1 -- 172.21.15.6:0/1205885792 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe40cb3f0 con 0x7effe4161090 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.518+0000 7effe9c03700 1 --2- 172.21.15.6:0/1205885792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe4161090 0x7effe40ca970 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7effe00098f0 tx=0x7effe0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aef647d89a00487 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.518+0000 7effe9402700 1 -- 172.21.15.6:0/1205885792 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0011830 con 0x7effe4161090 2023-11-28T13:01:36.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.518+0000 7effe9402700 1 -- 172.21.15.6:0/1205885792 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe0011990 con 0x7effe4161090 2023-11-28T13:01:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.518+0000 7effe9402700 1 -- 172.21.15.6:0/1205885792 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0011d20 con 0x7effe4161090 2023-11-28T13:01:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.519+0000 7effec668700 1 -- 172.21.15.6:0/1205885792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe4161090 msgr2=0x7effe40ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.519+0000 7effec668700 1 --2- 172.21.15.6:0/1205885792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe4161090 0x7effe40ca970 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7effe00098f0 tx=0x7effe0009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.519+0000 7effec668700 1 -- 172.21.15.6:0/1205885792 shutdown_connections 2023-11-28T13:01:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.519+0000 7effec668700 1 --2- 172.21.15.6:0/1205885792 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effe40caeb0 0x7effe40cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.519+0000 7effec668700 1 --2- 172.21.15.6:0/1205885792 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe4161090 0x7effe40ca970 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.519+0000 7effec668700 1 -- 172.21.15.6:0/1205885792 >> 172.21.15.6:0/1205885792 conn(0x7effe40b1650 msgr2=0x7effe40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.519+0000 7effec668700 1 -- 172.21.15.6:0/1205885792 shutdown_connections 2023-11-28T13:01:36.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.519+0000 7effec668700 1 -- 172.21.15.6:0/1205885792 wait complete. 2023-11-28T13:01:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.520+0000 7effec668700 1 Processor -- start 2023-11-28T13:01:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.520+0000 7effec668700 1 -- start start 2023-11-28T13:01:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effec668700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe40caeb0 0x7effe41f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effec668700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effe415c990 0x7effe41f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effec668700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effe4161090 0x7effe40c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effec668700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe40c9620 con 0x7effe40caeb0 2023-11-28T13:01:36.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effec668700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe40c8a10 con 0x7effe415c990 2023-11-28T13:01:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effec668700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe40c8cf0 con 0x7effe4161090 2023-11-28T13:01:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effea404700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe40caeb0 0x7effe41f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effea404700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe40caeb0 0x7effe41f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60170/0 (socket says 172.21.15.6:60170) 2023-11-28T13:01:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effea404700 1 -- 172.21.15.6:0/1848856486 learned_addr learned my addr 172.21.15.6:0/1848856486 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effeac05700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effe4161090 0x7effe40c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effe9c03700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effe415c990 0x7effe41f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effea404700 1 -- 172.21.15.6:0/1848856486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effe4161090 msgr2=0x7effe40c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effea404700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effe4161090 0x7effe40c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effea404700 1 -- 172.21.15.6:0/1848856486 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effe415c990 msgr2=0x7effe41f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effea404700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effe415c990 0x7effe41f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effea404700 1 -- 172.21.15.6:0/1848856486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe0009550 con 0x7effe40caeb0 2023-11-28T13:01:36.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.521+0000 7effea404700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe40caeb0 0x7effe41f5a10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7effd400c4e0 tx=0x7effd400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:36.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.522+0000 7effdaffd700 1 -- 172.21.15.6:0/1848856486 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40052d0 con 0x7effe40caeb0 2023-11-28T13:01:36.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.522+0000 7effdaffd700 1 -- 172.21.15.6:0/1848856486 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd4005430 con 0x7effe40caeb0 2023-11-28T13:01:36.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.522+0000 7effdaffd700 1 -- 172.21.15.6:0/1848856486 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40038c0 con 0x7effe40caeb0 2023-11-28T13:01:36.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.522+0000 7effec668700 1 -- 172.21.15.6:0/1848856486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe40ce870 con 0x7effe40caeb0 2023-11-28T13:01:36.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.522+0000 7effec668700 1 -- 172.21.15.6:0/1848856486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe40ce590 con 0x7effe40caeb0 2023-11-28T13:01:36.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.523+0000 7effec668700 1 -- 172.21.15.6:0/1848856486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe40cc850 con 0x7effe40caeb0 2023-11-28T13:01:36.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.523+0000 7effdaffd700 1 -- 172.21.15.6:0/1848856486 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7effd4019030 con 0x7effe40caeb0 2023-11-28T13:01:36.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.523+0000 7effdaffd700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7effc80682f0 0x7effc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.524+0000 7effdaffd700 1 -- 172.21.15.6:0/1848856486 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7effd4084d10 con 0x7effe40caeb0 2023-11-28T13:01:36.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.524+0000 7effe9c03700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7effc80682f0 0x7effc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.524+0000 7effe9c03700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7effc80682f0 0x7effc806a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7effe0009c40 tx=0x7effe0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:36.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.528+0000 7effdaffd700 1 -- 172.21.15.6:0/1848856486 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7effd4051720 con 0x7effe40caeb0 2023-11-28T13:01:36.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.795+0000 7effec668700 1 -- 172.21.15.6:0/1848856486 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7effe40675a0 con 0x7effe40caeb0 2023-11-28T13:01:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.796+0000 7effdaffd700 1 -- 172.21.15.6:0/1848856486 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7effd4017090 con 0x7effe40caeb0 2023-11-28T13:01:36.797 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:36.797 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:01:36.797 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_dump": [ 2023-11-28T13:01:36.797 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.797 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.a", 2023-11-28T13:01:36.797 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVl8nvUCxAA8XF3IdK1G4diiM0tQ5npVA==", 2023-11-28T13:01:36.797 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.b", 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlN9+EGBAA8WROC+gNBWJbQmf9ch69iA==", 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:01:36.798 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.c", 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVl/bQ5ExAAerGXuTwAj5uC6cL7ScyCZA==", 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.799 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.d", 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlo58FIBAAMZ4QjC/4ODXdO7T8j7SEYA==", 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.800 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.0", 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA15GVlma1kARAA50lI2mIZlVoGT0ks8qcvzw==", 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.1", 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA45GVl4YRiARAAZ7Qktx0lxuvP0H4ceAeVig==", 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:36.801 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.2", 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA75GVlqfAkARAAFyiIXHxyw6s3bPBB2+KINA==", 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.802 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.3", 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA+5GVlkjURARAANkhMOAGKdlDfEF9L29hxgA==", 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.4", 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBE5GVlNMQ9BxAA/gFfGDxTZ9TNAMQcCV2k9w==", 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.803 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.5", 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBH5GVl91Q8BxAAlVlRbZqylgmq9LkQp/O6jA==", 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.804 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.6", 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBK5GVlDoTjBhAAOXrgM8Q5MfN3I3aMZgjVvw==", 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.7", 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBN5GVlFzINBxAAN6E1A938lHTLYA1GamwdwQ==", 2023-11-28T13:01:36.805 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.0", 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlRYR3JBAADaTghk6Ix3IQVes6gTE56A==", 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow r", 2023-11-28T13:01:36.806 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow rw", 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rwx" 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.1", 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVloippKxAAsZK5iYerb15VlfiZ4svbdg==", 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow r", 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow rw", 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rwx" 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.807 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.2", 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlDlnhJxAAQG5yCtKJUjh6KGJ3Ennxjg==", 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow r", 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow rw", 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rwx" 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.3", 2023-11-28T13:01:36.808 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlbJU6MRAAd4WTJLkmaaL+ATXGlIH6ig==", 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow r", 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow rw", 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rwx" 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.admin", 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAw5GVlXWKtKRAAbAVVrygFCntIL9ljdMCXbg==", 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow *", 2023-11-28T13:01:36.809 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.bootstrap-mds", 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBT5GVlXUHDCRAAbVO+IAyXAoD1gsZIi6gv/A==", 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.810 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBT5GVlE0zDCRAATKAJWO5O1HAcWe+S5T2TjQ==", 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.bootstrap-osd", 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBT5GVlNVTDCRAAvVVB6LAz9IoYMqF8cRpi1Q==", 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.811 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBT5GVlf13DCRAAaSzDaAdV7JUYRzNnZQSKyQ==", 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBT5GVl4WXDCRAA/seZ5yMUNnwEUCUEG/51aQ==", 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T13:01:36.812 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBT5GVlFm7DCRAAYXHrde3owO+AJSenDLVtqg==", 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mgr.x", 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlIRViCBAADpzuU8y4HYepMLT1WkF08g==", 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.813 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow *", 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile mgr", 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mgr.y", 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVl4uVeBBAACyO9UoUP+RyJxFxE8rsFoQ==", 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow *", 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile mgr", 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.814 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:36.815 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-28T13:01:36.815 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:01:36.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 -- 172.21.15.6:0/1848856486 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7effc80682f0 msgr2=0x7effc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7effc80682f0 0x7effc806a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7effe0009c40 tx=0x7effe0008030 comp rx=0 tx=0).stop 2023-11-28T13:01:36.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 -- 172.21.15.6:0/1848856486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe40caeb0 msgr2=0x7effe41f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe40caeb0 0x7effe41f5a10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7effd400c4e0 tx=0x7effd400c7f0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 -- 172.21.15.6:0/1848856486 shutdown_connections 2023-11-28T13:01:36.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7effc80682f0 0x7effc806a7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7effe4161090 0x7effe40c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7effe40caeb0 0x7effe41f5a10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 --2- 172.21.15.6:0/1848856486 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effe415c990 0x7effe41f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 -- 172.21.15.6:0/1848856486 >> 172.21.15.6:0/1848856486 conn(0x7effe40b1650 msgr2=0x7effe40af060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:36.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 -- 172.21.15.6:0/1848856486 shutdown_connections 2023-11-28T13:01:36.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.798+0000 7effec668700 1 -- 172.21.15.6:0/1848856486 wait complete. 2023-11-28T13:01:36.817 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T13:01:36.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.941+0000 7ff40fbb6700 1 Processor -- start 2023-11-28T13:01:36.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.942+0000 7ff40fbb6700 1 -- start start 2023-11-28T13:01:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.942+0000 7ff40fbb6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff40806e400 0x7ff40815db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.942+0000 7ff40fbb6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff4080ce890 0x7ff4080ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.942+0000 7ff40fbb6700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40815e0b0 con 0x7ff408159590 2023-11-28T13:01:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.942+0000 7ff40fbb6700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4080cf250 con 0x7ff40806e400 2023-11-28T13:01:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.942+0000 7ff40fbb6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40815d120 con 0x7ff4080ce890 2023-11-28T13:01:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.942+0000 7ff40d151700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff40806e400 0x7ff40815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.942+0000 7ff40d952700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff408159590 0x7ff4080ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47498/0 (socket says 172.21.15.6:47498) 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.942+0000 7ff40d151700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff40806e400 0x7ff40815db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60180/0 (socket says 172.21.15.6:60180) 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.942+0000 7ff40d952700 1 -- 172.21.15.6:0/2220187803 learned_addr learned my addr 172.21.15.6:0/2220187803 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.943+0000 7ff40e153700 1 --2- 172.21.15.6:0/2220187803 >> v2:172.21.15.61:3300/0 conn(0x7ff4080ce890 0x7ff4080ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.943+0000 7ff40d151700 1 -- 172.21.15.6:0/2220187803 >> v2:172.21.15.61:3300/0 conn(0x7ff4080ce890 msgr2=0x7ff4080ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.943+0000 7ff40d151700 1 --2- 172.21.15.6:0/2220187803 >> v2:172.21.15.61:3300/0 conn(0x7ff4080ce890 0x7ff4080ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.943+0000 7ff40d151700 1 -- 172.21.15.6:0/2220187803 >> v1:172.21.15.6:6789/0 conn(0x7ff408159590 legacy=0x7ff4080ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.943+0000 7ff40d151700 1 -- 172.21.15.6:0/2220187803 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40815d6e0 con 0x7ff40806e400 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.943+0000 7ff40d151700 1 --2- 172.21.15.6:0/2220187803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 0x7ff40815db70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff4080cd500 tx=0x7ff404009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525d8a9c2cf9402d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.943+0000 7ff40c950700 1 -- 172.21.15.6:0/2220187803 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff404004230 con 0x7ff40806e400 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.943+0000 7ff40c950700 1 -- 172.21.15.6:0/2220187803 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff404004390 con 0x7ff40806e400 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.943+0000 7ff40c950700 1 -- 172.21.15.6:0/2220187803 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4040036a0 con 0x7ff40806e400 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.944+0000 7ff40fbb6700 1 -- 172.21.15.6:0/2220187803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 msgr2=0x7ff40815db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.944+0000 7ff40fbb6700 1 --2- 172.21.15.6:0/2220187803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 0x7ff40815db70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff4080cd500 tx=0x7ff404009b20 comp rx=0 tx=0).stop 2023-11-28T13:01:36.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.944+0000 7ff40fbb6700 1 -- 172.21.15.6:0/2220187803 shutdown_connections 2023-11-28T13:01:36.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.944+0000 7ff40fbb6700 1 --2- 172.21.15.6:0/2220187803 >> v2:172.21.15.61:3300/0 conn(0x7ff4080ce890 0x7ff4080ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.944+0000 7ff40fbb6700 1 --2- 172.21.15.6:0/2220187803 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 0x7ff40815db70 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.944+0000 7ff40fbb6700 1 -- 172.21.15.6:0/2220187803 >> 172.21.15.6:0/2220187803 conn(0x7ff4080b1650 msgr2=0x7ff4080b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:36.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.944+0000 7ff40fbb6700 1 -- 172.21.15.6:0/2220187803 shutdown_connections 2023-11-28T13:01:36.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.944+0000 7ff40fbb6700 1 -- 172.21.15.6:0/2220187803 wait complete. 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.945+0000 7ff40fbb6700 1 Processor -- start 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.945+0000 7ff40fbb6700 1 -- start start 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40fbb6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 0x7ff408153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40fbb6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4080ce890 0x7ff4080c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40fbb6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff408159590 0x7ff4081f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40fbb6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4081f3240 con 0x7ff40806e400 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40fbb6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408159ee0 con 0x7ff408159590 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40fbb6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff408153540 con 0x7ff4080ce890 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d952700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 0x7ff408153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d952700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 0x7ff408153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60184/0 (socket says 172.21.15.6:60184) 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d952700 1 -- 172.21.15.6:0/27554404 learned_addr learned my addr 172.21.15.6:0/27554404 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:36.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40e153700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff408159590 0x7ff4081f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d151700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4080ce890 0x7ff4080c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d952700 1 -- 172.21.15.6:0/27554404 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4080ce890 msgr2=0x7ff4080c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d952700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4080ce890 0x7ff4080c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d952700 1 -- 172.21.15.6:0/27554404 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff408159590 msgr2=0x7ff4081f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d952700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff408159590 0x7ff4081f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d952700 1 -- 172.21.15.6:0/27554404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040094a0 con 0x7ff40806e400 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d151700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4080ce890 0x7ff4080c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40e153700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff408159590 0x7ff4081f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff40d952700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 0x7ff408153000 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c780 tx=0x7ff3f800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.946+0000 7ff3fe7fc700 1 -- 172.21.15.6:0/27554404 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8004c80 con 0x7ff40806e400 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.947+0000 7ff3fe7fc700 1 -- 172.21.15.6:0/27554404 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f8004de0 con 0x7ff40806e400 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.947+0000 7ff3fe7fc700 1 -- 172.21.15.6:0/27554404 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8007310 con 0x7ff40806e400 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.947+0000 7ff40fbb6700 1 -- 172.21.15.6:0/27554404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4080af240 con 0x7ff40806e400 2023-11-28T13:01:36.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.947+0000 7ff40fbb6700 1 -- 172.21.15.6:0/27554404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4080aef60 con 0x7ff40806e400 2023-11-28T13:01:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.948+0000 7ff40fbb6700 1 -- 172.21.15.6:0/27554404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40815d370 con 0x7ff40806e400 2023-11-28T13:01:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.948+0000 7ff3fe7fc700 1 -- 172.21.15.6:0/27554404 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff3f8010070 con 0x7ff40806e400 2023-11-28T13:01:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.948+0000 7ff3fe7fc700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff3ec0682f0 0x7ff3ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.949+0000 7ff3fe7fc700 1 -- 172.21.15.6:0/27554404 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ff3f8084c90 con 0x7ff40806e400 2023-11-28T13:01:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.949+0000 7ff40d151700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff3ec0682f0 0x7ff3ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:36.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.949+0000 7ff40d151700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff3ec0682f0 0x7ff3ec06a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff4080cd500 tx=0x7ff404008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:36.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:36.953+0000 7ff3fe7fc700 1 -- 172.21.15.6:0/27554404 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff3f8051610 con 0x7ff40806e400 2023-11-28T13:01:37.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.223+0000 7ff40fbb6700 1 -- 172.21.15.6:0/27554404 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7ff40815e0b0 con 0x7ff40806e400 2023-11-28T13:01:37.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.227+0000 7ff3fe7fc700 1 -- 172.21.15.6:0/27554404 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff40815e0b0 con 0x7ff40806e400 2023-11-28T13:01:37.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 -- 172.21.15.6:0/27554404 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff3ec0682f0 msgr2=0x7ff3ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff3ec0682f0 0x7ff3ec06a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff4080cd500 tx=0x7ff404008030 comp rx=0 tx=0).stop 2023-11-28T13:01:37.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 -- 172.21.15.6:0/27554404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 msgr2=0x7ff408153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 0x7ff408153000 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c780 tx=0x7ff3f800cb40 comp rx=0 tx=0).stop 2023-11-28T13:01:37.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 -- 172.21.15.6:0/27554404 shutdown_connections 2023-11-28T13:01:37.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff408159590 0x7ff4081f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff40806e400 0x7ff408153000 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff3ec0682f0 0x7ff3ec06a7a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 --2- 172.21.15.6:0/27554404 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff4080ce890 0x7ff4080c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 -- 172.21.15.6:0/27554404 >> 172.21.15.6:0/27554404 conn(0x7ff4080b1650 msgr2=0x7ff4080b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:37.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 -- 172.21.15.6:0/27554404 shutdown_connections 2023-11-28T13:01:37.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.229+0000 7ff40fbb6700 1 -- 172.21.15.6:0/27554404 wait complete. 2023-11-28T13:01:37.242 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T13:01:37.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.366+0000 7f3f13a7c700 1 Processor -- start 2023-11-28T13:01:37.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.366+0000 7f3f13a7c700 1 -- start start 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.367+0000 7f3f13a7c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06aa00 0x7f3f0c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.367+0000 7f3f13a7c700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c06ae80 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.367+0000 7f3f13a7c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0cdec0 con 0x7f3f0c06a560 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.367+0000 7f3f13a7c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0c9cd0 con 0x7f3f0c06aa00 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.367+0000 7f3f11818700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f3f0c15d1e0 0x7f3f0c15d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47512/0 (socket says 172.21.15.6:47512) 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.367+0000 7f3f11818700 1 -- 172.21.15.6:0/1781782442 learned_addr learned my addr 172.21.15.6:0/1781782442 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.367+0000 7f3f12019700 1 --2- 172.21.15.6:0/1781782442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06aa00 0x7f3f0c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.367+0000 7f3f12019700 1 -- 172.21.15.6:0/1781782442 >> v1:172.21.15.61:6789/0 conn(0x7f3f0c06a560 legacy=0x7f3f0c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.367+0000 7f3f12019700 1 -- 172.21.15.6:0/1781782442 >> v1:172.21.15.6:6789/0 conn(0x7f3f0c15d1e0 legacy=0x7f3f0c15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.367+0000 7f3f12019700 1 -- 172.21.15.6:0/1781782442 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0c9fb0 con 0x7f3f0c06aa00 2023-11-28T13:01:37.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.368+0000 7f3f12019700 1 --2- 172.21.15.6:0/1781782442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06aa00 0x7f3f0c0cd5c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009dc0 tx=0x7f3f0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a741dfa10e7cc04b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:37.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.371+0000 7f3f10816700 1 -- 172.21.15.6:0/1781782442 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0400e070 con 0x7f3f0c06aa00 2023-11-28T13:01:37.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.371+0000 7f3f10816700 1 -- 172.21.15.6:0/1781782442 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0400bbd0 con 0x7f3f0c06aa00 2023-11-28T13:01:37.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.371+0000 7f3f10816700 1 -- 172.21.15.6:0/1781782442 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04012430 con 0x7f3f0c06aa00 2023-11-28T13:01:37.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.372+0000 7f3f13a7c700 1 -- 172.21.15.6:0/1781782442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06aa00 msgr2=0x7f3f0c0cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.372+0000 7f3f13a7c700 1 --2- 172.21.15.6:0/1781782442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06aa00 0x7f3f0c0cd5c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009dc0 tx=0x7f3f0400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:37.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.372+0000 7f3f13a7c700 1 -- 172.21.15.6:0/1781782442 shutdown_connections 2023-11-28T13:01:37.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.372+0000 7f3f13a7c700 1 --2- 172.21.15.6:0/1781782442 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06aa00 0x7f3f0c0cd5c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.372+0000 7f3f13a7c700 1 -- 172.21.15.6:0/1781782442 >> 172.21.15.6:0/1781782442 conn(0x7f3f0c0b0dd0 msgr2=0x7f3f0c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:37.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.372+0000 7f3f13a7c700 1 -- 172.21.15.6:0/1781782442 shutdown_connections 2023-11-28T13:01:37.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.372+0000 7f3f13a7c700 1 -- 172.21.15.6:0/1781782442 wait complete. 2023-11-28T13:01:37.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.373+0000 7f3f13a7c700 1 Processor -- start 2023-11-28T13:01:37.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.373+0000 7f3f13a7c700 1 -- start start 2023-11-28T13:01:37.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f13a7c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06a560 0x7f3f0c153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f13a7c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f0c06aa00 0x7f3f0c15f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f13a7c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f0c15d1e0 0x7f3f0c15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f13a7c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c15d5b0 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f13a7c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0adba0 con 0x7f3f0c06aa00 2023-11-28T13:01:37.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f13a7c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0ade80 con 0x7f3f0c06a560 2023-11-28T13:01:37.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f12019700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f0c15d1e0 0x7f3f0c15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:37.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f12019700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f0c15d1e0 0x7f3f0c15fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60202/0 (socket says 172.21.15.6:60202) 2023-11-28T13:01:37.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f12019700 1 -- 172.21.15.6:0/4084615135 learned_addr learned my addr 172.21.15.6:0/4084615135 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:37.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f11818700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06a560 0x7f3f0c153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:37.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f11017700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f0c06aa00 0x7f3f0c15f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:37.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f12019700 1 -- 172.21.15.6:0/4084615135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06a560 msgr2=0x7f3f0c153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f12019700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06a560 0x7f3f0c153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f12019700 1 -- 172.21.15.6:0/4084615135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f0c06aa00 msgr2=0x7f3f0c15f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f12019700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f0c06aa00 0x7f3f0c15f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.374+0000 7f3f12019700 1 -- 172.21.15.6:0/4084615135 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04009a70 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.375+0000 7f3f11017700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f0c06aa00 0x7f3f0c15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.375+0000 7f3f12019700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f0c15d1e0 0x7f3f0c15fcf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400b370 tx=0x7f3f04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.375+0000 7f3f027fc700 1 -- 172.21.15.6:0/4084615135 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040098f0 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.375+0000 7f3f027fc700 1 -- 172.21.15.6:0/4084615135 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04020070 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.375+0000 7f3f13a7c700 1 -- 172.21.15.6:0/4084615135 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c0ca570 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.375+0000 7f3f027fc700 1 -- 172.21.15.6:0/4084615135 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04004260 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.375+0000 7f3f13a7c700 1 -- 172.21.15.6:0/4084615135 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c0ca290 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.376+0000 7f3f13a7c700 1 -- 172.21.15.6:0/4084615135 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c0ca850 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.376+0000 7f3f027fc700 1 -- 172.21.15.6:0/4084615135 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f0402e070 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.377+0000 7f3f027fc700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ef00682f0 0x7f3ef006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.377+0000 7f3f027fc700 1 -- 172.21.15.6:0/4084615135 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3f04022070 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.377+0000 7f3f11818700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ef00682f0 0x7f3ef006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:37.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.377+0000 7f3f11818700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ef00682f0 0x7f3ef006a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3f08005910 tx=0x7f3f08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:37.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.381+0000 7f3f027fc700 1 -- 172.21.15.6:0/4084615135 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f04054450 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.656+0000 7f3f13a7c700 1 -- 172.21.15.6:0/4084615135 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f3f0c0675a0 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.660+0000 7f3f027fc700 1 -- 172.21.15.6:0/4084615135 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3f0402c020 con 0x7f3f0c15d1e0 2023-11-28T13:01:37.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 -- 172.21.15.6:0/4084615135 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ef00682f0 msgr2=0x7f3ef006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ef00682f0 0x7f3ef006a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3f08005910 tx=0x7f3f08009040 comp rx=0 tx=0).stop 2023-11-28T13:01:37.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 -- 172.21.15.6:0/4084615135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f0c15d1e0 msgr2=0x7f3f0c15fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f0c15d1e0 0x7f3f0c15fcf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400b370 tx=0x7f3f04004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 -- 172.21.15.6:0/4084615135 shutdown_connections 2023-11-28T13:01:37.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3f0c15d1e0 0x7f3f0c15fcf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3ef00682f0 0x7f3ef006a7a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3f0c06a560 0x7f3f0c153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 --2- 172.21.15.6:0/4084615135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3f0c06aa00 0x7f3f0c15f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 -- 172.21.15.6:0/4084615135 >> 172.21.15.6:0/4084615135 conn(0x7f3f0c0b0dd0 msgr2=0x7f3f0c15c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:37.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 -- 172.21.15.6:0/4084615135 shutdown_connections 2023-11-28T13:01:37.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.662+0000 7f3f13a7c700 1 -- 172.21.15.6:0/4084615135 wait complete. 2023-11-28T13:01:37.675 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T13:01:37.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.801+0000 7f33e1623700 1 Processor -- start 2023-11-28T13:01:37.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.801+0000 7f33e1623700 1 -- start start 2023-11-28T13:01:37.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.801+0000 7f33e1623700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f33dc15c640 0x7f33dc0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.801+0000 7f33e1623700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f33dc06bf30 0x7f33dc06c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.801+0000 7f33e1623700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33dc06b5b0 con 0x7f33dc0d2820 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.801+0000 7f33e1623700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc0d1810 con 0x7f33dc06bf30 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.801+0000 7f33e1623700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc15ca10 con 0x7f33dc15c640 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33db7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f33dc06bf30 0x7f33dc06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33da7fc700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f33dc0d2820 0x7f33dc0b5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47526/0 (socket says 172.21.15.6:47526) 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33da7fc700 1 -- 172.21.15.6:0/2898821691 learned_addr learned my addr 172.21.15.6:0/2898821691 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33db7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f33dc06bf30 0x7f33dc06c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60208/0 (socket says 172.21.15.6:60208) 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33db7fe700 1 -- 172.21.15.6:0/2898821691 >> v2:172.21.15.61:3300/0 conn(0x7f33dc15c640 msgr2=0x7f33dc0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33db7fe700 1 --2- 172.21.15.6:0/2898821691 >> v2:172.21.15.61:3300/0 conn(0x7f33dc15c640 0x7f33dc0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33db7fe700 1 -- 172.21.15.6:0/2898821691 >> v1:172.21.15.6:6789/0 conn(0x7f33dc0d2820 legacy=0x7f33dc0b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33d9ffb700 1 -- 172.21.15.6:0/2898821691 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771588570 0 0) 0x7f33dc06b5b0 con 0x7f33dc0d2820 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33db7fe700 1 -- 172.21.15.6:0/2898821691 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc0b6330 con 0x7f33dc06bf30 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33db7fe700 1 --2- 172.21.15.6:0/2898821691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc06bf30 0x7f33dc06c320 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f33dc06d420 tx=0x7f33d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e3211e639a95ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33d9ffb700 1 -- 172.21.15.6:0/2898821691 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d400e070 con 0x7f33dc06bf30 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33d9ffb700 1 -- 172.21.15.6:0/2898821691 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d400bb90 con 0x7f33dc06bf30 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.802+0000 7f33d9ffb700 1 -- 172.21.15.6:0/2898821691 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4012430 con 0x7f33dc06bf30 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.803+0000 7f33e1623700 1 -- 172.21.15.6:0/2898821691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc06bf30 msgr2=0x7f33dc06c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.803+0000 7f33e1623700 1 --2- 172.21.15.6:0/2898821691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc06bf30 0x7f33dc06c320 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f33dc06d420 tx=0x7f33d400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.803+0000 7f33e1623700 1 -- 172.21.15.6:0/2898821691 shutdown_connections 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.803+0000 7f33e1623700 1 --2- 172.21.15.6:0/2898821691 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc06bf30 0x7f33dc06c320 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.803+0000 7f33e1623700 1 --2- 172.21.15.6:0/2898821691 >> v2:172.21.15.61:3300/0 conn(0x7f33dc15c640 0x7f33dc0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.803+0000 7f33e1623700 1 -- 172.21.15.6:0/2898821691 >> 172.21.15.6:0/2898821691 conn(0x7f33dc0d2df0 msgr2=0x7f33dc0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.803+0000 7f33e1623700 1 -- 172.21.15.6:0/2898821691 shutdown_connections 2023-11-28T13:01:37.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.803+0000 7f33e1623700 1 -- 172.21.15.6:0/2898821691 wait complete. 2023-11-28T13:01:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.804+0000 7f33e1623700 1 Processor -- start 2023-11-28T13:01:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.804+0000 7f33e1623700 1 -- start start 2023-11-28T13:01:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33e1623700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33dc06bf30 0x7f33dc0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33e1623700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc0d2820 0x7f33dc14e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33e1623700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33dc15c640 0x7f33dc0b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33e1623700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc15cdf0 con 0x7f33dc0d2820 2023-11-28T13:01:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33e1623700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc14e910 con 0x7f33dc15c640 2023-11-28T13:01:37.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33e1623700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc15d040 con 0x7f33dc06bf30 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33da7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc0d2820 0x7f33dc14e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33da7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc0d2820 0x7f33dc14e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60214/0 (socket says 172.21.15.6:60214) 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33da7fc700 1 -- 172.21.15.6:0/1603616642 learned_addr learned my addr 172.21.15.6:0/1603616642 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33db7fe700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33dc15c640 0x7f33dc0b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33daffd700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33dc06bf30 0x7f33dc0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33da7fc700 1 -- 172.21.15.6:0/1603616642 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33dc06bf30 msgr2=0x7f33dc0ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33da7fc700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33dc06bf30 0x7f33dc0ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33da7fc700 1 -- 172.21.15.6:0/1603616642 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33dc15c640 msgr2=0x7f33dc0b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33da7fc700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33dc15c640 0x7f33dc0b6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33da7fc700 1 -- 172.21.15.6:0/1603616642 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4009a70 con 0x7f33dc0d2820 2023-11-28T13:01:37.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.805+0000 7f33da7fc700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc0d2820 0x7f33dc14e3d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f33d00011e0 tx=0x7f33d000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.806+0000 7f33c37fe700 1 -- 172.21.15.6:0/1603616642 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0004c80 con 0x7f33dc0d2820 2023-11-28T13:01:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.806+0000 7f33c37fe700 1 -- 172.21.15.6:0/1603616642 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0004e00 con 0x7f33dc0d2820 2023-11-28T13:01:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.806+0000 7f33e1623700 1 -- 172.21.15.6:0/1603616642 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc06c860 con 0x7f33dc0d2820 2023-11-28T13:01:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.806+0000 7f33c37fe700 1 -- 172.21.15.6:0/1603616642 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0007310 con 0x7f33dc0d2820 2023-11-28T13:01:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.806+0000 7f33e1623700 1 -- 172.21.15.6:0/1603616642 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc0d1470 con 0x7f33dc0d2820 2023-11-28T13:01:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.807+0000 7f33e1623700 1 -- 172.21.15.6:0/1603616642 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc0cd4e0 con 0x7f33dc0d2820 2023-11-28T13:01:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.807+0000 7f33c37fe700 1 -- 172.21.15.6:0/1603616642 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f33d001a020 con 0x7f33dc0d2820 2023-11-28T13:01:37.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.808+0000 7f33c37fe700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33c8068320 0x7f33c806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.808+0000 7f33c37fe700 1 -- 172.21.15.6:0/1603616642 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f33d00855d0 con 0x7f33dc0d2820 2023-11-28T13:01:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.808+0000 7f33daffd700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33c8068320 0x7f33c806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.808+0000 7f33daffd700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33c8068320 0x7f33c806a7d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0b4720 tx=0x7f33cc009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:37.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:37.812+0000 7f33c37fe700 1 -- 172.21.15.6:0/1603616642 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f33d00520f0 con 0x7f33dc0d2820 2023-11-28T13:01:38.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.082+0000 7f33e1623700 1 -- 172.21.15.6:0/1603616642 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f33dc0c9ea0 con 0x7f33dc0d2820 2023-11-28T13:01:38.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.086+0000 7f33c37fe700 1 -- 172.21.15.6:0/1603616642 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f33d0055710 con 0x7f33dc0d2820 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 -- 172.21.15.6:0/1603616642 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33c8068320 msgr2=0x7f33c806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33c8068320 0x7f33c806a7d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0b4720 tx=0x7f33cc009110 comp rx=0 tx=0).stop 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 -- 172.21.15.6:0/1603616642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc0d2820 msgr2=0x7f33dc14e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc0d2820 0x7f33dc14e3d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f33d00011e0 tx=0x7f33d000ca30 comp rx=0 tx=0).stop 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 -- 172.21.15.6:0/1603616642 shutdown_connections 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33dc15c640 0x7f33dc0b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33c8068320 0x7f33c806a7d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f33dc06bf30 0x7f33dc0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 --2- 172.21.15.6:0/1603616642 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33dc0d2820 0x7f33dc14e3d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 -- 172.21.15.6:0/1603616642 >> 172.21.15.6:0/1603616642 conn(0x7f33dc0d2df0 msgr2=0x7f33dc0b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:38.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 -- 172.21.15.6:0/1603616642 shutdown_connections 2023-11-28T13:01:38.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.088+0000 7f33e1623700 1 -- 172.21.15.6:0/1603616642 wait complete. 2023-11-28T13:01:38.102 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T13:01:38.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.228+0000 7f9331960700 1 Processor -- start 2023-11-28T13:01:38.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.228+0000 7f9331960700 1 -- start start 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f9331960700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f932c15c640 0x7f932c0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f9331960700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c0b6350 con 0x7f932c06bf30 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f9331960700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c0d1470 con 0x7f932c0d2820 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f9331960700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c15ca10 con 0x7f932c15c640 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f932affd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f932c15c640 0x7f932c0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f932b7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f932c06bf30 0x7f932c06c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47544/0 (socket says 172.21.15.6:47544) 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f932b7fe700 1 -- 172.21.15.6:0/2469680199 learned_addr learned my addr 172.21.15.6:0/2469680199 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f932affd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f932c15c640 0x7f932c0cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60220/0 (socket says 172.21.15.6:60220) 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f932affd700 1 -- 172.21.15.6:0/2469680199 >> v1:172.21.15.61:6789/0 conn(0x7f932c0d2820 legacy=0x7f932c0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f932affd700 1 -- 172.21.15.6:0/2469680199 >> v1:172.21.15.6:6789/0 conn(0x7f932c06bf30 legacy=0x7f932c06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f932affd700 1 -- 172.21.15.6:0/2469680199 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c0cd4e0 con 0x7f932c15c640 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.229+0000 7f9329ffb700 1 -- 172.21.15.6:0/2469680199 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177987429 0 0) 0x7f932c0b6350 con 0x7f932c06bf30 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f932affd700 1 --2- 172.21.15.6:0/2469680199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c15c640 0x7f932c0cf040 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9320009840 tx=0x7f9320009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe4fbaae4fcb694 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f9329ffb700 1 -- 172.21.15.6:0/2469680199 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320011830 con 0x7f932c15c640 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f9329ffb700 1 -- 172.21.15.6:0/2469680199 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9320011990 con 0x7f932c15c640 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f9329ffb700 1 -- 172.21.15.6:0/2469680199 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320011d20 con 0x7f932c15c640 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f9331960700 1 -- 172.21.15.6:0/2469680199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c15c640 msgr2=0x7f932c0cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f9331960700 1 --2- 172.21.15.6:0/2469680199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c15c640 0x7f932c0cf040 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9320009840 tx=0x7f9320009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f9331960700 1 -- 172.21.15.6:0/2469680199 shutdown_connections 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f9331960700 1 --2- 172.21.15.6:0/2469680199 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c15c640 0x7f932c0cf040 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f9331960700 1 -- 172.21.15.6:0/2469680199 >> 172.21.15.6:0/2469680199 conn(0x7f932c0d2df0 msgr2=0x7f932c0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f9331960700 1 -- 172.21.15.6:0/2469680199 shutdown_connections 2023-11-28T13:01:38.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.230+0000 7f9331960700 1 -- 172.21.15.6:0/2469680199 wait complete. 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.232+0000 7f9331960700 1 Processor -- start 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.232+0000 7f9331960700 1 -- start start 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.232+0000 7f9331960700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932c06bf30 0x7f932c14b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.232+0000 7f9331960700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c0d2820 0x7f932c0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.232+0000 7f9331960700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f932c15c640 0x7f932c15ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f9331960700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c14b970 con 0x7f932c0d2820 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f9331960700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c0c74a0 con 0x7f932c15c640 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f9331960700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c0c7780 con 0x7f932c06bf30 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932a7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c0d2820 0x7f932c0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932a7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c0d2820 0x7f932c0c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60228/0 (socket says 172.21.15.6:60228) 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932a7fc700 1 -- 172.21.15.6:0/322141756 learned_addr learned my addr 172.21.15.6:0/322141756 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:38.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932b7fe700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f932c15c640 0x7f932c15ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932affd700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932c06bf30 0x7f932c14b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932a7fc700 1 -- 172.21.15.6:0/322141756 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932c06bf30 msgr2=0x7f932c14b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932a7fc700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932c06bf30 0x7f932c14b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932a7fc700 1 -- 172.21.15.6:0/322141756 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f932c15c640 msgr2=0x7f932c15ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932a7fc700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f932c15c640 0x7f932c15ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932a7fc700 1 -- 172.21.15.6:0/322141756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93200094a0 con 0x7f932c0d2820 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f932a7fc700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c0d2820 0x7f932c0c8ec0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c4e0 tx=0x7f931c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f9313fff700 1 -- 172.21.15.6:0/322141756 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c0052d0 con 0x7f932c0d2820 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f9313fff700 1 -- 172.21.15.6:0/322141756 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c005430 con 0x7f932c0d2820 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f9331960700 1 -- 172.21.15.6:0/322141756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c0d1710 con 0x7f932c0d2820 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f9313fff700 1 -- 172.21.15.6:0/322141756 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c0038c0 con 0x7f932c0d2820 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.233+0000 7f9331960700 1 -- 172.21.15.6:0/322141756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c0cd7c0 con 0x7f932c0d2820 2023-11-28T13:01:38.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.234+0000 7f9331960700 1 -- 172.21.15.6:0/322141756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932c0cf580 con 0x7f932c0d2820 2023-11-28T13:01:38.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.235+0000 7f9313fff700 1 -- 172.21.15.6:0/322141756 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f931c019030 con 0x7f932c0d2820 2023-11-28T13:01:38.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.235+0000 7f9313fff700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93180682f0 0x7f931806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.235+0000 7f9313fff700 1 -- 172.21.15.6:0/322141756 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f931c084de0 con 0x7f932c0d2820 2023-11-28T13:01:38.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.235+0000 7f932affd700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93180682f0 0x7f931806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.235+0000 7f932affd700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93180682f0 0x7f931806a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f93200090b0 tx=0x7f932001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:38.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.239+0000 7f9313fff700 1 -- 172.21.15.6:0/322141756 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f931c051850 con 0x7f932c0d2820 2023-11-28T13:01:38.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.514+0000 7f9331960700 1 -- 172.21.15.6:0/322141756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f932c068570 con 0x7f932c0d2820 2023-11-28T13:01:38.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.518+0000 7f9313fff700 1 -- 172.21.15.6:0/322141756 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f931c054e70 con 0x7f932c0d2820 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 -- 172.21.15.6:0/322141756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93180682f0 msgr2=0x7f931806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93180682f0 0x7f931806a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f93200090b0 tx=0x7f932001c040 comp rx=0 tx=0).stop 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 -- 172.21.15.6:0/322141756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c0d2820 msgr2=0x7f932c0c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c0d2820 0x7f932c0c8ec0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c4e0 tx=0x7f931c00c8a0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 -- 172.21.15.6:0/322141756 shutdown_connections 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f932c15c640 0x7f932c15ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93180682f0 0x7f931806a7a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f932c06bf30 0x7f932c14b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 --2- 172.21.15.6:0/322141756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f932c0d2820 0x7f932c0c8ec0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 -- 172.21.15.6:0/322141756 >> 172.21.15.6:0/322141756 conn(0x7f932c0d2df0 msgr2=0x7f932c0b8510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:38.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 -- 172.21.15.6:0/322141756 shutdown_connections 2023-11-28T13:01:38.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.520+0000 7f9331960700 1 -- 172.21.15.6:0/322141756 wait complete. 2023-11-28T13:01:38.534 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T13:01:38.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.659+0000 7fdf6d662700 1 Processor -- start 2023-11-28T13:01:38.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.659+0000 7fdf6d662700 1 -- start start 2023-11-28T13:01:38.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf6d662700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdf6815c560 0x7fdf6806f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf6d662700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdf680b2660 0x7fdf680ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf6d662700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf680cf150 con 0x7fdf680cecf0 2023-11-28T13:01:38.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf6d662700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf680b30b0 con 0x7fdf680b2660 2023-11-28T13:01:38.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf6d662700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6806c020 con 0x7fdf6815c560 2023-11-28T13:01:38.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf667fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdf680b2660 0x7fdf680ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf667fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdf680b2660 0x7fdf680ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60234/0 (socket says 172.21.15.6:60234) 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf667fc700 1 -- 172.21.15.6:0/1263707837 learned_addr learned my addr 172.21.15.6:0/1263707837 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf66ffd700 1 --2- 172.21.15.6:0/1263707837 >> v2:172.21.15.61:3300/0 conn(0x7fdf6815c560 0x7fdf6806f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf667fc700 1 -- 172.21.15.6:0/1263707837 >> v2:172.21.15.61:3300/0 conn(0x7fdf6815c560 msgr2=0x7fdf6806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf667fc700 1 --2- 172.21.15.6:0/1263707837 >> v2:172.21.15.61:3300/0 conn(0x7fdf6815c560 0x7fdf6806f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf667fc700 1 -- 172.21.15.6:0/1263707837 >> v1:172.21.15.61:6789/0 conn(0x7fdf680cecf0 legacy=0x7fdf68160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf667fc700 1 -- 172.21.15.6:0/1263707837 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf680d09c0 con 0x7fdf680b2660 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.660+0000 7fdf667fc700 1 --2- 172.21.15.6:0/1263707837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf680b2660 0x7fdf680ce7b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdf68066920 tx=0x7fdf58009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f5a50ac4775eca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf65ffb700 1 -- 172.21.15.6:0/1263707837 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58004230 con 0x7fdf680b2660 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf65ffb700 1 -- 172.21.15.6:0/1263707837 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf58004390 con 0x7fdf680b2660 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf65ffb700 1 -- 172.21.15.6:0/1263707837 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf580036a0 con 0x7fdf680b2660 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf6d662700 1 -- 172.21.15.6:0/1263707837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf680b2660 msgr2=0x7fdf680ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf6d662700 1 --2- 172.21.15.6:0/1263707837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf680b2660 0x7fdf680ce7b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdf68066920 tx=0x7fdf58009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf6d662700 1 -- 172.21.15.6:0/1263707837 shutdown_connections 2023-11-28T13:01:38.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf6d662700 1 --2- 172.21.15.6:0/1263707837 >> v2:172.21.15.61:3300/0 conn(0x7fdf6815c560 0x7fdf6806f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf6d662700 1 --2- 172.21.15.6:0/1263707837 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf680b2660 0x7fdf680ce7b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf6d662700 1 -- 172.21.15.6:0/1263707837 >> 172.21.15.6:0/1263707837 conn(0x7fdf680cd660 msgr2=0x7fdf680cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:38.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf6d662700 1 -- 172.21.15.6:0/1263707837 shutdown_connections 2023-11-28T13:01:38.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.661+0000 7fdf6d662700 1 -- 172.21.15.6:0/1263707837 wait complete. 2023-11-28T13:01:38.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf6d662700 1 Processor -- start 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf6d662700 1 -- start start 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf6d662700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf680b2660 0x7fdf680ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf6d662700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf680cecf0 0x7fdf680ad940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf6d662700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6815c560 0x7fdf680ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf6d662700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf680aca40 con 0x7fdf6815c560 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf6d662700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68157a20 con 0x7fdf680b2660 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf6d662700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68157d00 con 0x7fdf680cecf0 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf677fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6815c560 0x7fdf680ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf677fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6815c560 0x7fdf680ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60244/0 (socket says 172.21.15.6:60244) 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.663+0000 7fdf677fe700 1 -- 172.21.15.6:0/3470969524 learned_addr learned my addr 172.21.15.6:0/3470969524 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:38.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf66ffd700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf680b2660 0x7fdf680ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf667fc700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf680cecf0 0x7fdf680ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf677fe700 1 -- 172.21.15.6:0/3470969524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf680cecf0 msgr2=0x7fdf680ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf677fe700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf680cecf0 0x7fdf680ad940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf677fe700 1 -- 172.21.15.6:0/3470969524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf680b2660 msgr2=0x7fdf680ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf677fe700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf680b2660 0x7fdf680ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf677fe700 1 -- 172.21.15.6:0/3470969524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf58009550 con 0x7fdf6815c560 2023-11-28T13:01:38.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf677fe700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6815c560 0x7fdf680ade80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c7a0 tx=0x7fdf6000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:38.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf4f7fe700 1 -- 172.21.15.6:0/3470969524 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60003e10 con 0x7fdf6815c560 2023-11-28T13:01:38.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf4f7fe700 1 -- 172.21.15.6:0/3470969524 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60005070 con 0x7fdf6815c560 2023-11-28T13:01:38.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf6d662700 1 -- 172.21.15.6:0/3470969524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf680b2af0 con 0x7fdf6815c560 2023-11-28T13:01:38.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf4f7fe700 1 -- 172.21.15.6:0/3470969524 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60005420 con 0x7fdf6815c560 2023-11-28T13:01:38.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.664+0000 7fdf6d662700 1 -- 172.21.15.6:0/3470969524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6806fdf0 con 0x7fdf6815c560 2023-11-28T13:01:38.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.665+0000 7fdf6d662700 1 -- 172.21.15.6:0/3470969524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf680b2dd0 con 0x7fdf6815c560 2023-11-28T13:01:38.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.666+0000 7fdf4f7fe700 1 -- 172.21.15.6:0/3470969524 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdf60019030 con 0x7fdf6815c560 2023-11-28T13:01:38.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.666+0000 7fdf4f7fe700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf540682f0 0x7fdf5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:38.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.666+0000 7fdf4f7fe700 1 -- 172.21.15.6:0/3470969524 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fdf60085060 con 0x7fdf6815c560 2023-11-28T13:01:38.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.666+0000 7fdf66ffd700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf540682f0 0x7fdf5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:38.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.667+0000 7fdf66ffd700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf540682f0 0x7fdf5406a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdf680b0820 tx=0x7fdf5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:38.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.670+0000 7fdf4f7fe700 1 -- 172.21.15.6:0/3470969524 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdf60051ad0 con 0x7fdf6815c560 2023-11-28T13:01:38.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.943+0000 7fdf6d662700 1 -- 172.21.15.6:0/3470969524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fdf680675a0 con 0x7fdf6815c560 2023-11-28T13:01:38.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.947+0000 7fdf4f7fe700 1 -- 172.21.15.6:0/3470969524 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdf60017020 con 0x7fdf6815c560 2023-11-28T13:01:38.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 -- 172.21.15.6:0/3470969524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf540682f0 msgr2=0x7fdf5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf540682f0 0x7fdf5406a7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdf680b0820 tx=0x7fdf5c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:38.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 -- 172.21.15.6:0/3470969524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6815c560 msgr2=0x7fdf680ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:38.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6815c560 0x7fdf680ade80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c7a0 tx=0x7fdf6000cb60 comp rx=0 tx=0).stop 2023-11-28T13:01:38.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 -- 172.21.15.6:0/3470969524 shutdown_connections 2023-11-28T13:01:38.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6815c560 0x7fdf680ade80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf680b2660 0x7fdf680ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdf540682f0 0x7fdf5406a7a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 --2- 172.21.15.6:0/3470969524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdf680cecf0 0x7fdf680ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:38.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 -- 172.21.15.6:0/3470969524 >> 172.21.15.6:0/3470969524 conn(0x7fdf680cd660 msgr2=0x7fdf680b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:38.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 -- 172.21.15.6:0/3470969524 shutdown_connections 2023-11-28T13:01:38.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:38.949+0000 7fdf6d662700 1 -- 172.21.15.6:0/3470969524 wait complete. 2023-11-28T13:01:38.963 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T13:01:39.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.088+0000 7fc4a8c7e700 1 Processor -- start 2023-11-28T13:01:39.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.088+0000 7fc4a8c7e700 1 -- start start 2023-11-28T13:01:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a8c7e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4a415c990 0x7fc4a40cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a8c7e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc4a4161090 0x7fc4a40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a8c7e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc4a40caeb0 0x7fc4a40cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a8c7e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a40ce340 con 0x7fc4a4161090 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a8c7e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a40cebe0 con 0x7fc4a40caeb0 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a8c7e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40cb3f0 con 0x7fc4a415c990 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a1d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc4a4161090 0x7fc4a40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a1d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc4a4161090 0x7fc4a40ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60248/0 (socket says 172.21.15.6:60248) 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a1d9b700 1 -- 172.21.15.6:0/2943384493 learned_addr learned my addr 172.21.15.6:0/2943384493 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a2d9d700 1 --2- 172.21.15.6:0/2943384493 >> v2:172.21.15.61:3300/0 conn(0x7fc4a40caeb0 0x7fc4a40cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a259c700 1 --2- 172.21.15.6:0/2943384493 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4a415c990 0x7fc4a40cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a1d9b700 1 -- 172.21.15.6:0/2943384493 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4a415c990 msgr2=0x7fc4a40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a1d9b700 1 --2- 172.21.15.6:0/2943384493 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4a415c990 0x7fc4a40cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a1d9b700 1 -- 172.21.15.6:0/2943384493 >> v2:172.21.15.61:3300/0 conn(0x7fc4a40caeb0 msgr2=0x7fc4a40cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a1d9b700 1 --2- 172.21.15.6:0/2943384493 >> v2:172.21.15.61:3300/0 conn(0x7fc4a40caeb0 0x7fc4a40cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a1d9b700 1 -- 172.21.15.6:0/2943384493 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a40ce590 con 0x7fc4a4161090 2023-11-28T13:01:39.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a2d9d700 1 --2- 172.21.15.6:0/2943384493 >> v2:172.21.15.61:3300/0 conn(0x7fc4a40caeb0 0x7fc4a40cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a1d9b700 1 --2- 172.21.15.6:0/2943384493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a4161090 0x7fc4a40ca970 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40ca080 tx=0x7fc494009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7e065a97b1dd0b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a159a700 1 -- 172.21.15.6:0/2943384493 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49400b870 con 0x7fc4a4161090 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a159a700 1 -- 172.21.15.6:0/2943384493 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49400b9d0 con 0x7fc4a4161090 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.089+0000 7fc4a159a700 1 -- 172.21.15.6:0/2943384493 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49400bd60 con 0x7fc4a4161090 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.090+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/2943384493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a4161090 msgr2=0x7fc4a40ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.090+0000 7fc4a8c7e700 1 --2- 172.21.15.6:0/2943384493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a4161090 0x7fc4a40ca970 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40ca080 tx=0x7fc494009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.090+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/2943384493 shutdown_connections 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.090+0000 7fc4a8c7e700 1 --2- 172.21.15.6:0/2943384493 >> v2:172.21.15.61:3300/0 conn(0x7fc4a40caeb0 0x7fc4a40cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.090+0000 7fc4a8c7e700 1 --2- 172.21.15.6:0/2943384493 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4a415c990 0x7fc4a40cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.090+0000 7fc4a8c7e700 1 --2- 172.21.15.6:0/2943384493 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a4161090 0x7fc4a40ca970 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.090+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/2943384493 >> 172.21.15.6:0/2943384493 conn(0x7fc4a40b1650 msgr2=0x7fc4a40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:39.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.090+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/2943384493 shutdown_connections 2023-11-28T13:01:39.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.090+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/2943384493 wait complete. 2023-11-28T13:01:39.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.091+0000 7fc4a8c7e700 1 Processor -- start 2023-11-28T13:01:39.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.091+0000 7fc4a8c7e700 1 -- start start 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a8c7e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40caeb0 0x7fc4a40ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a8c7e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4a415c990 0x7fc4a40b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a8c7e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4a4161090 0x7fc4a415edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a8c7e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a40ac980 con 0x7fc4a40caeb0 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a8c7e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a415f840 con 0x7fc4a4161090 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a8c7e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a415fb20 con 0x7fc4a415c990 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40caeb0 0x7fc4a40ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40caeb0 0x7fc4a40ac440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60256/0 (socket says 172.21.15.6:60256) 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a259c700 1 -- 172.21.15.6:0/3577533552 learned_addr learned my addr 172.21.15.6:0/3577533552 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a2d9d700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4a4161090 0x7fc4a415edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.092+0000 7fc4a1d9b700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4a415c990 0x7fc4a40b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc4a259c700 1 -- 172.21.15.6:0/3577533552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4a415c990 msgr2=0x7fc4a40b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc4a259c700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4a415c990 0x7fc4a40b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc4a259c700 1 -- 172.21.15.6:0/3577533552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4a4161090 msgr2=0x7fc4a415edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc4a259c700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4a4161090 0x7fc4a415edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc4a259c700 1 -- 172.21.15.6:0/3577533552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494009550 con 0x7fc4a40caeb0 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc4a259c700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40caeb0 0x7fc4a40ac440 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc49800e810 tx=0x7fc49800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc492ffd700 1 -- 172.21.15.6:0/3577533552 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49800c070 con 0x7fc4a40caeb0 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc492ffd700 1 -- 172.21.15.6:0/3577533552 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc498013610 con 0x7fc4a40caeb0 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/3577533552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a40cf5b0 con 0x7fc4a40caeb0 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc492ffd700 1 -- 172.21.15.6:0/3577533552 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4980139c0 con 0x7fc4a40caeb0 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.093+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/3577533552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a40ce870 con 0x7fc4a40caeb0 2023-11-28T13:01:39.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.094+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/3577533552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a40cf740 con 0x7fc4a40caeb0 2023-11-28T13:01:39.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.094+0000 7fc492ffd700 1 -- 172.21.15.6:0/3577533552 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc498023070 con 0x7fc4a40caeb0 2023-11-28T13:01:39.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.095+0000 7fc492ffd700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4840682f0 0x7fc48406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.095+0000 7fc492ffd700 1 -- 172.21.15.6:0/3577533552 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fc4980857e0 con 0x7fc4a40caeb0 2023-11-28T13:01:39.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.095+0000 7fc4a1d9b700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4840682f0 0x7fc48406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.095+0000 7fc4a1d9b700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4840682f0 0x7fc48406a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4a41614c0 tx=0x7fc494008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:39.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.099+0000 7fc492ffd700 1 -- 172.21.15.6:0/3577533552 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc4980521d0 con 0x7fc4a40caeb0 2023-11-28T13:01:39.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.372+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/3577533552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fc4a40675a0 con 0x7fc4a40caeb0 2023-11-28T13:01:39.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.375+0000 7fc492ffd700 1 -- 172.21.15.6:0/3577533552 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc49800f040 con 0x7fc4a40caeb0 2023-11-28T13:01:39.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/3577533552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4840682f0 msgr2=0x7fc48406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4840682f0 0x7fc48406a7a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4a41614c0 tx=0x7fc494008030 comp rx=0 tx=0).stop 2023-11-28T13:01:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/3577533552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40caeb0 msgr2=0x7fc4a40ac440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40caeb0 0x7fc4a40ac440 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc49800e810 tx=0x7fc49800eb20 comp rx=0 tx=0).stop 2023-11-28T13:01:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/3577533552 shutdown_connections 2023-11-28T13:01:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4a4161090 0x7fc4a415edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4a40caeb0 0x7fc4a40ac440 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4840682f0 0x7fc48406a7a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 --2- 172.21.15.6:0/3577533552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4a415c990 0x7fc4a40b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/3577533552 >> 172.21.15.6:0/3577533552 conn(0x7fc4a40b1650 msgr2=0x7fc4a406b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/3577533552 shutdown_connections 2023-11-28T13:01:39.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.378+0000 7fc4a8c7e700 1 -- 172.21.15.6:0/3577533552 wait complete. 2023-11-28T13:01:39.391 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:39.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.515+0000 7fedc7744700 1 Processor -- start 2023-11-28T13:01:39.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.516+0000 7fedc7744700 1 -- start start 2023-11-28T13:01:39.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.516+0000 7fedc7744700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 0x7fedc015b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.516+0000 7fedc7744700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fedc006dda0 0x7fedc015e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.516+0000 7fedc7744700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc006c3a0 con 0x7fedc006d9d0 2023-11-28T13:01:39.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.516+0000 7fedc7744700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc015ee90 con 0x7fedc006dda0 2023-11-28T13:01:39.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.516+0000 7fedc7744700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc006c0c0 con 0x7fedc006bc30 2023-11-28T13:01:39.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc5ce1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fedc006dda0 0x7fedc015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc4cdf700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 0x7fedc015b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc4cdf700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 0x7fedc015b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:49968/0 (socket says 172.21.15.6:49968) 2023-11-28T13:01:39.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc54e0700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fedc006d9d0 0x7fedc0150eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:37612/0 (socket says 172.21.15.6:37612) 2023-11-28T13:01:39.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc4cdf700 1 -- 172.21.15.6:0/2132748754 learned_addr learned my addr 172.21.15.6:0/2132748754 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:39.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc4cdf700 1 -- 172.21.15.6:0/2132748754 >> v2:172.21.15.61:3300/0 conn(0x7fedc006dda0 msgr2=0x7fedc015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:39.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc4cdf700 1 --2- 172.21.15.6:0/2132748754 >> v2:172.21.15.61:3300/0 conn(0x7fedc006dda0 0x7fedc015e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc4cdf700 1 -- 172.21.15.6:0/2132748754 >> v1:172.21.15.61:6789/0 conn(0x7fedc006d9d0 legacy=0x7fedc0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc4cdf700 1 -- 172.21.15.6:0/2132748754 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc015ab60 con 0x7fedc006bc30 2023-11-28T13:01:39.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc5ce1700 1 --2- 172.21.15.6:0/2132748754 >> v2:172.21.15.61:3300/0 conn(0x7fedc006dda0 0x7fedc015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:39.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.517+0000 7fedc4cdf700 1 --2- 172.21.15.6:0/2132748754 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 0x7fedc015b640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0098f0 tx=0x7fedbc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a7c2c3d328aac49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedb3fff700 1 -- 172.21.15.6:0/2132748754 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc011830 con 0x7fedc006bc30 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedb3fff700 1 -- 172.21.15.6:0/2132748754 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc011990 con 0x7fedc006bc30 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedb3fff700 1 -- 172.21.15.6:0/2132748754 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc011d20 con 0x7fedc006bc30 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedc7744700 1 -- 172.21.15.6:0/2132748754 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 msgr2=0x7fedc015b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedc7744700 1 --2- 172.21.15.6:0/2132748754 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 0x7fedc015b640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0098f0 tx=0x7fedbc009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedc7744700 1 -- 172.21.15.6:0/2132748754 shutdown_connections 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedc7744700 1 --2- 172.21.15.6:0/2132748754 >> v2:172.21.15.61:3300/0 conn(0x7fedc006dda0 0x7fedc015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedc7744700 1 --2- 172.21.15.6:0/2132748754 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 0x7fedc015b640 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedc7744700 1 -- 172.21.15.6:0/2132748754 >> 172.21.15.6:0/2132748754 conn(0x7fedc00b0dd0 msgr2=0x7fedc00b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedc7744700 1 -- 172.21.15.6:0/2132748754 shutdown_connections 2023-11-28T13:01:39.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.518+0000 7fedc7744700 1 -- 172.21.15.6:0/2132748754 wait complete. 2023-11-28T13:01:39.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.519+0000 7fedc7744700 1 Processor -- start 2023-11-28T13:01:39.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.519+0000 7fedc7744700 1 -- start start 2023-11-28T13:01:39.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc7744700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 0x7fedc00b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc7744700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedc006d9d0 0x7fedc0147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc7744700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedc006dda0 0x7fedc01f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc7744700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0147cb0 con 0x7fedc006d9d0 2023-11-28T13:01:39.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc7744700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00cceb0 con 0x7fedc006dda0 2023-11-28T13:01:39.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc7744700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc01f5fb0 con 0x7fedc006bc30 2023-11-28T13:01:39.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc4cdf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedc006d9d0 0x7fedc0147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc4cdf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedc006d9d0 0x7fedc0147770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60270/0 (socket says 172.21.15.6:60270) 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc4cdf700 1 -- 172.21.15.6:0/4066461570 learned_addr learned my addr 172.21.15.6:0/4066461570 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc54e0700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 0x7fedc00b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.520+0000 7fedc5ce1700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedc006dda0 0x7fedc01f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedc4cdf700 1 -- 172.21.15.6:0/4066461570 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 msgr2=0x7fedc00b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedc4cdf700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 0x7fedc00b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedc4cdf700 1 -- 172.21.15.6:0/4066461570 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedc006dda0 msgr2=0x7fedc01f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedc4cdf700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedc006dda0 0x7fedc01f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedc4cdf700 1 -- 172.21.15.6:0/4066461570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8009a70 con 0x7fedc006d9d0 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedc4cdf700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedc006d9d0 0x7fedc0147770 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fedbc000c00 tx=0x7fedbc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedb1ffb700 1 -- 172.21.15.6:0/4066461570 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0043c0 con 0x7fedc006d9d0 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedb1ffb700 1 -- 172.21.15.6:0/4066461570 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc01f070 con 0x7fedc006d9d0 2023-11-28T13:01:39.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedc7744700 1 -- 172.21.15.6:0/4066461570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedbc009550 con 0x7fedc006d9d0 2023-11-28T13:01:39.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedb1ffb700 1 -- 172.21.15.6:0/4066461570 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0038c0 con 0x7fedc006d9d0 2023-11-28T13:01:39.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.521+0000 7fedc7744700 1 -- 172.21.15.6:0/4066461570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc015b120 con 0x7fedc006d9d0 2023-11-28T13:01:39.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.522+0000 7fedc7744700 1 -- 172.21.15.6:0/4066461570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc015ae40 con 0x7fedc006d9d0 2023-11-28T13:01:39.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.523+0000 7fedb1ffb700 1 -- 172.21.15.6:0/4066461570 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fedbc02a020 con 0x7fedc006d9d0 2023-11-28T13:01:39.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.523+0000 7fedb1ffb700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feda40682f0 0x7feda406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.523+0000 7fedb1ffb700 1 -- 172.21.15.6:0/4066461570 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fedbc086f90 con 0x7fedc006d9d0 2023-11-28T13:01:39.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.523+0000 7fedc54e0700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feda40682f0 0x7feda406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.523+0000 7fedc54e0700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feda40682f0 0x7feda406a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fedb4000f60 tx=0x7fedb4007ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:39.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.527+0000 7fedb1ffb700 1 -- 172.21.15.6:0/4066461570 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fedbc053a00 con 0x7fedc006d9d0 2023-11-28T13:01:39.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.723+0000 7fedc7744700 1 -- 172.21.15.6:0/4066461570 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedc006fac0 con 0x7fedc006d9d0 2023-11-28T13:01:39.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.724+0000 7fedb1ffb700 1 -- 172.21.15.6:0/4066461570 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+14318 (secure 0 0 0) 0x7fedbc015090 con 0x7fedc006d9d0 2023-11-28T13:01:39.725 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:39.725 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":36,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:36.363211+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:39.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 -- 172.21.15.6:0/4066461570 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feda40682f0 msgr2=0x7feda406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feda40682f0 0x7feda406a7a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fedb4000f60 tx=0x7fedb4007ed0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 -- 172.21.15.6:0/4066461570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedc006d9d0 msgr2=0x7fedc0147770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedc006d9d0 0x7fedc0147770 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fedbc000c00 tx=0x7fedbc004750 comp rx=0 tx=0).stop 2023-11-28T13:01:39.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 -- 172.21.15.6:0/4066461570 shutdown_connections 2023-11-28T13:01:39.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fedc006dda0 0x7fedc01f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7feda40682f0 0x7feda406a7a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fedc006bc30 0x7fedc00b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 --2- 172.21.15.6:0/4066461570 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fedc006d9d0 0x7fedc0147770 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 -- 172.21.15.6:0/4066461570 >> 172.21.15.6:0/4066461570 conn(0x7fedc00b0dd0 msgr2=0x7fedc015a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:39.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 -- 172.21.15.6:0/4066461570 shutdown_connections 2023-11-28T13:01:39.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.726+0000 7fedc7744700 1 -- 172.21.15.6:0/4066461570 wait complete. 2023-11-28T13:01:39.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:01:39.740 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:39.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.866+0000 7f85c230b700 1 Processor -- start 2023-11-28T13:01:39.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.866+0000 7f85c230b700 1 -- start start 2023-11-28T13:01:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.867+0000 7f85c230b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f85bc15c950 0x7f85bc06a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.867+0000 7f85c230b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85bc0cae50 0x7f85bc0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.867+0000 7f85c230b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85bc0caab0 con 0x7f85bc0ca360 2023-11-28T13:01:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.867+0000 7f85c230b700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc15cd20 con 0x7f85bc0cae50 2023-11-28T13:01:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.867+0000 7f85c230b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85bc0cb7b0 con 0x7f85bc15c950 2023-11-28T13:01:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.867+0000 7f85c08a8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85bc0cae50 0x7f85bc0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.867+0000 7f85c08a8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85bc0cae50 0x7f85bc0cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60280/0 (socket says 172.21.15.6:60280) 2023-11-28T13:01:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.867+0000 7f85c08a8700 1 -- 172.21.15.6:0/1853865293 learned_addr learned my addr 172.21.15.6:0/1853865293 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:39.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85bbfff700 1 --2- 172.21.15.6:0/1853865293 >> v2:172.21.15.61:3300/0 conn(0x7f85bc15c950 0x7f85bc06a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85c08a8700 1 -- 172.21.15.6:0/1853865293 >> v2:172.21.15.61:3300/0 conn(0x7f85bc15c950 msgr2=0x7f85bc06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85c08a8700 1 --2- 172.21.15.6:0/1853865293 >> v2:172.21.15.61:3300/0 conn(0x7f85bc15c950 0x7f85bc06a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85c08a8700 1 -- 172.21.15.6:0/1853865293 >> v1:172.21.15.61:6789/0 conn(0x7f85bc0ca360 legacy=0x7f85bc06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85c08a8700 1 -- 172.21.15.6:0/1853865293 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc06fc00 con 0x7f85bc0cae50 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85c08a8700 1 --2- 172.21.15.6:0/1853865293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0cae50 0x7f85bc0cb270 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f85bc070dc0 tx=0x7f85b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0b1fb718732272 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85baffd700 1 -- 172.21.15.6:0/1853865293 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400e070 con 0x7f85bc0cae50 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85baffd700 1 -- 172.21.15.6:0/1853865293 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b4003a20 con 0x7f85bc0cae50 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85baffd700 1 -- 172.21.15.6:0/1853865293 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4003dd0 con 0x7f85bc0cae50 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85c230b700 1 -- 172.21.15.6:0/1853865293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0cae50 msgr2=0x7f85bc0cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85c230b700 1 --2- 172.21.15.6:0/1853865293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0cae50 0x7f85bc0cb270 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f85bc070dc0 tx=0x7f85b400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85c230b700 1 -- 172.21.15.6:0/1853865293 shutdown_connections 2023-11-28T13:01:39.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85c230b700 1 --2- 172.21.15.6:0/1853865293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0cae50 0x7f85bc0cb270 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.868+0000 7f85c230b700 1 --2- 172.21.15.6:0/1853865293 >> v2:172.21.15.61:3300/0 conn(0x7f85bc15c950 0x7f85bc06a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.869+0000 7f85c230b700 1 -- 172.21.15.6:0/1853865293 >> 172.21.15.6:0/1853865293 conn(0x7f85bc0b0df0 msgr2=0x7f85bc0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.869+0000 7f85c230b700 1 -- 172.21.15.6:0/1853865293 shutdown_connections 2023-11-28T13:01:39.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.869+0000 7f85c230b700 1 -- 172.21.15.6:0/1853865293 wait complete. 2023-11-28T13:01:39.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.870+0000 7f85c230b700 1 Processor -- start 2023-11-28T13:01:39.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.870+0000 7f85c230b700 1 -- start start 2023-11-28T13:01:39.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.871+0000 7f85c230b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0ca360 0x7f85bc15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.871+0000 7f85c230b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85bc0cae50 0x7f85bc153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.871+0000 7f85c230b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85bc15c950 0x7f85bc1502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.871+0000 7f85c230b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc155370 con 0x7f85bc0ca360 2023-11-28T13:01:39.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.871+0000 7f85c230b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc153c20 con 0x7f85bc0cae50 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.871+0000 7f85c230b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85bc15c1d0 con 0x7f85bc15c950 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.871+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0ca360 0x7f85bc15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.871+0000 7f85bbfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0ca360 0x7f85bc15ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60282/0 (socket says 172.21.15.6:60282) 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.871+0000 7f85bbfff700 1 -- 172.21.15.6:0/1003736987 learned_addr learned my addr 172.21.15.6:0/1003736987 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85bb7fe700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85bc0cae50 0x7f85bc153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85c08a8700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85bc15c950 0x7f85bc1502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85bbfff700 1 -- 172.21.15.6:0/1003736987 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85bc15c950 msgr2=0x7f85bc1502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85bbfff700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85bc15c950 0x7f85bc1502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85bbfff700 1 -- 172.21.15.6:0/1003736987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85bc0cae50 msgr2=0x7f85bc153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85bbfff700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85bc0cae50 0x7f85bc153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85bbfff700 1 -- 172.21.15.6:0/1003736987 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4009b30 con 0x7f85bc0ca360 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85bbfff700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0ca360 0x7f85bc15ab80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00c7f0 tx=0x7f85ac00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:39.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85b8ff9700 1 -- 172.21.15.6:0/1003736987 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac011800 con 0x7f85bc0ca360 2023-11-28T13:01:39.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85b8ff9700 1 -- 172.21.15.6:0/1003736987 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac011960 con 0x7f85bc0ca360 2023-11-28T13:01:39.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85b8ff9700 1 -- 172.21.15.6:0/1003736987 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac011cf0 con 0x7f85bc0ca360 2023-11-28T13:01:39.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85c230b700 1 -- 172.21.15.6:0/1003736987 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc0ca7d0 con 0x7f85bc0ca360 2023-11-28T13:01:39.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.872+0000 7f85c230b700 1 -- 172.21.15.6:0/1003736987 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85bc06e770 con 0x7f85bc0ca360 2023-11-28T13:01:39.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.873+0000 7f85c230b700 1 -- 172.21.15.6:0/1003736987 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85bc06ea10 con 0x7f85bc0ca360 2023-11-28T13:01:39.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.873+0000 7f85b8ff9700 1 -- 172.21.15.6:0/1003736987 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f85ac021070 con 0x7f85bc0ca360 2023-11-28T13:01:39.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.874+0000 7f85b8ff9700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85a80682f0 0x7f85a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:39.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.874+0000 7f85b8ff9700 1 -- 172.21.15.6:0/1003736987 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f85ac085770 con 0x7f85bc0ca360 2023-11-28T13:01:39.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.874+0000 7f85bb7fe700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85a80682f0 0x7f85a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:39.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.874+0000 7f85bb7fe700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85a80682f0 0x7f85a806a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f85bc159db0 tx=0x7f85b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:39.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:39.878+0000 7f85b8ff9700 1 -- 172.21.15.6:0/1003736987 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f85ac0521e0 con 0x7f85bc0ca360 2023-11-28T13:01:40.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.070+0000 7f85c230b700 1 -- 172.21.15.6:0/1003736987 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f85bc155120 con 0x7f85bc0ca360 2023-11-28T13:01:40.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.070+0000 7f85b8ff9700 1 -- 172.21.15.6:0/1003736987 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7f85ac01f020 con 0x7f85bc0ca360 2023-11-28T13:01:40.072 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:40.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.072+0000 7f85c230b700 1 -- 172.21.15.6:0/1003736987 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85a80682f0 msgr2=0x7f85a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.072+0000 7f85c230b700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85a80682f0 0x7f85a806a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f85bc159db0 tx=0x7f85b0009040 comp rx=0 tx=0).stop 2023-11-28T13:01:40.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.072+0000 7f85c230b700 1 -- 172.21.15.6:0/1003736987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0ca360 msgr2=0x7f85bc15ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.072+0000 7f85c230b700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0ca360 0x7f85bc15ab80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00c7f0 tx=0x7f85ac00cbb0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.072+0000 7f85c230b700 1 -- 172.21.15.6:0/1003736987 shutdown_connections 2023-11-28T13:01:40.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.072+0000 7f85c230b700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85a80682f0 0x7f85a806a7a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.072+0000 7f85c230b700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85bc15c950 0x7f85bc1502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.072+0000 7f85c230b700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85bc0ca360 0x7f85bc15ab80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.072+0000 7f85c230b700 1 --2- 172.21.15.6:0/1003736987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85bc0cae50 0x7f85bc153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.072+0000 7f85c230b700 1 -- 172.21.15.6:0/1003736987 >> 172.21.15.6:0/1003736987 conn(0x7f85bc0b0df0 msgr2=0x7f85bc0cd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.073+0000 7f85c230b700 1 -- 172.21.15.6:0/1003736987 shutdown_connections 2023-11-28T13:01:40.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.073+0000 7f85c230b700 1 -- 172.21.15.6:0/1003736987 wait complete. 2023-11-28T13:01:40.086 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:40.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.213+0000 7f1e23ee9700 1 Processor -- start 2023-11-28T13:01:40.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.214+0000 7f1e23ee9700 1 -- start start 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.214+0000 7f1e23ee9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0b62d0 0x7f1e1c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.214+0000 7f1e23ee9700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c0cd2a0 con 0x7f1e1c06f8b0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.214+0000 7f1e23ee9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c0d0400 con 0x7f1e1c0b2d90 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.214+0000 7f1e23ee9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0d06e0 con 0x7f1e1c0b62d0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e21484700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1e1c06f8b0 0x7f1e1c0b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47610/0 (socket says 172.21.15.6:47610) 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e21484700 1 -- 172.21.15.6:0/1755623094 learned_addr learned my addr 172.21.15.6:0/1755623094 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e21c85700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0b62d0 0x7f1e1c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262108430 0 0) 0x7f1e1c0cd2a0 con 0x7f1e1c06f8b0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e080035b0 con 0x7f1e1c06f8b0 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70002541 0 0) 0x7f1e1c0d0400 con 0x7f1e1c0b2d90 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e1c0cd2a0 con 0x7f1e1c0b2d90 2023-11-28T13:01:40.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3087090989 0 0) 0x7f1e080035b0 con 0x7f1e1c06f8b0 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0b62d0 msgr2=0x7f1e1c0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0b62d0 0x7f1e1c0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 >> v1:172.21.15.61:6789/0 conn(0x7f1e1c0b2d90 legacy=0x7f1e1c0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c0cffb0 con 0x7f1e1c06f8b0 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f1e18001670 con 0x7f1e1c06f8b0 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 >> v1:172.21.15.6:6789/0 conn(0x7f1e1c06f8b0 legacy=0x7f1e1c0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 0x7f1e08003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e08004850 0x7f1e08004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0b62d0 0x7f1e08005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0d06e0 con 0x7f1e08003bc0 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0800bac0 con 0x7f1e08004850 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0800bc30 con 0x7f1e1c0b62d0 2023-11-28T13:01:40.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e21c85700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 0x7f1e08003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.215+0000 7f1e21484700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e08004850 0x7f1e08004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e22486700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0b62d0 0x7f1e08005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e21c85700 1 -- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0b62d0 msgr2=0x7f1e08005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e21c85700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0b62d0 0x7f1e08005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e21c85700 1 -- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e08004850 msgr2=0x7f1e08004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e21c85700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e08004850 0x7f1e08004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e21c85700 1 -- 172.21.15.6:0/1755623094 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e080035b0 con 0x7f1e08003bc0 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e21c85700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 0x7f1e08003fb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009b60 tx=0x7f1e08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c0313642bd85a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00e6c0 con 0x7f1e08003bc0 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c00e820 con 0x7f1e08003bc0 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e20c83700 1 -- 172.21.15.6:0/1755623094 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00eb90 con 0x7f1e08003bc0 2023-11-28T13:01:40.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e23ee9700 1 -- 172.21.15.6:0/1755623094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 msgr2=0x7f1e08003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.216+0000 7f1e23ee9700 1 --2- 172.21.15.6:0/1755623094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 0x7f1e08003fb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009b60 tx=0x7f1e08005880 comp rx=0 tx=0).stop 2023-11-28T13:01:40.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.217+0000 7f1e22486700 1 -- 172.21.15.6:0/1755623094 reap_dead start 2023-11-28T13:01:40.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.217+0000 7f1e23ee9700 1 -- 172.21.15.6:0/1755623094 shutdown_connections 2023-11-28T13:01:40.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.217+0000 7f1e23ee9700 1 -- 172.21.15.6:0/1755623094 >> 172.21.15.6:0/1755623094 conn(0x7f1e1c0cd680 msgr2=0x7f1e1c0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.217+0000 7f1e23ee9700 1 -- 172.21.15.6:0/1755623094 shutdown_connections 2023-11-28T13:01:40.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.217+0000 7f1e23ee9700 1 -- 172.21.15.6:0/1755623094 wait complete. 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.218+0000 7f1e23ee9700 1 Processor -- start 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.218+0000 7f1e23ee9700 1 -- start start 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e23ee9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 0x7f1e1c0ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e23ee9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0c7cd0 0x7f1e1c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e23ee9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e1c0ad030 0x7f1e1c14db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e23ee9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0ae200 con 0x7f1e08003bc0 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e23ee9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0ca760 con 0x7f1e1c0ad030 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e23ee9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0ad5f0 con 0x7f1e1c0c7cd0 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21c85700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 0x7f1e1c0ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21c85700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 0x7f1e1c0ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60302/0 (socket says 172.21.15.6:60302) 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21c85700 1 -- 172.21.15.6:0/179745374 learned_addr learned my addr 172.21.15.6:0/179745374 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21484700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0c7cd0 0x7f1e1c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e22486700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e1c0ad030 0x7f1e1c14db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21c85700 1 -- 172.21.15.6:0/179745374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0c7cd0 msgr2=0x7f1e1c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21c85700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0c7cd0 0x7f1e1c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21c85700 1 -- 172.21.15.6:0/179745374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e1c0ad030 msgr2=0x7f1e1c14db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21c85700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e1c0ad030 0x7f1e1c14db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21c85700 1 -- 172.21.15.6:0/179745374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c06fd20 con 0x7f1e08003bc0 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21484700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0c7cd0 0x7f1e1c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e21c85700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 0x7f1e1c0ce640 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1e08004780 tx=0x7f1e0c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e127fc700 1 -- 172.21.15.6:0/179745374 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c021070 con 0x7f1e08003bc0 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e127fc700 1 -- 172.21.15.6:0/179745374 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c004520 con 0x7f1e08003bc0 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e23ee9700 1 -- 172.21.15.6:0/179745374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c0b66a0 con 0x7f1e08003bc0 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.219+0000 7f1e127fc700 1 -- 172.21.15.6:0/179745374 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00e6c0 con 0x7f1e08003bc0 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.220+0000 7f1e23ee9700 1 -- 172.21.15.6:0/179745374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c0cce50 con 0x7f1e08003bc0 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.220+0000 7f1e23ee9700 1 -- 172.21.15.6:0/179745374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1c0739f0 con 0x7f1e08003bc0 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.221+0000 7f1e127fc700 1 -- 172.21.15.6:0/179745374 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1e0c02e030 con 0x7f1e08003bc0 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.221+0000 7f1e127fc700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1e000682f0 0x7f1e0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.221+0000 7f1e21484700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1e000682f0 0x7f1e0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.222+0000 7f1e127fc700 1 -- 172.21.15.6:0/179745374 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1e0c086e20 con 0x7f1e08003bc0 2023-11-28T13:01:40.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.222+0000 7f1e21484700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1e000682f0 0x7f1e0006a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1e18000fd0 tx=0x7f1e18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.226+0000 7f1e127fc700 1 -- 172.21.15.6:0/179745374 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1e0c053890 con 0x7f1e08003bc0 2023-11-28T13:01:40.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.418+0000 7f1e23ee9700 1 -- 172.21.15.6:0/179745374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e1c0cdf50 con 0x7f1e08003bc0 2023-11-28T13:01:40.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.418+0000 7f1e127fc700 1 -- 172.21.15.6:0/179745374 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e0c02c020 con 0x7f1e08003bc0 2023-11-28T13:01:40.420 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:40.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.420+0000 7f1e23ee9700 1 -- 172.21.15.6:0/179745374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1e000682f0 msgr2=0x7f1e0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1e000682f0 0x7f1e0006a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1e18000fd0 tx=0x7f1e18009040 comp rx=0 tx=0).stop 2023-11-28T13:01:40.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 -- 172.21.15.6:0/179745374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 msgr2=0x7f1e1c0ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 0x7f1e1c0ce640 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1e08004780 tx=0x7f1e0c0047c0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 -- 172.21.15.6:0/179745374 shutdown_connections 2023-11-28T13:01:40.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e1c0ad030 0x7f1e1c14db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1e08003bc0 0x7f1e1c0ce640 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1e000682f0 0x7f1e0006a7a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 --2- 172.21.15.6:0/179745374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1e1c0c7cd0 0x7f1e1c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 -- 172.21.15.6:0/179745374 >> 172.21.15.6:0/179745374 conn(0x7f1e1c0cd680 msgr2=0x7f1e1c0b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 -- 172.21.15.6:0/179745374 shutdown_connections 2023-11-28T13:01:40.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.421+0000 7f1e23ee9700 1 -- 172.21.15.6:0/179745374 wait complete. 2023-11-28T13:01:40.434 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:01:40.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.557+0000 7f06c7fd1700 1 Processor -- start 2023-11-28T13:01:40.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.557+0000 7f06c7fd1700 1 -- start start 2023-11-28T13:01:40.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.558+0000 7f06c7fd1700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f06c015a6c0 0x7f06c00cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.558+0000 7f06c7fd1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f06c00b2660 0x7f06c00b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.558+0000 7f06c7fd1700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c00cfb90 con 0x7f06c00d0020 2023-11-28T13:01:40.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.558+0000 7f06c7fd1700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00b3020 con 0x7f06c015a6c0 2023-11-28T13:01:40.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.558+0000 7f06c7fd1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00cb930 con 0x7f06c00b2660 2023-11-28T13:01:40.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.558+0000 7f06c5d6d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f06c015a6c0 0x7f06c00cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.558+0000 7f06c5d6d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f06c015a6c0 0x7f06c00cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60310/0 (socket says 172.21.15.6:60310) 2023-11-28T13:01:40.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.558+0000 7f06c5d6d700 1 -- 172.21.15.6:0/1956027432 learned_addr learned my addr 172.21.15.6:0/1956027432 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:40.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.558+0000 7f06c656e700 1 --2- 172.21.15.6:0/1956027432 >> v2:172.21.15.61:3300/0 conn(0x7f06c00b2660 0x7f06c00b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c5d6d700 1 -- 172.21.15.6:0/1956027432 >> v2:172.21.15.61:3300/0 conn(0x7f06c00b2660 msgr2=0x7f06c00b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c5d6d700 1 --2- 172.21.15.6:0/1956027432 >> v2:172.21.15.61:3300/0 conn(0x7f06c00b2660 0x7f06c00b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c5d6d700 1 -- 172.21.15.6:0/1956027432 >> v1:172.21.15.61:6789/0 conn(0x7f06c00d0020 legacy=0x7f06c00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c5d6d700 1 -- 172.21.15.6:0/1956027432 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00cf260 con 0x7f06c015a6c0 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c5d6d700 1 --2- 172.21.15.6:0/1956027432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c015a6c0 0x7f06c00cb230 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f06c00cb1c0 tx=0x7f06b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5838f1e4a43f7c4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c4d6b700 1 -- 172.21.15.6:0/1956027432 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0004230 con 0x7f06c015a6c0 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c4d6b700 1 -- 172.21.15.6:0/1956027432 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b0004390 con 0x7f06c015a6c0 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c4d6b700 1 -- 172.21.15.6:0/1956027432 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00036a0 con 0x7f06c015a6c0 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c7fd1700 1 -- 172.21.15.6:0/1956027432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c015a6c0 msgr2=0x7f06c00cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c7fd1700 1 --2- 172.21.15.6:0/1956027432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c015a6c0 0x7f06c00cb230 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f06c00cb1c0 tx=0x7f06b0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c7fd1700 1 -- 172.21.15.6:0/1956027432 shutdown_connections 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c7fd1700 1 --2- 172.21.15.6:0/1956027432 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c015a6c0 0x7f06c00cb230 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c7fd1700 1 --2- 172.21.15.6:0/1956027432 >> v2:172.21.15.61:3300/0 conn(0x7f06c00b2660 0x7f06c00b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.559+0000 7f06c7fd1700 1 -- 172.21.15.6:0/1956027432 >> 172.21.15.6:0/1956027432 conn(0x7f06c00cd660 msgr2=0x7f06c00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.560+0000 7f06c7fd1700 1 -- 172.21.15.6:0/1956027432 shutdown_connections 2023-11-28T13:01:40.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.560+0000 7f06c7fd1700 1 -- 172.21.15.6:0/1956027432 wait complete. 2023-11-28T13:01:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.561+0000 7f06c7fd1700 1 Processor -- start 2023-11-28T13:01:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.561+0000 7f06c7fd1700 1 -- start start 2023-11-28T13:01:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.561+0000 7f06c7fd1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06c00b2660 0x7f06c0156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.561+0000 7f06c7fd1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c00d0020 0x7f06c006c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.561+0000 7f06c7fd1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06c015a6c0 0x7f06c006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.561+0000 7f06c7fd1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c006bb80 con 0x7f06c00d0020 2023-11-28T13:01:40.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.561+0000 7f06c7fd1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c015d8a0 con 0x7f06c015a6c0 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.561+0000 7f06c7fd1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c006bd90 con 0x7f06c00b2660 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c556c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c00d0020 0x7f06c006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c556c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c00d0020 0x7f06c006c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60316/0 (socket says 172.21.15.6:60316) 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c556c700 1 -- 172.21.15.6:0/834555256 learned_addr learned my addr 172.21.15.6:0/834555256 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c5d6d700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06c00b2660 0x7f06c0156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c556c700 1 -- 172.21.15.6:0/834555256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06c00b2660 msgr2=0x7f06c0156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c556c700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06c00b2660 0x7f06c0156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c656e700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06c015a6c0 0x7f06c006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c556c700 1 -- 172.21.15.6:0/834555256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06c015a6c0 msgr2=0x7f06c006d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c556c700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06c015a6c0 0x7f06c006d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c556c700 1 -- 172.21.15.6:0/834555256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b0009550 con 0x7f06c00d0020 2023-11-28T13:01:40.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c556c700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c00d0020 0x7f06c006c870 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00c4d0 tx=0x7f06bc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06b67fc700 1 -- 172.21.15.6:0/834555256 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc011790 con 0x7f06c00d0020 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06b67fc700 1 -- 172.21.15.6:0/834555256 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc0118f0 con 0x7f06c00d0020 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c7fd1700 1 -- 172.21.15.6:0/834555256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c00cf820 con 0x7f06c00d0020 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06b67fc700 1 -- 172.21.15.6:0/834555256 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc011ca0 con 0x7f06c00d0020 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.562+0000 7f06c7fd1700 1 -- 172.21.15.6:0/834555256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c00cf540 con 0x7f06c00d0020 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.563+0000 7f06c7fd1700 1 -- 172.21.15.6:0/834555256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c00d0470 con 0x7f06c00d0020 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.564+0000 7f06b67fc700 1 -- 172.21.15.6:0/834555256 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f06bc010050 con 0x7f06c00d0020 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.564+0000 7f06b67fc700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06a40682f0 0x7f06a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.564+0000 7f06b67fc700 1 -- 172.21.15.6:0/834555256 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f06bc085760 con 0x7f06c00d0020 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.564+0000 7f06c5d6d700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06a40682f0 0x7f06a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.565+0000 7f06c5d6d700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06a40682f0 0x7f06a406a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f06c00727d0 tx=0x7f06b0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.568+0000 7f06b67fc700 1 -- 172.21.15.6:0/834555256 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f06bc0521d0 con 0x7f06c00d0020 2023-11-28T13:01:40.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.762+0000 7f06c7fd1700 1 -- 172.21.15.6:0/834555256 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f06c00675a0 con 0x7f06c00d0020 2023-11-28T13:01:40.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.763+0000 7f06b67fc700 1 -- 172.21.15.6:0/834555256 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+126 (secure 0 0 0) 0x7f06bc0557f0 con 0x7f06c00d0020 2023-11-28T13:01:40.764 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:40.764 INFO:teuthology.orchestra.run.smithi006.stdout:[ 2023-11-28T13:01:40.764 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:40.764 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 1, 2023-11-28T13:01:40.765 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": "rbd" 2023-11-28T13:01:40.765 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:40.765 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:01:40.765 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 2, 2023-11-28T13:01:40.765 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": ".mgr" 2023-11-28T13:01:40.765 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:40.765 INFO:teuthology.orchestra.run.smithi006.stdout:] 2023-11-28T13:01:40.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 -- 172.21.15.6:0/834555256 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06a40682f0 msgr2=0x7f06a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06a40682f0 0x7f06a406a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f06c00727d0 tx=0x7f06b0008030 comp rx=0 tx=0).stop 2023-11-28T13:01:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 -- 172.21.15.6:0/834555256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c00d0020 msgr2=0x7f06c006c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c00d0020 0x7f06c006c870 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00c4d0 tx=0x7f06bc00c890 comp rx=0 tx=0).stop 2023-11-28T13:01:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 -- 172.21.15.6:0/834555256 shutdown_connections 2023-11-28T13:01:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06c015a6c0 0x7f06c006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06a40682f0 0x7f06a406a7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06c00b2660 0x7f06c0156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 --2- 172.21.15.6:0/834555256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06c00d0020 0x7f06c006c870 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 -- 172.21.15.6:0/834555256 >> 172.21.15.6:0/834555256 conn(0x7f06c00cd660 msgr2=0x7f06c00b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 -- 172.21.15.6:0/834555256 shutdown_connections 2023-11-28T13:01:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.765+0000 7f06c7fd1700 1 -- 172.21.15.6:0/834555256 wait complete. 2023-11-28T13:01:40.780 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:01:40.780 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:01:40.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.907+0000 7f1541c78700 1 Processor -- start 2023-11-28T13:01:40.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.908+0000 7f1541c78700 1 -- start start 2023-11-28T13:01:40.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.908+0000 7f1541c78700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f153c15c560 0x7f153c06f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.908+0000 7f1541c78700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c0b30b0 con 0x7f153c0b2660 2023-11-28T13:01:40.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.908+0000 7f1541c78700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c06c020 con 0x7f153c0ceec0 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.908+0000 7f1541c78700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153c0d09c0 con 0x7f153c15c560 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.908+0000 7f153b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f153c15c560 0x7f153c06f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.908+0000 7f153affd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f153c0b2660 0x7f153c0ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47644/0 (socket says 172.21.15.6:47644) 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.908+0000 7f153b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f153c15c560 0x7f153c06f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60322/0 (socket says 172.21.15.6:60322) 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.908+0000 7f153affd700 1 -- 172.21.15.6:0/1258179376 learned_addr learned my addr 172.21.15.6:0/1258179376 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.909+0000 7f153b7fe700 1 -- 172.21.15.6:0/1258179376 >> v1:172.21.15.61:6789/0 conn(0x7f153c0ceec0 legacy=0x7f153c06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.909+0000 7f153b7fe700 1 -- 172.21.15.6:0/1258179376 >> v1:172.21.15.6:6789/0 conn(0x7f153c0b2660 legacy=0x7f153c0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.909+0000 7f153b7fe700 1 -- 172.21.15.6:0/1258179376 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c06fdf0 con 0x7f153c15c560 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.909+0000 7f153a7fc700 1 -- 172.21.15.6:0/1258179376 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740372636 0 0) 0x7f153c0b30b0 con 0x7f153c0b2660 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.909+0000 7f153b7fe700 1 --2- 172.21.15.6:0/1258179376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c15c560 0x7f153c06f8b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1524009840 tx=0x7f1524009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b087ab3876c4b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.909+0000 7f153a7fc700 1 -- 172.21.15.6:0/1258179376 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524011830 con 0x7f153c15c560 2023-11-28T13:01:40.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.909+0000 7f153a7fc700 1 -- 172.21.15.6:0/1258179376 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1524011990 con 0x7f153c15c560 2023-11-28T13:01:40.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.909+0000 7f153a7fc700 1 -- 172.21.15.6:0/1258179376 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524011d20 con 0x7f153c15c560 2023-11-28T13:01:40.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.910+0000 7f1541c78700 1 -- 172.21.15.6:0/1258179376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c15c560 msgr2=0x7f153c06f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.910+0000 7f1541c78700 1 --2- 172.21.15.6:0/1258179376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c15c560 0x7f153c06f8b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1524009840 tx=0x7f1524009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:40.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.910+0000 7f1541c78700 1 -- 172.21.15.6:0/1258179376 shutdown_connections 2023-11-28T13:01:40.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.910+0000 7f1541c78700 1 --2- 172.21.15.6:0/1258179376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c15c560 0x7f153c06f8b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.910+0000 7f1541c78700 1 -- 172.21.15.6:0/1258179376 >> 172.21.15.6:0/1258179376 conn(0x7f153c0cd660 msgr2=0x7f153c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:40.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.910+0000 7f1541c78700 1 -- 172.21.15.6:0/1258179376 shutdown_connections 2023-11-28T13:01:40.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.910+0000 7f1541c78700 1 -- 172.21.15.6:0/1258179376 wait complete. 2023-11-28T13:01:40.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.911+0000 7f1541c78700 1 Processor -- start 2023-11-28T13:01:40.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.911+0000 7f1541c78700 1 -- start start 2023-11-28T13:01:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f1541c78700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f153c0b2660 0x7f153c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f1541c78700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c0ceec0 0x7f153c156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f1541c78700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f153c15c560 0x7f153c1573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f1541c78700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c1602c0 con 0x7f153c0ceec0 2023-11-28T13:01:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f1541c78700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c1578f0 con 0x7f153c0b2660 2023-11-28T13:01:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f1541c78700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c15db20 con 0x7f153c15c560 2023-11-28T13:01:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c0ceec0 0x7f153c156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c0ceec0 0x7f153c156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60330/0 (socket says 172.21.15.6:60330) 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153affd700 1 -- 172.21.15.6:0/3557630379 learned_addr learned my addr 172.21.15.6:0/3557630379 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153b7fe700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f153c0b2660 0x7f153c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153bfff700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f153c15c560 0x7f153c1573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153affd700 1 -- 172.21.15.6:0/3557630379 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f153c15c560 msgr2=0x7f153c1573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153affd700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f153c15c560 0x7f153c1573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153affd700 1 -- 172.21.15.6:0/3557630379 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f153c0b2660 msgr2=0x7f153c06d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153affd700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f153c0b2660 0x7f153c06d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153affd700 1 -- 172.21.15.6:0/3557630379 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15240094a0 con 0x7f153c0ceec0 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.912+0000 7f153affd700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c0ceec0 0x7f153c156e70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f152c00c520 tx=0x7f152c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.913+0000 7f1533fff700 1 -- 172.21.15.6:0/3557630379 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c004c80 con 0x7f153c0ceec0 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.913+0000 7f1533fff700 1 -- 172.21.15.6:0/3557630379 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c004de0 con 0x7f153c0ceec0 2023-11-28T13:01:40.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.913+0000 7f1541c78700 1 -- 172.21.15.6:0/3557630379 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c0b2dd0 con 0x7f153c0ceec0 2023-11-28T13:01:40.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.913+0000 7f1533fff700 1 -- 172.21.15.6:0/3557630379 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c007310 con 0x7f153c0ceec0 2023-11-28T13:01:40.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.913+0000 7f1541c78700 1 -- 172.21.15.6:0/3557630379 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c0b2af0 con 0x7f153c0ceec0 2023-11-28T13:01:40.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.914+0000 7f1541c78700 1 -- 172.21.15.6:0/3557630379 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153c06bcb0 con 0x7f153c0ceec0 2023-11-28T13:01:40.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.914+0000 7f1533fff700 1 -- 172.21.15.6:0/3557630379 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f152c01a020 con 0x7f153c0ceec0 2023-11-28T13:01:40.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.915+0000 7f1533fff700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f151c0682f0 0x7f151c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:40.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.915+0000 7f153b7fe700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f151c0682f0 0x7f151c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:40.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.915+0000 7f1533fff700 1 -- 172.21.15.6:0/3557630379 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f152c085660 con 0x7f153c0ceec0 2023-11-28T13:01:40.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.915+0000 7f153b7fe700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f151c0682f0 0x7f151c06a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f15240090b0 tx=0x7f152401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:40.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:40.919+0000 7f1533fff700 1 -- 172.21.15.6:0/3557630379 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f152c051fe0 con 0x7f153c0ceec0 2023-11-28T13:01:41.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.113+0000 7f1541c78700 1 -- 172.21.15.6:0/3557630379 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f153c0739f0 con 0x7f153c0ceec0 2023-11-28T13:01:41.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.167+0000 7f1533fff700 1 -- 172.21.15.6:0/3557630379 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7f152c055600 con 0x7f153c0ceec0 2023-11-28T13:01:41.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.172+0000 7f1541c78700 1 -- 172.21.15.6:0/3557630379 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f151c0682f0 msgr2=0x7f151c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.172+0000 7f1541c78700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f151c0682f0 0x7f151c06a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f15240090b0 tx=0x7f152401c040 comp rx=0 tx=0).stop 2023-11-28T13:01:41.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.172+0000 7f1541c78700 1 -- 172.21.15.6:0/3557630379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c0ceec0 msgr2=0x7f153c156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.172+0000 7f1541c78700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c0ceec0 0x7f153c156e70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f152c00c520 tx=0x7f152c00c830 comp rx=0 tx=0).stop 2023-11-28T13:01:41.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.172+0000 7f1541c78700 1 -- 172.21.15.6:0/3557630379 shutdown_connections 2023-11-28T13:01:41.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.172+0000 7f1541c78700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f151c0682f0 0x7f151c06a7a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.172+0000 7f1541c78700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f153c15c560 0x7f153c1573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.172+0000 7f1541c78700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f153c0b2660 0x7f153c06d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.172+0000 7f1541c78700 1 --2- 172.21.15.6:0/3557630379 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f153c0ceec0 0x7f153c156e70 secure :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f152c00c520 tx=0x7f152c00c830 comp rx=0 tx=0).stop 2023-11-28T13:01:41.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.172+0000 7f1541c78700 1 -- 172.21.15.6:0/3557630379 >> 172.21.15.6:0/3557630379 conn(0x7f153c0cd660 msgr2=0x7f153c0b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:41.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.174+0000 7f1541c78700 1 -- 172.21.15.6:0/3557630379 shutdown_connections 2023-11-28T13:01:41.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.174+0000 7f1541c78700 1 -- 172.21.15.6:0/3557630379 wait complete. 2023-11-28T13:01:41.176 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' created 2023-11-28T13:01:41.188 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:01:41.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.318+0000 7fcb6ad8f700 1 Processor -- start 2023-11-28T13:01:41.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.318+0000 7fcb6ad8f700 1 -- start start 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.319+0000 7fcb6ad8f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcb64159500 0x7fcb640cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.319+0000 7fcb6ad8f700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6415e040 con 0x7fcb6415dbf0 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.319+0000 7fcb6ad8f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb640ceff0 con 0x7fcb640b1d40 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.319+0000 7fcb6ad8f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb640cf2d0 con 0x7fcb64159500 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.319+0000 7fcb63fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fcb6415dbf0 0x7fcb640b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47656/0 (socket says 172.21.15.6:47656) 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.319+0000 7fcb63fff700 1 -- 172.21.15.6:0/567955474 learned_addr learned my addr 172.21.15.6:0/567955474 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb68b2b700 1 --2- 172.21.15.6:0/567955474 >> v2:172.21.15.61:3300/0 conn(0x7fcb64159500 0x7fcb640cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997653180 0 0) 0x7fcb6415e040 con 0x7fcb6415dbf0 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb540035b0 con 0x7fcb6415dbf0 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477550016 0 0) 0x7fcb540035b0 con 0x7fcb6415dbf0 2023-11-28T13:01:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 >> v2:172.21.15.61:3300/0 conn(0x7fcb64159500 msgr2=0x7fcb640cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 --2- 172.21.15.6:0/567955474 >> v2:172.21.15.61:3300/0 conn(0x7fcb64159500 0x7fcb640cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 >> v1:172.21.15.61:6789/0 conn(0x7fcb640b1d40 legacy=0x7fcb6406bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6415d110 con 0x7fcb6415dbf0 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fcb50001670 con 0x7fcb6415dbf0 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 >> v1:172.21.15.6:6789/0 conn(0x7fcb6415dbf0 legacy=0x7fcb640b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb54003a00 0x7fcb54003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb54004850 0x7fcb54004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb54005200 0x7fcb5400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640cf2d0 con 0x7fcb54004850 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5400c060 con 0x7fcb54005200 2023-11-28T13:01:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5400c220 con 0x7fcb54003a00 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb63fff700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb54004850 0x7fcb54004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb68b2b700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb54003a00 0x7fcb54003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb6932c700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb54005200 0x7fcb5400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.320+0000 7fcb63fff700 1 -- 172.21.15.6:0/567955474 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb54003a00 msgr2=0x7fcb54003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb63fff700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb54003a00 0x7fcb54003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb63fff700 1 -- 172.21.15.6:0/567955474 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb54005200 msgr2=0x7fcb5400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb63fff700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb54005200 0x7fcb5400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb63fff700 1 -- 172.21.15.6:0/567955474 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540035b0 con 0x7fcb54004850 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb6932c700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb54005200 0x7fcb5400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb6932c700 1 -- 172.21.15.6:0/567955474 reap_dead start 2023-11-28T13:01:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb63fff700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb54004850 0x7fcb54004cc0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c210 tx=0x7fcb54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c66e9785f38851e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50007c10 con 0x7fcb54004850 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5000cdb0 con 0x7fcb54004850 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb637fe700 1 -- 172.21.15.6:0/567955474 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50005ba0 con 0x7fcb54004850 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/567955474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb54004850 msgr2=0x7fcb54004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb6ad8f700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb54004850 0x7fcb54004cc0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c210 tx=0x7fcb54005880 comp rx=0 tx=0).stop 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/567955474 shutdown_connections 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb6ad8f700 1 --2- 172.21.15.6:0/567955474 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb54004850 0x7fcb54004cc0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/567955474 >> 172.21.15.6:0/567955474 conn(0x7fcb640cc630 msgr2=0x7fcb640cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/567955474 shutdown_connections 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.321+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/567955474 wait complete. 2023-11-28T13:01:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.323+0000 7fcb6ad8f700 1 Processor -- start 2023-11-28T13:01:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.323+0000 7fcb6ad8f700 1 -- start start 2023-11-28T13:01:41.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6ad8f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb54004850 0x7fcb641e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6ad8f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb641551a0 0x7fcb641e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6ad8f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb640b4af0 0x7fcb640b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6ad8f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640c3fe0 con 0x7fcb640b4af0 2023-11-28T13:01:41.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6ad8f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640c4400 con 0x7fcb641551a0 2023-11-28T13:01:41.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6ad8f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640b54d0 con 0x7fcb54004850 2023-11-28T13:01:41.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6932c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb640b4af0 0x7fcb640b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6932c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb640b4af0 0x7fcb640b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60352/0 (socket says 172.21.15.6:60352) 2023-11-28T13:01:41.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6932c700 1 -- 172.21.15.6:0/3511061814 learned_addr learned my addr 172.21.15.6:0/3511061814 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:41.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb63fff700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb641551a0 0x7fcb641e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb68b2b700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb54004850 0x7fcb641e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6932c700 1 -- 172.21.15.6:0/3511061814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb54004850 msgr2=0x7fcb641e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6932c700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb54004850 0x7fcb641e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6932c700 1 -- 172.21.15.6:0/3511061814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb641551a0 msgr2=0x7fcb641e69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:41.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6932c700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb641551a0 0x7fcb641e69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:41.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6932c700 1 -- 172.21.15.6:0/3511061814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6415d3f0 con 0x7fcb640b4af0 2023-11-28T13:01:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6932c700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb640b4af0 0x7fcb640b4f90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcb6406eb30 tx=0x7fcb5c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb617fa700 1 -- 172.21.15.6:0/3511061814 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c00eed0 con 0x7fcb640b4af0 2023-11-28T13:01:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb617fa700 1 -- 172.21.15.6:0/3511061814 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c003c10 con 0x7fcb640b4af0 2023-11-28T13:01:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb617fa700 1 -- 172.21.15.6:0/3511061814 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0050d0 con 0x7fcb640b4af0 2023-11-28T13:01:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.324+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/3511061814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb640caf40 con 0x7fcb640b4af0 2023-11-28T13:01:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.325+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/3511061814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6415d6d0 con 0x7fcb640b4af0 2023-11-28T13:01:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.325+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/3511061814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb640c4120 con 0x7fcb640b4af0 2023-11-28T13:01:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.326+0000 7fcb617fa700 1 -- 172.21.15.6:0/3511061814 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcb5c005230 con 0x7fcb640b4af0 2023-11-28T13:01:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.326+0000 7fcb617fa700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb480682f0 0x7fcb4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:41.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.327+0000 7fcb617fa700 1 -- 172.21.15.6:0/3511061814 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fcb5c0864a0 con 0x7fcb640b4af0 2023-11-28T13:01:41.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.327+0000 7fcb68b2b700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb480682f0 0x7fcb4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:41.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.327+0000 7fcb68b2b700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb480682f0 0x7fcb4806a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009ce0 tx=0x7fcb58009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:41.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.331+0000 7fcb617fa700 1 -- 172.21.15.6:0/3511061814 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcb5c056870 con 0x7fcb640b4af0 2023-11-28T13:01:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:41.523+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/3511061814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcb640c4120 con 0x7fcb640b4af0 2023-11-28T13:01:42.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.174+0000 7fcb617fa700 1 -- 172.21.15.6:0/3511061814 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcb640c4120 con 0x7fcb640b4af0 2023-11-28T13:01:42.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.180+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/3511061814 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb480682f0 msgr2=0x7fcb4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.182 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.180+0000 7fcb6ad8f700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb480682f0 0x7fcb4806a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009ce0 tx=0x7fcb58009210 comp rx=0 tx=0).stop 2023-11-28T13:01:42.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.180+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/3511061814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb640b4af0 msgr2=0x7fcb640b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.180+0000 7fcb6ad8f700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb640b4af0 0x7fcb640b4f90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcb6406eb30 tx=0x7fcb5c00ed50 comp rx=0 tx=0).stop 2023-11-28T13:01:42.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.181+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/3511061814 shutdown_connections 2023-11-28T13:01:42.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.181+0000 7fcb6ad8f700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb640b4af0 0x7fcb640b4f90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.181+0000 7fcb6ad8f700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcb480682f0 0x7fcb4806a7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.181+0000 7fcb6ad8f700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcb54004850 0x7fcb641e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.181+0000 7fcb6ad8f700 1 --2- 172.21.15.6:0/3511061814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb641551a0 0x7fcb641e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.181+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/3511061814 >> 172.21.15.6:0/3511061814 conn(0x7fcb640cc630 msgr2=0x7fcb6415f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:42.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.181+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/3511061814 shutdown_connections 2023-11-28T13:01:42.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.182+0000 7fcb6ad8f700 1 -- 172.21.15.6:0/3511061814 wait complete. 2023-11-28T13:01:42.190 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' created 2023-11-28T13:01:42.208 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:01:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.340+0000 7f26b1f26700 1 Processor -- start 2023-11-28T13:01:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.340+0000 7f26b1f26700 1 -- start start 2023-11-28T13:01:42.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.340+0000 7f26b1f26700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f26ac165670 0x7f26ac167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.340+0000 7f26b1f26700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac0ce550 con 0x7f26ac0b1630 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.340+0000 7f26b1f26700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26ac0ce830 con 0x7f26ac0cf5f0 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.340+0000 7f26b1f26700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26ac0ceb10 con 0x7f26ac165670 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26abfff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f26ac165670 0x7f26ac167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26ab7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f26ac0b1630 0x7f26ac0b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47676/0 (socket says 172.21.15.6:47676) 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26ab7fe700 1 -- 172.21.15.6:0/2939352050 learned_addr learned my addr 172.21.15.6:0/2939352050 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26abfff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f26ac165670 0x7f26ac167a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60360/0 (socket says 172.21.15.6:60360) 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26abfff700 1 -- 172.21.15.6:0/2939352050 >> v1:172.21.15.61:6789/0 conn(0x7f26ac0cf5f0 legacy=0x7f26ac0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26abfff700 1 -- 172.21.15.6:0/2939352050 >> v1:172.21.15.6:6789/0 conn(0x7f26ac0b1630 legacy=0x7f26ac0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26abfff700 1 -- 172.21.15.6:0/2939352050 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac0cedf0 con 0x7f26ac165670 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26abfff700 1 --2- 172.21.15.6:0/2939352050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 0x7f26ac167a50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009dc0 tx=0x7f26a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2eb6348cbeedd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26aa7fc700 1 -- 172.21.15.6:0/2939352050 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a400e070 con 0x7f26ac165670 2023-11-28T13:01:42.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26aa7fc700 1 -- 172.21.15.6:0/2939352050 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a400bb50 con 0x7f26ac165670 2023-11-28T13:01:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.341+0000 7f26aa7fc700 1 -- 172.21.15.6:0/2939352050 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4003a20 con 0x7f26ac165670 2023-11-28T13:01:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.342+0000 7f26b1f26700 1 -- 172.21.15.6:0/2939352050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 msgr2=0x7f26ac167a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.342+0000 7f26b1f26700 1 --2- 172.21.15.6:0/2939352050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 0x7f26ac167a50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009dc0 tx=0x7f26a400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.342+0000 7f26b1f26700 1 -- 172.21.15.6:0/2939352050 shutdown_connections 2023-11-28T13:01:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.342+0000 7f26b1f26700 1 --2- 172.21.15.6:0/2939352050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 0x7f26ac167a50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.342+0000 7f26b1f26700 1 -- 172.21.15.6:0/2939352050 >> 172.21.15.6:0/2939352050 conn(0x7f26ac0cc630 msgr2=0x7f26ac0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.342+0000 7f26b1f26700 1 -- 172.21.15.6:0/2939352050 shutdown_connections 2023-11-28T13:01:42.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.342+0000 7f26b1f26700 1 -- 172.21.15.6:0/2939352050 wait complete. 2023-11-28T13:01:42.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.343+0000 7f26b1f26700 1 Processor -- start 2023-11-28T13:01:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.343+0000 7f26b1f26700 1 -- start start 2023-11-28T13:01:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26b1f26700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26ac0b1630 0x7f26ac0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26b1f26700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26ac0cf5f0 0x7f26ac154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26b1f26700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 0x7f26ac15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26b1f26700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0b39c0 con 0x7f26ac165670 2023-11-28T13:01:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26b1f26700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac15f9c0 con 0x7f26ac0cf5f0 2023-11-28T13:01:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26b1f26700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0b3c60 con 0x7f26ac0b1630 2023-11-28T13:01:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26abfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 0x7f26ac15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26abfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 0x7f26ac15f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60366/0 (socket says 172.21.15.6:60366) 2023-11-28T13:01:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26abfff700 1 -- 172.21.15.6:0/2010088708 learned_addr learned my addr 172.21.15.6:0/2010088708 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:42.350 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26ab7fe700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26ac0b1630 0x7f26ac0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26aaffd700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26ac0cf5f0 0x7f26ac154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26abfff700 1 -- 172.21.15.6:0/2010088708 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26ac0b1630 msgr2=0x7f26ac0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26abfff700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26ac0b1630 0x7f26ac0c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26abfff700 1 -- 172.21.15.6:0/2010088708 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26ac0cf5f0 msgr2=0x7f26ac154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26abfff700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26ac0cf5f0 0x7f26ac154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.351 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.344+0000 7f26abfff700 1 -- 172.21.15.6:0/2010088708 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a4009a70 con 0x7f26ac165670 2023-11-28T13:01:42.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.345+0000 7f26abfff700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 0x7f26ac15f3c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f26a400b370 tx=0x7f26a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.345+0000 7f2693fff700 1 -- 172.21.15.6:0/2010088708 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4012430 con 0x7f26ac165670 2023-11-28T13:01:42.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.345+0000 7f2693fff700 1 -- 172.21.15.6:0/2010088708 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a40125d0 con 0x7f26ac165670 2023-11-28T13:01:42.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.345+0000 7f26b1f26700 1 -- 172.21.15.6:0/2010088708 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac0cb5b0 con 0x7f26ac165670 2023-11-28T13:01:42.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.345+0000 7f2693fff700 1 -- 172.21.15.6:0/2010088708 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4012980 con 0x7f26ac165670 2023-11-28T13:01:42.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.345+0000 7f26b1f26700 1 -- 172.21.15.6:0/2010088708 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac0cf0d0 con 0x7f26ac165670 2023-11-28T13:01:42.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.346+0000 7f26b1f26700 1 -- 172.21.15.6:0/2010088708 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac0ce140 con 0x7f26ac165670 2023-11-28T13:01:42.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.346+0000 7f2693fff700 1 -- 172.21.15.6:0/2010088708 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f26a4012ae0 con 0x7f26ac165670 2023-11-28T13:01:42.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.347+0000 7f2693fff700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26980682f0 0x7f269806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.347+0000 7f2693fff700 1 -- 172.21.15.6:0/2010088708 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f26a4088470 con 0x7f26ac165670 2023-11-28T13:01:42.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.347+0000 7f26ab7fe700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26980682f0 0x7f269806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.347+0000 7f26ab7fe700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26980682f0 0x7f269806a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f269c005910 tx=0x7f269c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.351+0000 7f2693fff700 1 -- 172.21.15.6:0/2010088708 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f26a4054bf0 con 0x7f26ac165670 2023-11-28T13:01:42.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.593+0000 7f26b1f26700 1 -- 172.21.15.6:0/2010088708 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f26ac0739f0 con 0x7f26ac165670 2023-11-28T13:01:42.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.600+0000 7f2693fff700 1 -- 172.21.15.6:0/2010088708 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f26a4058210 con 0x7f26ac165670 2023-11-28T13:01:42.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.604+0000 7f26b1f26700 1 -- 172.21.15.6:0/2010088708 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26980682f0 msgr2=0x7f269806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.604+0000 7f26b1f26700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26980682f0 0x7f269806a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f269c005910 tx=0x7f269c009040 comp rx=0 tx=0).stop 2023-11-28T13:01:42.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.604+0000 7f26b1f26700 1 -- 172.21.15.6:0/2010088708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 msgr2=0x7f26ac15f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.604+0000 7f26b1f26700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 0x7f26ac15f3c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f26a400b370 tx=0x7f26a4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.611+0000 7f26b1f26700 1 -- 172.21.15.6:0/2010088708 shutdown_connections 2023-11-28T13:01:42.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.611+0000 7f26b1f26700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26ac165670 0x7f26ac15f3c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.611+0000 7f26b1f26700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26980682f0 0x7f269806a7a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.611+0000 7f26b1f26700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26ac0b1630 0x7f26ac0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.611+0000 7f26b1f26700 1 --2- 172.21.15.6:0/2010088708 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26ac0cf5f0 0x7f26ac154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.611+0000 7f26b1f26700 1 -- 172.21.15.6:0/2010088708 >> 172.21.15.6:0/2010088708 conn(0x7f26ac0cc630 msgr2=0x7f26ac0caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.611+0000 7f26b1f26700 1 -- 172.21.15.6:0/2010088708 shutdown_connections 2023-11-28T13:01:42.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.611+0000 7f26b1f26700 1 -- 172.21.15.6:0/2010088708 wait complete. 2023-11-28T13:01:42.630 INFO:teuthology.orchestra.run.smithi006.stderr:new fs with metadata pool 6 and data pool 7 2023-11-28T13:01:42.637 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:42.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.760+0000 7f01bd7c4700 1 Processor -- start 2023-11-28T13:01:42.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.760+0000 7f01bd7c4700 1 -- start start 2023-11-28T13:01:42.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.761+0000 7f01bd7c4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f01b8159590 0x7f01b806e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.761+0000 7f01bd7c4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 0x7f01b80ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.761+0000 7f01bd7c4700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f01b80ce890 0x7f01b80ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.761+0000 7f01bd7c4700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b80cf250 con 0x7f01b80ce890 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.761+0000 7f01bd7c4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b815d120 con 0x7f01b8159590 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.761+0000 7f01bd7c4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b815d6e0 con 0x7f01b815db70 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.761+0000 7f01b77fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f01b80ce890 0x7f01b80ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.761+0000 7f01b77fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f01b80ce890 0x7f01b80ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60388/0 (socket says 172.21.15.6:60388) 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.761+0000 7f01b77fe700 1 -- 172.21.15.6:0/1940803249 learned_addr learned my addr 172.21.15.6:0/1940803249 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b6ffd700 1 --2- 172.21.15.6:0/1940803249 >> v2:172.21.15.61:3300/0 conn(0x7f01b8159590 0x7f01b806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b67fc700 1 --2- 172.21.15.6:0/1940803249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 0x7f01b80ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b77fe700 1 -- 172.21.15.6:0/1940803249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 msgr2=0x7f01b80ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b77fe700 1 --2- 172.21.15.6:0/1940803249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 0x7f01b80ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b77fe700 1 -- 172.21.15.6:0/1940803249 >> v2:172.21.15.61:3300/0 conn(0x7f01b8159590 msgr2=0x7f01b806e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b77fe700 1 --2- 172.21.15.6:0/1940803249 >> v2:172.21.15.61:3300/0 conn(0x7f01b8159590 0x7f01b806e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b77fe700 1 -- 172.21.15.6:0/1940803249 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b80aef60 con 0x7f01b80ce890 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b67fc700 1 --2- 172.21.15.6:0/1940803249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 0x7f01b80ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b6ffd700 1 --2- 172.21.15.6:0/1940803249 >> v2:172.21.15.61:3300/0 conn(0x7f01b8159590 0x7f01b806e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b77fe700 1 --2- 172.21.15.6:0/1940803249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 0x7f01b80ced10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f01b806bd10 tx=0x7f01b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7ea8bd62f66f31 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b5ffb700 1 -- 172.21.15.6:0/1940803249 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b000e070 con 0x7f01b80ce890 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b5ffb700 1 -- 172.21.15.6:0/1940803249 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b000b9c0 con 0x7f01b80ce890 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.762+0000 7f01b5ffb700 1 -- 172.21.15.6:0/1940803249 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b000bd70 con 0x7f01b80ce890 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.763+0000 7f01bd7c4700 1 -- 172.21.15.6:0/1940803249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 msgr2=0x7f01b80ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.763+0000 7f01bd7c4700 1 --2- 172.21.15.6:0/1940803249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 0x7f01b80ced10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f01b806bd10 tx=0x7f01b000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.763+0000 7f01bd7c4700 1 -- 172.21.15.6:0/1940803249 shutdown_connections 2023-11-28T13:01:42.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.763+0000 7f01bd7c4700 1 --2- 172.21.15.6:0/1940803249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 0x7f01b80ced10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.763+0000 7f01bd7c4700 1 --2- 172.21.15.6:0/1940803249 >> v2:172.21.15.61:3300/0 conn(0x7f01b8159590 0x7f01b806e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.763+0000 7f01bd7c4700 1 --2- 172.21.15.6:0/1940803249 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 0x7f01b80ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.763+0000 7f01bd7c4700 1 -- 172.21.15.6:0/1940803249 >> 172.21.15.6:0/1940803249 conn(0x7f01b80b1650 msgr2=0x7f01b80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:42.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.763+0000 7f01bd7c4700 1 -- 172.21.15.6:0/1940803249 shutdown_connections 2023-11-28T13:01:42.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.763+0000 7f01bd7c4700 1 -- 172.21.15.6:0/1940803249 wait complete. 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.764+0000 7f01bd7c4700 1 Processor -- start 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.764+0000 7f01bd7c4700 1 -- start start 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01bd7c4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 0x7f01b80ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01bd7c4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01b8159590 0x7f01b806ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01bd7c4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 0x7f01b80abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01bd7c4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b80ac890 con 0x7f01b80ce890 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01bd7c4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b80adc00 con 0x7f01b8159590 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01bd7c4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b80ad2d0 con 0x7f01b815db70 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01b6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 0x7f01b80ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01b6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 0x7f01b80ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60390/0 (socket says 172.21.15.6:60390) 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01b6ffd700 1 -- 172.21.15.6:0/23614450 learned_addr learned my addr 172.21.15.6:0/23614450 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:42.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01b77fe700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 0x7f01b80abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01b67fc700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01b8159590 0x7f01b806ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01b6ffd700 1 -- 172.21.15.6:0/23614450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 msgr2=0x7f01b80abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01b6ffd700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 0x7f01b80abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01b6ffd700 1 -- 172.21.15.6:0/23614450 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01b8159590 msgr2=0x7f01b806ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01b6ffd700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01b8159590 0x7f01b806ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.765+0000 7f01b6ffd700 1 -- 172.21.15.6:0/23614450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b0009b30 con 0x7f01b80ce890 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.766+0000 7f01b6ffd700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 0x7f01b80ae090 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00c780 tx=0x7f01ac00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.766+0000 7f019f7fe700 1 -- 172.21.15.6:0/23614450 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac011800 con 0x7f01b80ce890 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.766+0000 7f019f7fe700 1 -- 172.21.15.6:0/23614450 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac011960 con 0x7f01b80ce890 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.766+0000 7f01bd7c4700 1 -- 172.21.15.6:0/23614450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b815d370 con 0x7f01b80ce890 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.766+0000 7f019f7fe700 1 -- 172.21.15.6:0/23614450 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac011cd0 con 0x7f01b80ce890 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.766+0000 7f01bd7c4700 1 -- 172.21.15.6:0/23614450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b80af240 con 0x7f01b80ce890 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.767+0000 7f01bd7c4700 1 -- 172.21.15.6:0/23614450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b80aec70 con 0x7f01b80ce890 2023-11-28T13:01:42.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.767+0000 7f019f7fe700 1 -- 172.21.15.6:0/23614450 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f01ac020070 con 0x7f01b80ce890 2023-11-28T13:01:42.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.767+0000 7f019f7fe700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01a40682f0 0x7f01a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:42.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.768+0000 7f019f7fe700 1 -- 172.21.15.6:0/23614450 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f01ac085ab0 con 0x7f01b80ce890 2023-11-28T13:01:42.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.768+0000 7f01b67fc700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01a40682f0 0x7f01a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:42.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.768+0000 7f01b67fc700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01a40682f0 0x7f01a406a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f01b80c6630 tx=0x7f01a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:42.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.772+0000 7f019f7fe700 1 -- 172.21.15.6:0/23614450 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f01ac052170 con 0x7f01b80ce890 2023-11-28T13:01:42.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.969+0000 7f01bd7c4700 1 -- 172.21.15.6:0/23614450 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01b80675a0 con 0x7f01b80ce890 2023-11-28T13:01:42.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.970+0000 7f019f7fe700 1 -- 172.21.15.6:0/23614450 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17987 (secure 0 0 0) 0x7f01ac055790 con 0x7f01b80ce890 2023-11-28T13:01:42.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:42.971 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":39,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:42.594889+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:41.144348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:41.554568+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:42.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 -- 172.21.15.6:0/23614450 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01a40682f0 msgr2=0x7f01a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01a40682f0 0x7f01a406a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f01b80c6630 tx=0x7f01a8008040 comp rx=0 tx=0).stop 2023-11-28T13:01:42.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 -- 172.21.15.6:0/23614450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 msgr2=0x7f01b80ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:42.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 0x7f01b80ae090 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00c780 tx=0x7f01ac00ca90 comp rx=0 tx=0).stop 2023-11-28T13:01:42.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 -- 172.21.15.6:0/23614450 shutdown_connections 2023-11-28T13:01:42.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01a40682f0 0x7f01a406a7a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f01b815db70 0x7f01b80abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01b80ce890 0x7f01b80ae090 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 --2- 172.21.15.6:0/23614450 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01b8159590 0x7f01b806ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:42.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 -- 172.21.15.6:0/23614450 >> 172.21.15.6:0/23614450 conn(0x7f01b80b1650 msgr2=0x7f01b80b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:42.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 -- 172.21.15.6:0/23614450 shutdown_connections 2023-11-28T13:01:42.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:42.972+0000 7f01bd7c4700 1 -- 172.21.15.6:0/23614450 wait complete. 2023-11-28T13:01:42.986 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:43.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.108+0000 7f158eaca700 1 Processor -- start 2023-11-28T13:01:43.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.108+0000 7f158eaca700 1 -- start start 2023-11-28T13:01:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.109+0000 7f158eaca700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f158815c990 0x7f15880cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.109+0000 7f158eaca700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1588161090 0x7f15880ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.109+0000 7f158eaca700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f15880caeb0 0x7f15880cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.109+0000 7f158eaca700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15880ce340 con 0x7f158815c990 2023-11-28T13:01:43.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.109+0000 7f158eaca700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15880cebe0 con 0x7f15880caeb0 2023-11-28T13:01:43.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.109+0000 7f158eaca700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15880cb3f0 con 0x7f1588161090 2023-11-28T13:01:43.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.109+0000 7f158c866700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f158815c990 0x7f15880cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.109+0000 7f158c866700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f158815c990 0x7f15880cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60398/0 (socket says 172.21.15.6:60398) 2023-11-28T13:01:43.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.109+0000 7f158c866700 1 -- 172.21.15.6:0/369012536 learned_addr learned my addr 172.21.15.6:0/369012536 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:43.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.109+0000 7f1587fff700 1 --2- 172.21.15.6:0/369012536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1588161090 0x7f15880ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158d067700 1 --2- 172.21.15.6:0/369012536 >> v2:172.21.15.61:3300/0 conn(0x7f15880caeb0 0x7f15880cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158c866700 1 -- 172.21.15.6:0/369012536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1588161090 msgr2=0x7f15880ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:43.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158c866700 1 --2- 172.21.15.6:0/369012536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1588161090 0x7f15880ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158c866700 1 -- 172.21.15.6:0/369012536 >> v2:172.21.15.61:3300/0 conn(0x7f15880caeb0 msgr2=0x7f15880cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:43.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158c866700 1 --2- 172.21.15.6:0/369012536 >> v2:172.21.15.61:3300/0 conn(0x7f15880caeb0 0x7f15880cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158c866700 1 -- 172.21.15.6:0/369012536 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880ce590 con 0x7f158815c990 2023-11-28T13:01:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158c866700 1 --2- 172.21.15.6:0/369012536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 0x7f15880cf070 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f15880ca080 tx=0x7f157c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c726261419d458 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f15877fe700 1 -- 172.21.15.6:0/369012536 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00b870 con 0x7f158815c990 2023-11-28T13:01:43.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f15877fe700 1 -- 172.21.15.6:0/369012536 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c00b9d0 con 0x7f158815c990 2023-11-28T13:01:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f15877fe700 1 -- 172.21.15.6:0/369012536 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00bd60 con 0x7f158815c990 2023-11-28T13:01:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158eaca700 1 -- 172.21.15.6:0/369012536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 msgr2=0x7f15880cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158eaca700 1 --2- 172.21.15.6:0/369012536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 0x7f15880cf070 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f15880ca080 tx=0x7f157c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158eaca700 1 -- 172.21.15.6:0/369012536 shutdown_connections 2023-11-28T13:01:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158eaca700 1 --2- 172.21.15.6:0/369012536 >> v2:172.21.15.61:3300/0 conn(0x7f15880caeb0 0x7f15880cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158eaca700 1 --2- 172.21.15.6:0/369012536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 0x7f15880cf070 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158eaca700 1 --2- 172.21.15.6:0/369012536 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1588161090 0x7f15880ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.110+0000 7f158eaca700 1 -- 172.21.15.6:0/369012536 >> 172.21.15.6:0/369012536 conn(0x7f15880b1650 msgr2=0x7f15880b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.111+0000 7f158eaca700 1 -- 172.21.15.6:0/369012536 shutdown_connections 2023-11-28T13:01:43.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.111+0000 7f158eaca700 1 -- 172.21.15.6:0/369012536 wait complete. 2023-11-28T13:01:43.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.112+0000 7f158eaca700 1 Processor -- start 2023-11-28T13:01:43.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.112+0000 7f158eaca700 1 -- start start 2023-11-28T13:01:43.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f158eaca700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15880caeb0 0x7f15880ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f158eaca700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 0x7f15880c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f158eaca700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1588161090 0x7f15880c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f158eaca700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880c43a0 con 0x7f158815c990 2023-11-28T13:01:43.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f158eaca700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158814b1d0 con 0x7f15880caeb0 2023-11-28T13:01:43.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f158eaca700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15880c9a30 con 0x7f1588161090 2023-11-28T13:01:43.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f1587fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 0x7f15880c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f1587fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 0x7f15880c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60406/0 (socket says 172.21.15.6:60406) 2023-11-28T13:01:43.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f1587fff700 1 -- 172.21.15.6:0/3537068035 learned_addr learned my addr 172.21.15.6:0/3537068035 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:43.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f158c866700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15880caeb0 0x7f15880ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f158d067700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1588161090 0x7f15880c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f1587fff700 1 -- 172.21.15.6:0/3537068035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1588161090 msgr2=0x7f15880c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f1587fff700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1588161090 0x7f15880c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f1587fff700 1 -- 172.21.15.6:0/3537068035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15880caeb0 msgr2=0x7f15880ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f1587fff700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15880caeb0 0x7f15880ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f1587fff700 1 -- 172.21.15.6:0/3537068035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c009550 con 0x7f158815c990 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f1587fff700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 0x7f15880c5120 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f157400d800 tx=0x7f157400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f15857fa700 1 -- 172.21.15.6:0/3537068035 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740126b0 con 0x7f158815c990 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f15857fa700 1 -- 172.21.15.6:0/3537068035 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1574012810 con 0x7f158815c990 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.113+0000 7f158eaca700 1 -- 172.21.15.6:0/3537068035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15880cf5b0 con 0x7f158815c990 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.114+0000 7f15857fa700 1 -- 172.21.15.6:0/3537068035 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574012ba0 con 0x7f158815c990 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.114+0000 7f158eaca700 1 -- 172.21.15.6:0/3537068035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15880ce870 con 0x7f158815c990 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.114+0000 7f158eaca700 1 -- 172.21.15.6:0/3537068035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15880cf740 con 0x7f158815c990 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.115+0000 7f15857fa700 1 -- 172.21.15.6:0/3537068035 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1574012d00 con 0x7f158815c990 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.115+0000 7f15857fa700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f156c0682f0 0x7f156c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.115+0000 7f15857fa700 1 -- 172.21.15.6:0/3537068035 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1574086100 con 0x7f158815c990 2023-11-28T13:01:43.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.115+0000 7f158c866700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f156c0682f0 0x7f156c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.116+0000 7f158c866700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f156c0682f0 0x7f156c06a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f157c000c00 tx=0x7f157c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.120+0000 7f15857fa700 1 -- 172.21.15.6:0/3537068035 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1574052730 con 0x7f158815c990 2023-11-28T13:01:43.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.315+0000 7f158eaca700 1 -- 172.21.15.6:0/3537068035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15880675a0 con 0x7f158815c990 2023-11-28T13:01:43.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.316+0000 7f15857fa700 1 -- 172.21.15.6:0/3537068035 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17987 (secure 0 0 0) 0x7f1574055d50 con 0x7f158815c990 2023-11-28T13:01:43.318 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:43.318 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":39,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:42.594889+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:41.144348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:41.554568+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:43.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.318+0000 7f158eaca700 1 -- 172.21.15.6:0/3537068035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f156c0682f0 msgr2=0x7f156c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.318+0000 7f158eaca700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f156c0682f0 0x7f156c06a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f157c000c00 tx=0x7f157c008030 comp rx=0 tx=0).stop 2023-11-28T13:01:43.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.318+0000 7f158eaca700 1 -- 172.21.15.6:0/3537068035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 msgr2=0x7f15880c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.318+0000 7f158eaca700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 0x7f15880c5120 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f157400d800 tx=0x7f157400db10 comp rx=0 tx=0).stop 2023-11-28T13:01:43.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.318+0000 7f158eaca700 1 -- 172.21.15.6:0/3537068035 shutdown_connections 2023-11-28T13:01:43.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.318+0000 7f158eaca700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f156c0682f0 0x7f156c06a7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.318+0000 7f158eaca700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1588161090 0x7f15880c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.318+0000 7f158eaca700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15880caeb0 0x7f15880ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.318+0000 7f158eaca700 1 --2- 172.21.15.6:0/3537068035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f158815c990 0x7f15880c5120 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.318+0000 7f158eaca700 1 -- 172.21.15.6:0/3537068035 >> 172.21.15.6:0/3537068035 conn(0x7f15880b1650 msgr2=0x7f15880b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:43.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.319+0000 7f158eaca700 1 -- 172.21.15.6:0/3537068035 shutdown_connections 2023-11-28T13:01:43.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.319+0000 7f158eaca700 1 -- 172.21.15.6:0/3537068035 wait complete. 2023-11-28T13:01:43.332 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:01:43.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.459+0000 7f421d8b5700 1 Processor -- start 2023-11-28T13:01:43.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.459+0000 7f421d8b5700 1 -- start start 2023-11-28T13:01:43.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f421d8b5700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f42180d2820 0x7f42180b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f421d8b5700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42180d1470 con 0x7f421806bf30 2023-11-28T13:01:43.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f421d8b5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42180cd4e0 con 0x7f421815c640 2023-11-28T13:01:43.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f421d8b5700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421815ca10 con 0x7f42180d2820 2023-11-28T13:01:43.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f42167fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f42180d2820 0x7f42180b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f42177fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f421806bf30 0x7f421806c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47736/0 (socket says 172.21.15.6:47736) 2023-11-28T13:01:43.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f42167fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f42180d2820 0x7f42180b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60414/0 (socket says 172.21.15.6:60414) 2023-11-28T13:01:43.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f42177fe700 1 -- 172.21.15.6:0/4276723520 learned_addr learned my addr 172.21.15.6:0/4276723520 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:43.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f42167fc700 1 -- 172.21.15.6:0/4276723520 >> v1:172.21.15.61:6789/0 conn(0x7f421815c640 legacy=0x7f42180cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f42167fc700 1 -- 172.21.15.6:0/4276723520 >> v1:172.21.15.6:6789/0 conn(0x7f421806bf30 legacy=0x7f421806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.460+0000 7f42167fc700 1 -- 172.21.15.6:0/4276723520 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42180cd7c0 con 0x7f42180d2820 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f42167fc700 1 --2- 172.21.15.6:0/4276723520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 0x7f42180b5c40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4208009840 tx=0x7f4208009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6973918f9ee63664 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f4215ffb700 1 -- 172.21.15.6:0/4276723520 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208011830 con 0x7f42180d2820 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f4215ffb700 1 -- 172.21.15.6:0/4276723520 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4208011990 con 0x7f42180d2820 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f4215ffb700 1 -- 172.21.15.6:0/4276723520 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208011d20 con 0x7f42180d2820 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f421d8b5700 1 -- 172.21.15.6:0/4276723520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 msgr2=0x7f42180b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f421d8b5700 1 --2- 172.21.15.6:0/4276723520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 0x7f42180b5c40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4208009840 tx=0x7f4208009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f421d8b5700 1 -- 172.21.15.6:0/4276723520 shutdown_connections 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f421d8b5700 1 --2- 172.21.15.6:0/4276723520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 0x7f42180b5c40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f421d8b5700 1 -- 172.21.15.6:0/4276723520 >> 172.21.15.6:0/4276723520 conn(0x7f42180d2df0 msgr2=0x7f42180d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f421d8b5700 1 -- 172.21.15.6:0/4276723520 shutdown_connections 2023-11-28T13:01:43.464 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.461+0000 7f421d8b5700 1 -- 172.21.15.6:0/4276723520 wait complete. 2023-11-28T13:01:43.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.463+0000 7f421d8b5700 1 Processor -- start 2023-11-28T13:01:43.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.463+0000 7f421d8b5700 1 -- start start 2023-11-28T13:01:43.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.463+0000 7f421d8b5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f421806bf30 0x7f42181492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.463+0000 7f421d8b5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 0x7f42180b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.463+0000 7f421d8b5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f421815c640 0x7f421815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.463+0000 7f421d8b5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4218149810 con 0x7f42180d2820 2023-11-28T13:01:43.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.463+0000 7f421d8b5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180c7940 con 0x7f421806bf30 2023-11-28T13:01:43.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.463+0000 7f421d8b5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42180b74e0 con 0x7f421815c640 2023-11-28T13:01:43.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f42167fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 0x7f42180b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f42167fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 0x7f42180b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60420/0 (socket says 172.21.15.6:60420) 2023-11-28T13:01:43.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f42167fc700 1 -- 172.21.15.6:0/432530631 learned_addr learned my addr 172.21.15.6:0/432530631 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:43.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f4216ffd700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f421806bf30 0x7f42181492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f42177fe700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f421815c640 0x7f421815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f42167fc700 1 -- 172.21.15.6:0/432530631 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f421815c640 msgr2=0x7f421815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f42167fc700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f421815c640 0x7f421815f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f42167fc700 1 -- 172.21.15.6:0/432530631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f421806bf30 msgr2=0x7f42181492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:43.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f42167fc700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f421806bf30 0x7f42181492d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:43.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f42167fc700 1 -- 172.21.15.6:0/432530631 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42080094a0 con 0x7f42180d2820 2023-11-28T13:01:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f42167fc700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 0x7f42180b6fa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f421808a670 tx=0x7f4208004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f420ffff700 1 -- 172.21.15.6:0/432530631 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42080036a0 con 0x7f42180d2820 2023-11-28T13:01:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f420ffff700 1 -- 172.21.15.6:0/432530631 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420801f070 con 0x7f42180d2820 2023-11-28T13:01:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f421d8b5700 1 -- 172.21.15.6:0/432530631 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42180cdaa0 con 0x7f42180d2820 2023-11-28T13:01:43.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f420ffff700 1 -- 172.21.15.6:0/432530631 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208003a60 con 0x7f42180d2820 2023-11-28T13:01:43.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.464+0000 7f421d8b5700 1 -- 172.21.15.6:0/432530631 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42180d1710 con 0x7f42180d2820 2023-11-28T13:01:43.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.465+0000 7f421d8b5700 1 -- 172.21.15.6:0/432530631 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42180cdc30 con 0x7f42180d2820 2023-11-28T13:01:43.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.466+0000 7f420ffff700 1 -- 172.21.15.6:0/432530631 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f420800f050 con 0x7f42180d2820 2023-11-28T13:01:43.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.466+0000 7f420ffff700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f41f80682f0 0x7f41f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:43.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.466+0000 7f420ffff700 1 -- 172.21.15.6:0/432530631 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4208021070 con 0x7f42180d2820 2023-11-28T13:01:43.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.466+0000 7f4216ffd700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f41f80682f0 0x7f41f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:43.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.467+0000 7f4216ffd700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f41f80682f0 0x7f41f806a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4200007fd0 tx=0x7f4200009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:43.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.470+0000 7f420ffff700 1 -- 172.21.15.6:0/432530631 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4208053ec0 con 0x7f42180d2820 2023-11-28T13:01:43.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:43.714+0000 7f421d8b5700 1 -- 172.21.15.6:0/432530631 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f42180739f0 con 0x7f42180d2820 2023-11-28T13:01:44.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.603+0000 7f420ffff700 1 -- 172.21.15.6:0/432530631 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f42080574e0 con 0x7f42180d2820 2023-11-28T13:01:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 -- 172.21.15.6:0/432530631 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f41f80682f0 msgr2=0x7f41f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f41f80682f0 0x7f41f806a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4200007fd0 tx=0x7f4200009040 comp rx=0 tx=0).stop 2023-11-28T13:01:44.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 -- 172.21.15.6:0/432530631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 msgr2=0x7f42180b6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 0x7f42180b6fa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f421808a670 tx=0x7f4208004750 comp rx=0 tx=0).stop 2023-11-28T13:01:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 -- 172.21.15.6:0/432530631 shutdown_connections 2023-11-28T13:01:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f41f80682f0 0x7f41f806a7a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f421815c640 0x7f421815f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f421806bf30 0x7f42181492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 --2- 172.21.15.6:0/432530631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42180d2820 0x7f42180b6fa0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 -- 172.21.15.6:0/432530631 >> 172.21.15.6:0/432530631 conn(0x7f42180d2df0 msgr2=0x7f421806a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 -- 172.21.15.6:0/432530631 shutdown_connections 2023-11-28T13:01:44.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.606+0000 7f421d8b5700 1 -- 172.21.15.6:0/432530631 wait complete. 2023-11-28T13:01:44.623 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:01:44.624 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:01:44.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.758+0000 7f65985ae700 1 Processor -- start 2023-11-28T13:01:44.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.758+0000 7f65985ae700 1 -- start start 2023-11-28T13:01:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f65985ae700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6590159500 0x7f65900cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f65985ae700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f659015dbf0 0x7f65900ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f65985ae700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659015e040 con 0x7f65900b1630 2023-11-28T13:01:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f65985ae700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65900cede0 con 0x7f659015dbf0 2023-11-28T13:01:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f65985ae700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65900ceff0 con 0x7f6590159500 2023-11-28T13:01:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f6595b49700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f659015dbf0 0x7f65900ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f6595b49700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f659015dbf0 0x7f65900ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60426/0 (socket says 172.21.15.6:60426) 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f6595b49700 1 -- 172.21.15.6:0/4142727815 learned_addr learned my addr 172.21.15.6:0/4142727815 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f659634a700 1 --2- 172.21.15.6:0/4142727815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6590159500 0x7f65900cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f6595b49700 1 -- 172.21.15.6:0/4142727815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6590159500 msgr2=0x7f65900cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f6595b49700 1 --2- 172.21.15.6:0/4142727815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6590159500 0x7f65900cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f6595b49700 1 -- 172.21.15.6:0/4142727815 >> v1:172.21.15.61:6789/0 conn(0x7f65900b1630 legacy=0x7f659006bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f6595b49700 1 -- 172.21.15.6:0/4142727815 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65900cf2d0 con 0x7f659015dbf0 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.760+0000 7f6595b49700 1 --2- 172.21.15.6:0/4142727815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f659015dbf0 0x7f65900ce8a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f658c0098f0 tx=0x7f658c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44608d08ac156ab1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f6595348700 1 -- 172.21.15.6:0/4142727815 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c00b8a0 con 0x7f659015dbf0 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f6595348700 1 -- 172.21.15.6:0/4142727815 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c00ba00 con 0x7f659015dbf0 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f6595348700 1 -- 172.21.15.6:0/4142727815 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c00bd90 con 0x7f659015dbf0 2023-11-28T13:01:44.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f65985ae700 1 -- 172.21.15.6:0/4142727815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f659015dbf0 msgr2=0x7f65900ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f65985ae700 1 --2- 172.21.15.6:0/4142727815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f659015dbf0 0x7f65900ce8a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f658c0098f0 tx=0x7f658c009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f65985ae700 1 -- 172.21.15.6:0/4142727815 shutdown_connections 2023-11-28T13:01:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f65985ae700 1 --2- 172.21.15.6:0/4142727815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6590159500 0x7f65900cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f65985ae700 1 --2- 172.21.15.6:0/4142727815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f659015dbf0 0x7f65900ce8a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f65985ae700 1 -- 172.21.15.6:0/4142727815 >> 172.21.15.6:0/4142727815 conn(0x7f65900cc630 msgr2=0x7f65900cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f65985ae700 1 -- 172.21.15.6:0/4142727815 shutdown_connections 2023-11-28T13:01:44.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.761+0000 7f65985ae700 1 -- 172.21.15.6:0/4142727815 wait complete. 2023-11-28T13:01:44.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.762+0000 7f65985ae700 1 Processor -- start 2023-11-28T13:01:44.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.762+0000 7f65985ae700 1 -- start start 2023-11-28T13:01:44.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f65985ae700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65900b1630 0x7f659014a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f65985ae700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590159500 0x7f6590148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f65985ae700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f659015dbf0 0x7f659015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f65985ae700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659014a570 con 0x7f6590159500 2023-11-28T13:01:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f65985ae700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6590148d40 con 0x7f65900b1630 2023-11-28T13:01:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f65985ae700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6590148f90 con 0x7f659015dbf0 2023-11-28T13:01:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f6595b49700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590159500 0x7f6590148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f6595b49700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590159500 0x7f6590148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60434/0 (socket says 172.21.15.6:60434) 2023-11-28T13:01:44.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f6595b49700 1 -- 172.21.15.6:0/2999991111 learned_addr learned my addr 172.21.15.6:0/2999991111 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:44.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f659634a700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65900b1630 0x7f659014a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f6596b4b700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f659015dbf0 0x7f659015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f6595b49700 1 -- 172.21.15.6:0/2999991111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f659015dbf0 msgr2=0x7f659015b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f6595b49700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f659015dbf0 0x7f659015b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.763+0000 7f6595b49700 1 -- 172.21.15.6:0/2999991111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65900b1630 msgr2=0x7f659014a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:44.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.764+0000 7f6595b49700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65900b1630 0x7f659014a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:44.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.764+0000 7f6595b49700 1 -- 172.21.15.6:0/2999991111 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c009550 con 0x7f6590159500 2023-11-28T13:01:44.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.764+0000 7f659634a700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65900b1630 0x7f659014a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:44.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.764+0000 7f6596b4b700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f659015dbf0 0x7f659015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:44.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.764+0000 7f6595b49700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590159500 0x7f6590148590 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f658c000c00 tx=0x7f658c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:44.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.764+0000 7f6586ffd700 1 -- 172.21.15.6:0/2999991111 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c004480 con 0x7f6590159500 2023-11-28T13:01:44.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.764+0000 7f6586ffd700 1 -- 172.21.15.6:0/2999991111 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c01f070 con 0x7f6590159500 2023-11-28T13:01:44.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.764+0000 7f65985ae700 1 -- 172.21.15.6:0/2999991111 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659015d3f0 con 0x7f6590159500 2023-11-28T13:01:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.764+0000 7f6586ffd700 1 -- 172.21.15.6:0/2999991111 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c0038c0 con 0x7f6590159500 2023-11-28T13:01:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.764+0000 7f65985ae700 1 -- 172.21.15.6:0/2999991111 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659015d110 con 0x7f6590159500 2023-11-28T13:01:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.765+0000 7f65985ae700 1 -- 172.21.15.6:0/2999991111 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659015d6d0 con 0x7f6590159500 2023-11-28T13:01:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.765+0000 7f6586ffd700 1 -- 172.21.15.6:0/2999991111 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f658c02a020 con 0x7f6590159500 2023-11-28T13:01:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.766+0000 7f6586ffd700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65740682f0 0x7f657406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.766+0000 7f6586ffd700 1 -- 172.21.15.6:0/2999991111 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f658c087380 con 0x7f6590159500 2023-11-28T13:01:44.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.766+0000 7f659634a700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65740682f0 0x7f657406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:44.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.766+0000 7f659634a700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65740682f0 0x7f657406a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6580009a00 tx=0x7f6580008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:44.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:44.770+0000 7f6586ffd700 1 -- 172.21.15.6:0/2999991111 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f658c053aa0 con 0x7f6590159500 2023-11-28T13:01:45.015 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.014+0000 7f65985ae700 1 -- 172.21.15.6:0/2999991111 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f65900739f0 con 0x7f6590159500 2023-11-28T13:01:45.611 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.610+0000 7f6586ffd700 1 -- 172.21.15.6:0/2999991111 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f658c0570c0 con 0x7f6590159500 2023-11-28T13:01:45.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 -- 172.21.15.6:0/2999991111 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65740682f0 msgr2=0x7f657406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65740682f0 0x7f657406a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6580009a00 tx=0x7f6580008040 comp rx=0 tx=0).stop 2023-11-28T13:01:45.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 -- 172.21.15.6:0/2999991111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590159500 msgr2=0x7f6590148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590159500 0x7f6590148590 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f658c000c00 tx=0x7f658c004750 comp rx=0 tx=0).stop 2023-11-28T13:01:45.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 -- 172.21.15.6:0/2999991111 shutdown_connections 2023-11-28T13:01:45.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65740682f0 0x7f657406a7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f659015dbf0 0x7f659015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65900b1630 0x7f659014a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 --2- 172.21.15.6:0/2999991111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6590159500 0x7f6590148590 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 -- 172.21.15.6:0/2999991111 >> 172.21.15.6:0/2999991111 conn(0x7f65900cc630 msgr2=0x7f65900b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:45.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 -- 172.21.15.6:0/2999991111 shutdown_connections 2023-11-28T13:01:45.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.613+0000 7f65985ae700 1 -- 172.21.15.6:0/2999991111 wait complete. 2023-11-28T13:01:45.629 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:01:45.629 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:45.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.759+0000 7fca4c408700 1 Processor -- start 2023-11-28T13:01:45.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.759+0000 7fca4c408700 1 -- start start 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.760+0000 7fca4c408700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fca440b2660 0x7fca440b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.760+0000 7fca4c408700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca440cfb90 con 0x7fca440cb230 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.760+0000 7fca4c408700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca440cb680 con 0x7fca4415a6c0 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.760+0000 7fca4c408700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca440b3020 con 0x7fca440b2660 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.760+0000 7fca4a9a5700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fca440b2660 0x7fca440b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.760+0000 7fca499a3700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fca440cb230 0x7fca440ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47762/0 (socket says 172.21.15.6:47762) 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.760+0000 7fca499a3700 1 -- 172.21.15.6:0/1278334178 learned_addr learned my addr 172.21.15.6:0/1278334178 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.760+0000 7fca4a9a5700 1 -- 172.21.15.6:0/1278334178 >> v1:172.21.15.61:6789/0 conn(0x7fca4415a6c0 legacy=0x7fca440d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.760+0000 7fca4a9a5700 1 -- 172.21.15.6:0/1278334178 >> v1:172.21.15.6:6789/0 conn(0x7fca440cb230 legacy=0x7fca440ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca4a9a5700 1 -- 172.21.15.6:0/1278334178 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca440cb930 con 0x7fca440b2660 2023-11-28T13:01:45.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca491a2700 1 -- 172.21.15.6:0/1278334178 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656965401 0 0) 0x7fca440cfb90 con 0x7fca440cb230 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca4a9a5700 1 --2- 172.21.15.6:0/1278334178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 0x7fca440b2ae0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009d90 tx=0x7fca3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc2f3ac110e3532 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca491a2700 1 -- 172.21.15.6:0/1278334178 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00e070 con 0x7fca440b2660 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca491a2700 1 -- 172.21.15.6:0/1278334178 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c00bb90 con 0x7fca440b2660 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca491a2700 1 -- 172.21.15.6:0/1278334178 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c003a20 con 0x7fca440b2660 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca4c408700 1 -- 172.21.15.6:0/1278334178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 msgr2=0x7fca440b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca4c408700 1 --2- 172.21.15.6:0/1278334178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 0x7fca440b2ae0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009d90 tx=0x7fca3c00b330 comp rx=0 tx=0).stop 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca4c408700 1 -- 172.21.15.6:0/1278334178 shutdown_connections 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca4c408700 1 --2- 172.21.15.6:0/1278334178 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 0x7fca440b2ae0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.761+0000 7fca4c408700 1 -- 172.21.15.6:0/1278334178 >> 172.21.15.6:0/1278334178 conn(0x7fca440cd660 msgr2=0x7fca440cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.762+0000 7fca4c408700 1 -- 172.21.15.6:0/1278334178 shutdown_connections 2023-11-28T13:01:45.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.762+0000 7fca4c408700 1 -- 172.21.15.6:0/1278334178 wait complete. 2023-11-28T13:01:45.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.763+0000 7fca4c408700 1 Processor -- start 2023-11-28T13:01:45.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.763+0000 7fca4c408700 1 -- start start 2023-11-28T13:01:45.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.763+0000 7fca4c408700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 0x7fca44151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:45.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.763+0000 7fca4c408700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca440cb230 0x7fca440ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:45.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.763+0000 7fca4c408700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca4415a6c0 0x7fca4415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:45.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.763+0000 7fca4c408700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440c7220 con 0x7fca440b2660 2023-11-28T13:01:45.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.763+0000 7fca4c408700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4406ccc0 con 0x7fca440cb230 2023-11-28T13:01:45.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.763+0000 7fca4c408700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4406bc20 con 0x7fca4415a6c0 2023-11-28T13:01:45.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4a1a4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 0x7fca44151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:45.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4a1a4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 0x7fca44151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60446/0 (socket says 172.21.15.6:60446) 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4a1a4700 1 -- 172.21.15.6:0/1899577578 learned_addr learned my addr 172.21.15.6:0/1899577578 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4a1a4700 1 -- 172.21.15.6:0/1899577578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca4415a6c0 msgr2=0x7fca4415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4a1a4700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca4415a6c0 0x7fca4415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca499a3700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca440cb230 0x7fca440ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4a1a4700 1 -- 172.21.15.6:0/1899577578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca440cb230 msgr2=0x7fca440ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4a1a4700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca440cb230 0x7fca440ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4a1a4700 1 -- 172.21.15.6:0/1899577578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c009a70 con 0x7fca440b2660 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca499a3700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca440cb230 0x7fca440ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4a1a4700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 0x7fca44151290 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c4d0 tx=0x7fca3400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca3affd700 1 -- 172.21.15.6:0/1899577578 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34011790 con 0x7fca440b2660 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca3affd700 1 -- 172.21.15.6:0/1899577578 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca340118f0 con 0x7fca440b2660 2023-11-28T13:01:45.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4c408700 1 -- 172.21.15.6:0/1899577578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca440cf540 con 0x7fca440b2660 2023-11-28T13:01:45.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca3affd700 1 -- 172.21.15.6:0/1899577578 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34011ca0 con 0x7fca440b2660 2023-11-28T13:01:45.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.764+0000 7fca4c408700 1 -- 172.21.15.6:0/1899577578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca440cf260 con 0x7fca440b2660 2023-11-28T13:01:45.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.765+0000 7fca4c408700 1 -- 172.21.15.6:0/1899577578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca440cf820 con 0x7fca440b2660 2023-11-28T13:01:45.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.766+0000 7fca3affd700 1 -- 172.21.15.6:0/1899577578 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fca34010050 con 0x7fca440b2660 2023-11-28T13:01:45.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.766+0000 7fca3affd700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca280682f0 0x7fca2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:45.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.766+0000 7fca3affd700 1 -- 172.21.15.6:0/1899577578 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fca34085ba0 con 0x7fca440b2660 2023-11-28T13:01:45.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.766+0000 7fca499a3700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca280682f0 0x7fca2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:45.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.767+0000 7fca499a3700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca280682f0 0x7fca2806a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca40007fd0 tx=0x7fca40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:45.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:45.770+0000 7fca3affd700 1 -- 172.21.15.6:0/1899577578 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fca340521d0 con 0x7fca440b2660 2023-11-28T13:01:46.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.009+0000 7fca4c408700 1 -- 172.21.15.6:0/1899577578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca440675a0 con 0x7fca440b2660 2023-11-28T13:01:46.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.009+0000 7fca3affd700 1 -- 172.21.15.6:0/1899577578 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4642 (secure 0 0 0) 0x7fca340557f0 con 0x7fca440b2660 2023-11-28T13:01:46.011 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:46.011 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4026920958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4026920958},{"type":"v1","addr":"172.21.15.61:6833","nonce":4026920958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2212733196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2212733196},{"type":"v1","addr":"172.21.15.61:6835","nonce":2212733196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1674709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1674709721},{"type":"v1","addr":"172.21.15.6:6836","nonce":1674709721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:42.595217+0000","modified":"2023-11-28T13:01:45.602537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/2789212407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":2789212407},{"type":"v1","addr":"172.21.15.6:6837","nonce":2789212407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:01:46.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 -- 172.21.15.6:0/1899577578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca280682f0 msgr2=0x7fca2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca280682f0 0x7fca2806a7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca40007fd0 tx=0x7fca40009040 comp rx=0 tx=0).stop 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 -- 172.21.15.6:0/1899577578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 msgr2=0x7fca44151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 0x7fca44151290 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c4d0 tx=0x7fca3400c890 comp rx=0 tx=0).stop 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 -- 172.21.15.6:0/1899577578 shutdown_connections 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fca280682f0 0x7fca2806a7a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fca4415a6c0 0x7fca4415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca440b2660 0x7fca44151290 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 --2- 172.21.15.6:0/1899577578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca440cb230 0x7fca440ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 -- 172.21.15.6:0/1899577578 >> 172.21.15.6:0/1899577578 conn(0x7fca440cd660 msgr2=0x7fca440b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 -- 172.21.15.6:0/1899577578 shutdown_connections 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.012+0000 7fca4c408700 1 -- 172.21.15.6:0/1899577578 wait complete. 2023-11-28T13:01:46.014 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2023-11-28T13:01:46.025 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:46.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.148+0000 7fbc7813a700 1 Processor -- start 2023-11-28T13:01:46.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.148+0000 7fbc7813a700 1 -- start start 2023-11-28T13:01:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.149+0000 7fbc7813a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbc7015a6c0 0x7fbc700cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.149+0000 7fbc7813a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700b2660 0x7fbc700b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.149+0000 7fbc7813a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc700cfb90 con 0x7fbc700d0020 2023-11-28T13:01:46.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.149+0000 7fbc7813a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc700b3020 con 0x7fbc7015a6c0 2023-11-28T13:01:46.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.149+0000 7fbc7813a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700cb930 con 0x7fbc700b2660 2023-11-28T13:01:46.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.149+0000 7fbc75ed6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbc7015a6c0 0x7fbc700cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.149+0000 7fbc766d7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700b2660 0x7fbc700b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc766d7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700b2660 0x7fbc700b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:50162/0 (socket says 172.21.15.6:50162) 2023-11-28T13:01:46.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc75ed6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbc7015a6c0 0x7fbc700cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:35940/0 (socket says 172.21.15.6:35940) 2023-11-28T13:01:46.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc756d5700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fbc700d0020 0x7fbc700ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:37804/0 (socket says 172.21.15.6:37804) 2023-11-28T13:01:46.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc766d7700 1 -- 172.21.15.6:0/3945037700 learned_addr learned my addr 172.21.15.6:0/3945037700 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:46.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc75ed6700 1 -- 172.21.15.6:0/3945037700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700b2660 msgr2=0x7fbc700b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:46.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc75ed6700 1 --2- 172.21.15.6:0/3945037700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700b2660 0x7fbc700b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc75ed6700 1 -- 172.21.15.6:0/3945037700 >> v1:172.21.15.61:6789/0 conn(0x7fbc700d0020 legacy=0x7fbc700ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc74ed4700 1 -- 172.21.15.6:0/3945037700 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714513175 0 0) 0x7fbc700cfb90 con 0x7fbc700d0020 2023-11-28T13:01:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc75ed6700 1 -- 172.21.15.6:0/3945037700 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc700cf260 con 0x7fbc7015a6c0 2023-11-28T13:01:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc766d7700 1 --2- 172.21.15.6:0/3945037700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700b2660 0x7fbc700b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc75ed6700 1 --2- 172.21.15.6:0/3945037700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc7015a6c0 0x7fbc700cb230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009840 tx=0x7fbc60009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd2fdc452d900bd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc74ed4700 1 -- 172.21.15.6:0/3945037700 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60011830 con 0x7fbc7015a6c0 2023-11-28T13:01:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc74ed4700 1 -- 172.21.15.6:0/3945037700 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc60011990 con 0x7fbc7015a6c0 2023-11-28T13:01:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.150+0000 7fbc74ed4700 1 -- 172.21.15.6:0/3945037700 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60011d20 con 0x7fbc7015a6c0 2023-11-28T13:01:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.151+0000 7fbc7813a700 1 -- 172.21.15.6:0/3945037700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc7015a6c0 msgr2=0x7fbc700cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.151+0000 7fbc7813a700 1 --2- 172.21.15.6:0/3945037700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc7015a6c0 0x7fbc700cb230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009840 tx=0x7fbc60009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.151+0000 7fbc7813a700 1 -- 172.21.15.6:0/3945037700 shutdown_connections 2023-11-28T13:01:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.151+0000 7fbc7813a700 1 --2- 172.21.15.6:0/3945037700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700b2660 0x7fbc700b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.151+0000 7fbc7813a700 1 --2- 172.21.15.6:0/3945037700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc7015a6c0 0x7fbc700cb230 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.151+0000 7fbc7813a700 1 -- 172.21.15.6:0/3945037700 >> 172.21.15.6:0/3945037700 conn(0x7fbc700cd660 msgr2=0x7fbc700cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.151+0000 7fbc7813a700 1 -- 172.21.15.6:0/3945037700 shutdown_connections 2023-11-28T13:01:46.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.151+0000 7fbc7813a700 1 -- 172.21.15.6:0/3945037700 wait complete. 2023-11-28T13:01:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.152+0000 7fbc7813a700 1 Processor -- start 2023-11-28T13:01:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.152+0000 7fbc7813a700 1 -- start start 2023-11-28T13:01:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc7813a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc700b2660 0x7fbc701f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc7813a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700d0020 0x7fbc7015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc7813a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc7015a6c0 0x7fbc7015c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc7813a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7006ca70 con 0x7fbc700b2660 2023-11-28T13:01:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc7813a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7006cd50 con 0x7fbc7015a6c0 2023-11-28T13:01:46.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc7813a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7006d030 con 0x7fbc700d0020 2023-11-28T13:01:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc75ed6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc700b2660 0x7fbc701f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc75ed6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc700b2660 0x7fbc701f15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60462/0 (socket says 172.21.15.6:60462) 2023-11-28T13:01:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc75ed6700 1 -- 172.21.15.6:0/623559856 learned_addr learned my addr 172.21.15.6:0/623559856 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc766d7700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc7015a6c0 0x7fbc7015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc756d5700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700d0020 0x7fbc7015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc75ed6700 1 -- 172.21.15.6:0/623559856 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700d0020 msgr2=0x7fbc7015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc75ed6700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700d0020 0x7fbc7015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc75ed6700 1 -- 172.21.15.6:0/623559856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc7015a6c0 msgr2=0x7fbc7015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc75ed6700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc7015a6c0 0x7fbc7015c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc75ed6700 1 -- 172.21.15.6:0/623559856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc68009b30 con 0x7fbc700b2660 2023-11-28T13:01:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.153+0000 7fbc766d7700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc7015a6c0 0x7fbc7015c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.154+0000 7fbc75ed6700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc700b2660 0x7fbc701f15e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009fd0 tx=0x7fbc60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.154+0000 7fbc667fc700 1 -- 172.21.15.6:0/623559856 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc600036a0 con 0x7fbc700b2660 2023-11-28T13:01:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.154+0000 7fbc667fc700 1 -- 172.21.15.6:0/623559856 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6001f070 con 0x7fbc700b2660 2023-11-28T13:01:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.154+0000 7fbc667fc700 1 -- 172.21.15.6:0/623559856 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60003a60 con 0x7fbc700b2660 2023-11-28T13:01:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.154+0000 7fbc7813a700 1 -- 172.21.15.6:0/623559856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc600094a0 con 0x7fbc700b2660 2023-11-28T13:01:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.154+0000 7fbc7813a700 1 -- 172.21.15.6:0/623559856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc700cf820 con 0x7fbc700b2660 2023-11-28T13:01:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.155+0000 7fbc7813a700 1 -- 172.21.15.6:0/623559856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc700cf540 con 0x7fbc700b2660 2023-11-28T13:01:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.155+0000 7fbc667fc700 1 -- 172.21.15.6:0/623559856 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbc6000f050 con 0x7fbc700b2660 2023-11-28T13:01:46.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.155+0000 7fbc667fc700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc540682f0 0x7fbc5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.156+0000 7fbc667fc700 1 -- 172.21.15.6:0/623559856 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbc60021070 con 0x7fbc700b2660 2023-11-28T13:01:46.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.156+0000 7fbc756d5700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc540682f0 0x7fbc5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.156+0000 7fbc756d5700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc540682f0 0x7fbc5406a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbc700b1b70 tx=0x7fbc6c007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.161 INFO:tasks.ceph.mds.c.smithi006.stderr:2023-11-28T13:01:46.159+0000 7f6c26fd7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:01:46.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.160+0000 7fbc667fc700 1 -- 172.21.15.6:0/623559856 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbc60053e20 con 0x7fbc700b2660 2023-11-28T13:01:46.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.354+0000 7fbc7813a700 1 -- 172.21.15.6:0/623559856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc700d0470 con 0x7fbc700b2660 2023-11-28T13:01:46.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.355+0000 7fbc667fc700 1 -- 172.21.15.6:0/623559856 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17974 (secure 0 0 0) 0x7fbc60057440 con 0x7fbc700b2660 2023-11-28T13:01:46.356 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:46.357 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":40,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:43.596467+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:41.144348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:41.554568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:46.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 -- 172.21.15.6:0/623559856 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc540682f0 msgr2=0x7fbc5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc540682f0 0x7fbc5406a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbc700b1b70 tx=0x7fbc6c007be0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 -- 172.21.15.6:0/623559856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc700b2660 msgr2=0x7fbc701f15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc700b2660 0x7fbc701f15e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009fd0 tx=0x7fbc60004750 comp rx=0 tx=0).stop 2023-11-28T13:01:46.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 -- 172.21.15.6:0/623559856 shutdown_connections 2023-11-28T13:01:46.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc7015a6c0 0x7fbc7015c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc700b2660 0x7fbc701f15e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbc540682f0 0x7fbc5406a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 --2- 172.21.15.6:0/623559856 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbc700d0020 0x7fbc7015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 -- 172.21.15.6:0/623559856 >> 172.21.15.6:0/623559856 conn(0x7fbc700cd660 msgr2=0x7fbc701596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:46.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 -- 172.21.15.6:0/623559856 shutdown_connections 2023-11-28T13:01:46.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.357+0000 7fbc7813a700 1 -- 172.21.15.6:0/623559856 wait complete. 2023-11-28T13:01:46.371 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:01:46.371 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:01:46.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.495+0000 7f1a28312700 1 Processor -- start 2023-11-28T13:01:46.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.495+0000 7f1a28312700 1 -- start start 2023-11-28T13:01:46.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a28312700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2015aa20 0x7f1a200b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a28312700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1a200b5500 0x7f1a200cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a28312700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a200ca990 con 0x7f1a2015a650 2023-11-28T13:01:46.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a28312700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a200caf50 con 0x7f1a200b5500 2023-11-28T13:01:46.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a28312700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200aef10 con 0x7f1a2015aa20 2023-11-28T13:01:46.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a260ae700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1a2015a650 0x7f1a200ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47788/0 (socket says 172.21.15.6:47788) 2023-11-28T13:01:46.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a260ae700 1 -- 172.21.15.6:0/2825087159 learned_addr learned my addr 172.21.15.6:0/2825087159 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:46.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a268af700 1 --2- 172.21.15.6:0/2825087159 >> v2:172.21.15.61:3300/0 conn(0x7f1a200b5500 0x7f1a200cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041074005 0 0) 0x7f1a200ca990 con 0x7f1a2015a650 2023-11-28T13:01:46.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a258ad700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2015aa20 0x7f1a200b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a0c0035b0 con 0x7f1a2015a650 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1181622890 0 0) 0x7f1a0c0035b0 con 0x7f1a2015a650 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2015aa20 msgr2=0x7f1a200b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2015aa20 0x7f1a200b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 >> v2:172.21.15.61:3300/0 conn(0x7f1a200b5500 msgr2=0x7f1a200cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 --2- 172.21.15.6:0/2825087159 >> v2:172.21.15.61:3300/0 conn(0x7f1a200b5500 0x7f1a200cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a200af1f0 con 0x7f1a2015a650 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f1a1c001670 con 0x7f1a2015a650 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 >> v1:172.21.15.6:6789/0 conn(0x7f1a2015a650 legacy=0x7f1a200ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2015aa20 0x7f1a0c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a0c003f80 0x7f1a0c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0c004d90 0x7f1a0c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200caf50 con 0x7f1a0c003f80 2023-11-28T13:01:46.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200aef10 con 0x7f1a0c004d90 2023-11-28T13:01:46.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.496+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c00ba10 con 0x7f1a2015aa20 2023-11-28T13:01:46.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a258ad700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a0c003f80 0x7f1a0c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a260ae700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2015aa20 0x7f1a0c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a268af700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0c004d90 0x7f1a0c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a258ad700 1 -- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2015aa20 msgr2=0x7f1a0c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:46.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a258ad700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a2015aa20 0x7f1a0c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a258ad700 1 -- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0c004d90 msgr2=0x7f1a0c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:46.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a258ad700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0c004d90 0x7f1a0c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a258ad700 1 -- 172.21.15.6:0/2825087159 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c0035b0 con 0x7f1a0c003f80 2023-11-28T13:01:46.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a268af700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0c004d90 0x7f1a0c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:46.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a258ad700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a0c003f80 0x7f1a0c004850 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009b60 tx=0x7f1a0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51faa170f34bd38 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000e6c0 con 0x7f1a0c003f80 2023-11-28T13:01:46.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1000e820 con 0x7f1a0c003f80 2023-11-28T13:01:46.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.497+0000 7f1a250ac700 1 -- 172.21.15.6:0/2825087159 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000eb90 con 0x7f1a0c003f80 2023-11-28T13:01:46.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.498+0000 7f1a28312700 1 -- 172.21.15.6:0/2825087159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a0c003f80 msgr2=0x7f1a0c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.498+0000 7f1a28312700 1 --2- 172.21.15.6:0/2825087159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a0c003f80 0x7f1a0c004850 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009b60 tx=0x7f1a0c005880 comp rx=0 tx=0).stop 2023-11-28T13:01:46.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.498+0000 7f1a268af700 1 -- 172.21.15.6:0/2825087159 reap_dead start 2023-11-28T13:01:46.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.498+0000 7f1a28312700 1 -- 172.21.15.6:0/2825087159 shutdown_connections 2023-11-28T13:01:46.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.498+0000 7f1a28312700 1 -- 172.21.15.6:0/2825087159 >> 172.21.15.6:0/2825087159 conn(0x7f1a200b1650 msgr2=0x7f1a200b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:46.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.498+0000 7f1a28312700 1 -- 172.21.15.6:0/2825087159 shutdown_connections 2023-11-28T13:01:46.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.498+0000 7f1a28312700 1 -- 172.21.15.6:0/2825087159 wait complete. 2023-11-28T13:01:46.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.499+0000 7f1a28312700 1 Processor -- start 2023-11-28T13:01:46.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.499+0000 7f1a28312700 1 -- start start 2023-11-28T13:01:46.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a28312700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0c003f80 0x7f1a2014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a28312700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a200c6320 0x7f1a200b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a28312700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a200b4010 0x7f1a2015c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a28312700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200b0950 con 0x7f1a200c6320 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a28312700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200afd40 con 0x7f1a200b4010 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a28312700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200b0020 con 0x7f1a0c003f80 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a258ad700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a200c6320 0x7f1a200b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a258ad700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a200c6320 0x7f1a200b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60482/0 (socket says 172.21.15.6:60482) 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a258ad700 1 -- 172.21.15.6:0/1135569746 learned_addr learned my addr 172.21.15.6:0/1135569746 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a260ae700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0c003f80 0x7f1a2014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a268af700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a200b4010 0x7f1a2015c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a258ad700 1 -- 172.21.15.6:0/1135569746 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0c003f80 msgr2=0x7f1a2014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a258ad700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0c003f80 0x7f1a2014cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a258ad700 1 -- 172.21.15.6:0/1135569746 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a200b4010 msgr2=0x7f1a2015c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a258ad700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a200b4010 0x7f1a2015c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a258ad700 1 -- 172.21.15.6:0/1135569746 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a200b2950 con 0x7f1a200c6320 2023-11-28T13:01:46.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.500+0000 7f1a258ad700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a200c6320 0x7f1a200b0de0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1a100090b0 tx=0x7f1a100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.501+0000 7f1a16ffd700 1 -- 172.21.15.6:0/1135569746 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a10021070 con 0x7f1a200c6320 2023-11-28T13:01:46.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.501+0000 7f1a16ffd700 1 -- 172.21.15.6:0/1135569746 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a10004430 con 0x7f1a200c6320 2023-11-28T13:01:46.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.501+0000 7f1a28312700 1 -- 172.21.15.6:0/1135569746 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a200cabe0 con 0x7f1a200c6320 2023-11-28T13:01:46.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.501+0000 7f1a16ffd700 1 -- 172.21.15.6:0/1135569746 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000e6c0 con 0x7f1a200c6320 2023-11-28T13:01:46.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.501+0000 7f1a28312700 1 -- 172.21.15.6:0/1135569746 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a200b2ae0 con 0x7f1a200c6320 2023-11-28T13:01:46.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.501+0000 7f1a28312700 1 -- 172.21.15.6:0/1135569746 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a200675a0 con 0x7f1a200c6320 2023-11-28T13:01:46.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.502+0000 7f1a16ffd700 1 -- 172.21.15.6:0/1135569746 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1a1002e030 con 0x7f1a200c6320 2023-11-28T13:01:46.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.502+0000 7f1a16ffd700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a040682f0 0x7f1a0406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.502+0000 7f1a16ffd700 1 -- 172.21.15.6:0/1135569746 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1a10087070 con 0x7f1a200c6320 2023-11-28T13:01:46.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.502+0000 7f1a260ae700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a040682f0 0x7f1a0406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.503+0000 7f1a260ae700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a040682f0 0x7f1a0406a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c000c00 tx=0x7f1a1c007980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.507+0000 7f1a16ffd700 1 -- 172.21.15.6:0/1135569746 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1a10053710 con 0x7f1a200c6320 2023-11-28T13:01:46.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.698+0000 7f1a28312700 1 -- 172.21.15.6:0/1135569746 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a200cb960 con 0x7f1a040682f0 2023-11-28T13:01:46.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.707+0000 7f1a16ffd700 1 -- 172.21.15.6:0/1135569746 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312009 (secure 0 0 0) 0x7f1a200cb960 con 0x7f1a040682f0 2023-11-28T13:01:46.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:46.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.710+0000 7f1a28312700 1 -- 172.21.15.6:0/1135569746 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a040682f0 msgr2=0x7f1a0406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.710+0000 7f1a28312700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a040682f0 0x7f1a0406a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c000c00 tx=0x7f1a1c007980 comp rx=0 tx=0).stop 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.710+0000 7f1a28312700 1 -- 172.21.15.6:0/1135569746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a200c6320 msgr2=0x7f1a200b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.710+0000 7f1a28312700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a200c6320 0x7f1a200b0de0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1a100090b0 tx=0x7f1a100047c0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.711+0000 7f1a28312700 1 -- 172.21.15.6:0/1135569746 shutdown_connections 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.711+0000 7f1a28312700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a200b4010 0x7f1a2015c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.711+0000 7f1a28312700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1a040682f0 0x7f1a0406a7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.711+0000 7f1a28312700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0c003f80 0x7f1a2014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.711+0000 7f1a28312700 1 --2- 172.21.15.6:0/1135569746 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a200c6320 0x7f1a200b0de0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.711+0000 7f1a28312700 1 -- 172.21.15.6:0/1135569746 >> 172.21.15.6:0/1135569746 conn(0x7f1a200b1650 msgr2=0x7f1a20158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.711+0000 7f1a28312700 1 -- 172.21.15.6:0/1135569746 shutdown_connections 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.711+0000 7f1a28312700 1 -- 172.21.15.6:0/1135569746 wait complete. 2023-11-28T13:01:46.716 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:01:46.724 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-11-28T13:01:44.959977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71332,"kb_used_data":4772,"kb_used_omap":0,"kb_used_meta":66496,"kb_avail":754903388,"statfs":{"total":773094113280,"available":773021069312,"internally_reserved":0,"allocated":4886528,"data_stored":2817636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68091904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000633"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606172+0000","last_change":"2023-11-28T13:01:42.620897+0000","last_active":"2023-11-28T13:01:43.606172+0000","last_peered":"2023-11-28T13:01:43.606172+0000","last_clean":"2023-11-28T13:01:43.606172+0000","last_became_active":"2023-11-28T13:01:42.620606+0000","last_became_peered":"2023-11-28T13:01:42.620606+0000","last_unstale":"2023-11-28T13:01:43.606172+0000","last_undegraded":"2023-11-28T13:01:43.606172+0000","last_fullsized":"2023-11-28T13:01:43.606172+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:08.291193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.632405+0000","last_change":"2023-11-28T13:01:42.627974+0000","last_active":"2023-11-28T13:01:42.632405+0000","last_peered":"2023-11-28T13:01:42.632405+0000","last_clean":"2023-11-28T13:01:42.632405+0000","last_became_active":"2023-11-28T13:01:42.627568+0000","last_became_peered":"2023-11-28T13:01:42.627568+0000","last_unstale":"2023-11-28T13:01:42.632405+0000","last_undegraded":"2023-11-28T13:01:42.632405+0000","last_fullsized":"2023-11-28T13:01:42.632405+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:59:55.382836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733463+0000","last_change":"2023-11-28T13:01:42.622185+0000","last_active":"2023-11-28T13:01:43.733463+0000","last_peered":"2023-11-28T13:01:43.733463+0000","last_clean":"2023-11-28T13:01:43.733463+0000","last_became_active":"2023-11-28T13:01:42.621835+0000","last_became_peered":"2023-11-28T13:01:42.621835+0000","last_unstale":"2023-11-28T13:01:43.733463+0000","last_undegraded":"2023-11-28T13:01:43.733463+0000","last_fullsized":"2023-11-28T13:01:43.733463+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:03.332763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652850+0000","last_change":"2023-11-28T13:01:42.620346+0000","last_active":"2023-11-28T13:01:43.652850+0000","last_peered":"2023-11-28T13:01:43.652850+0000","last_clean":"2023-11-28T13:01:43.652850+0000","last_became_active":"2023-11-28T13:01:42.620099+0000","last_became_peered":"2023-11-28T13:01:42.620099+0000","last_unstale":"2023-11-28T13:01:43.652850+0000","last_undegraded":"2023-11-28T13:01:43.652850+0000","last_fullsized":"2023-11-28T13:01:43.652850+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:08.054061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.629693+0000","last_change":"2023-11-28T13:01:42.626346+0000","last_active":"2023-11-28T13:01:42.629693+0000","last_peered":"2023-11-28T13:01:42.629693+0000","last_clean":"2023-11-28T13:01:42.629693+0000","last_became_active":"2023-11-28T13:01:42.625994+0000","last_became_peered":"2023-11-28T13:01:42.625994+0000","last_unstale":"2023-11-28T13:01:42.629693+0000","last_undegraded":"2023-11-28T13:01:42.629693+0000","last_fullsized":"2023-11-28T13:01:42.629693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:51:16.256643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602827+0000","last_change":"2023-11-28T13:01:42.610186+0000","last_active":"2023-11-28T13:01:43.602827+0000","last_peered":"2023-11-28T13:01:43.602827+0000","last_clean":"2023-11-28T13:01:43.602827+0000","last_became_active":"2023-11-28T13:01:42.609896+0000","last_became_peered":"2023-11-28T13:01:42.609896+0000","last_unstale":"2023-11-28T13:01:43.602827+0000","last_undegraded":"2023-11-28T13:01:43.602827+0000","last_fullsized":"2023-11-28T13:01:43.602827+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:02:43.012209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.605769+0000","last_change":"2023-11-28T13:01:42.613906+0000","last_active":"2023-11-28T13:01:43.605769+0000","last_peered":"2023-11-28T13:01:43.605769+0000","last_clean":"2023-11-28T13:01:43.605769+0000","last_became_active":"2023-11-28T13:01:42.613597+0000","last_became_peered":"2023-11-28T13:01:42.613597+0000","last_unstale":"2023-11-28T13:01:43.605769+0000","last_undegraded":"2023-11-28T13:01:43.605769+0000","last_fullsized":"2023-11-28T13:01:43.605769+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:40.884368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652550+0000","last_change":"2023-11-28T13:01:42.621574+0000","last_active":"2023-11-28T13:01:43.652550+0000","last_peered":"2023-11-28T13:01:43.652550+0000","last_clean":"2023-11-28T13:01:43.652550+0000","last_became_active":"2023-11-28T13:01:42.620644+0000","last_became_peered":"2023-11-28T13:01:42.620644+0000","last_unstale":"2023-11-28T13:01:43.652550+0000","last_undegraded":"2023-11-28T13:01:43.652550+0000","last_fullsized":"2023-11-28T13:01:43.652550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:58.604562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652933+0000","last_change":"2023-11-28T13:01:42.615811+0000","last_active":"2023-11-28T13:01:43.652933+0000","last_peered":"2023-11-28T13:01:43.652933+0000","last_clean":"2023-11-28T13:01:43.652933+0000","last_became_active":"2023-11-28T13:01:42.615553+0000","last_became_peered":"2023-11-28T13:01:42.615553+0000","last_unstale":"2023-11-28T13:01:43.652933+0000","last_undegraded":"2023-11-28T13:01:43.652933+0000","last_fullsized":"2023-11-28T13:01:43.652933+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:07:49.751548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.653097+0000","last_change":"2023-11-28T13:01:42.607788+0000","last_active":"2023-11-28T13:01:43.653097+0000","last_peered":"2023-11-28T13:01:43.653097+0000","last_clean":"2023-11-28T13:01:43.653097+0000","last_became_active":"2023-11-28T13:01:42.607188+0000","last_became_peered":"2023-11-28T13:01:42.607188+0000","last_unstale":"2023-11-28T13:01:43.653097+0000","last_undegraded":"2023-11-28T13:01:43.653097+0000","last_fullsized":"2023-11-28T13:01:43.653097+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:25:04.329423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652894+0000","last_change":"2023-11-28T13:01:42.619702+0000","last_active":"2023-11-28T13:01:43.652894+0000","last_peered":"2023-11-28T13:01:43.652894+0000","last_clean":"2023-11-28T13:01:43.652894+0000","last_became_active":"2023-11-28T13:01:42.619410+0000","last_became_peered":"2023-11-28T13:01:42.619410+0000","last_unstale":"2023-11-28T13:01:43.652894+0000","last_undegraded":"2023-11-28T13:01:43.652894+0000","last_fullsized":"2023-11-28T13:01:43.652894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:55.538689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627748+0000","last_change":"2023-11-28T13:01:42.623415+0000","last_active":"2023-11-28T13:01:42.627748+0000","last_peered":"2023-11-28T13:01:42.627748+0000","last_clean":"2023-11-28T13:01:42.627748+0000","last_became_active":"2023-11-28T13:01:42.623094+0000","last_became_peered":"2023-11-28T13:01:42.623094+0000","last_unstale":"2023-11-28T13:01:42.627748+0000","last_undegraded":"2023-11-28T13:01:42.627748+0000","last_fullsized":"2023-11-28T13:01:42.627748+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:43.658059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603224+0000","last_change":"2023-11-28T13:01:42.624892+0000","last_active":"2023-11-28T13:01:43.603224+0000","last_peered":"2023-11-28T13:01:43.603224+0000","last_clean":"2023-11-28T13:01:43.603224+0000","last_became_active":"2023-11-28T13:01:42.624667+0000","last_became_peered":"2023-11-28T13:01:42.624667+0000","last_unstale":"2023-11-28T13:01:43.603224+0000","last_undegraded":"2023-11-28T13:01:43.603224+0000","last_fullsized":"2023-11-28T13:01:43.603224+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:34.061420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652763+0000","last_change":"2023-11-28T13:01:42.619651+0000","last_active":"2023-11-28T13:01:43.652763+0000","last_peered":"2023-11-28T13:01:43.652763+0000","last_clean":"2023-11-28T13:01:43.652763+0000","last_became_active":"2023-11-28T13:01:42.619398+0000","last_became_peered":"2023-11-28T13:01:42.619398+0000","last_unstale":"2023-11-28T13:01:43.652763+0000","last_undegraded":"2023-11-28T13:01:43.652763+0000","last_fullsized":"2023-11-28T13:01:43.652763+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:08.655229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602628+0000","last_change":"2023-11-28T13:01:42.614356+0000","last_active":"2023-11-28T13:01:43.602628+0000","last_peered":"2023-11-28T13:01:43.602628+0000","last_clean":"2023-11-28T13:01:43.602628+0000","last_became_active":"2023-11-28T13:01:42.614064+0000","last_became_peered":"2023-11-28T13:01:42.614064+0000","last_unstale":"2023-11-28T13:01:43.602628+0000","last_undegraded":"2023-11-28T13:01:43.602628+0000","last_fullsized":"2023-11-28T13:01:43.602628+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:08.054061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627142+0000","last_change":"2023-11-28T13:01:42.618032+0000","last_active":"2023-11-28T13:01:42.627142+0000","last_peered":"2023-11-28T13:01:42.627142+0000","last_clean":"2023-11-28T13:01:42.627142+0000","last_became_active":"2023-11-28T13:01:42.617526+0000","last_became_peered":"2023-11-28T13:01:42.617526+0000","last_unstale":"2023-11-28T13:01:42.627142+0000","last_undegraded":"2023-11-28T13:01:42.627142+0000","last_fullsized":"2023-11-28T13:01:42.627142+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:07.898274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733909+0000","last_change":"2023-11-28T13:01:42.622381+0000","last_active":"2023-11-28T13:01:43.733909+0000","last_peered":"2023-11-28T13:01:43.733909+0000","last_clean":"2023-11-28T13:01:43.733909+0000","last_became_active":"2023-11-28T13:01:42.622014+0000","last_became_peered":"2023-11-28T13:01:42.622014+0000","last_unstale":"2023-11-28T13:01:43.733909+0000","last_undegraded":"2023-11-28T13:01:43.733909+0000","last_fullsized":"2023-11-28T13:01:43.733909+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:07.815817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733811+0000","last_change":"2023-11-28T13:01:42.625656+0000","last_active":"2023-11-28T13:01:43.733811+0000","last_peered":"2023-11-28T13:01:43.733811+0000","last_clean":"2023-11-28T13:01:43.733811+0000","last_became_active":"2023-11-28T13:01:42.625335+0000","last_became_peered":"2023-11-28T13:01:42.625335+0000","last_unstale":"2023-11-28T13:01:43.733811+0000","last_undegraded":"2023-11-28T13:01:43.733811+0000","last_fullsized":"2023-11-28T13:01:43.733811+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:33.547445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.616609+0000","last_change":"2023-11-28T13:01:42.610597+0000","last_active":"2023-11-28T13:01:42.616609+0000","last_peered":"2023-11-28T13:01:42.616609+0000","last_clean":"2023-11-28T13:01:42.616609+0000","last_became_active":"2023-11-28T13:01:42.610350+0000","last_became_peered":"2023-11-28T13:01:42.610350+0000","last_unstale":"2023-11-28T13:01:42.616609+0000","last_undegraded":"2023-11-28T13:01:42.616609+0000","last_fullsized":"2023-11-28T13:01:42.616609+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:20.903612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602964+0000","last_change":"2023-11-28T13:01:42.622745+0000","last_active":"2023-11-28T13:01:43.602964+0000","last_peered":"2023-11-28T13:01:43.602964+0000","last_clean":"2023-11-28T13:01:43.602964+0000","last_became_active":"2023-11-28T13:01:42.622386+0000","last_became_peered":"2023-11-28T13:01:42.622386+0000","last_unstale":"2023-11-28T13:01:43.602964+0000","last_undegraded":"2023-11-28T13:01:43.602964+0000","last_fullsized":"2023-11-28T13:01:43.602964+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:52.405752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603017+0000","last_change":"2023-11-28T13:01:42.610652+0000","last_active":"2023-11-28T13:01:43.603017+0000","last_peered":"2023-11-28T13:01:43.603017+0000","last_clean":"2023-11-28T13:01:43.603017+0000","last_became_active":"2023-11-28T13:01:42.610321+0000","last_became_peered":"2023-11-28T13:01:42.610321+0000","last_unstale":"2023-11-28T13:01:43.603017+0000","last_undegraded":"2023-11-28T13:01:43.603017+0000","last_fullsized":"2023-11-28T13:01:43.603017+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:37.363198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652753+0000","last_change":"2023-11-28T13:01:42.620156+0000","last_active":"2023-11-28T13:01:43.652753+0000","last_peered":"2023-11-28T13:01:43.652753+0000","last_clean":"2023-11-28T13:01:43.652753+0000","last_became_active":"2023-11-28T13:01:42.619906+0000","last_became_peered":"2023-11-28T13:01:42.619906+0000","last_unstale":"2023-11-28T13:01:43.652753+0000","last_undegraded":"2023-11-28T13:01:43.652753+0000","last_fullsized":"2023-11-28T13:01:43.652753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:40.189110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.614414+0000","last_change":"2023-11-28T13:01:42.607938+0000","last_active":"2023-11-28T13:01:42.614414+0000","last_peered":"2023-11-28T13:01:42.614414+0000","last_clean":"2023-11-28T13:01:42.614414+0000","last_became_active":"2023-11-28T13:01:42.607560+0000","last_became_peered":"2023-11-28T13:01:42.607560+0000","last_unstale":"2023-11-28T13:01:42.614414+0000","last_undegraded":"2023-11-28T13:01:42.614414+0000","last_fullsized":"2023-11-28T13:01:42.614414+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:27:15.550145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.614790+0000","last_change":"2023-11-28T13:01:42.607375+0000","last_active":"2023-11-28T13:01:42.614790+0000","last_peered":"2023-11-28T13:01:42.614790+0000","last_clean":"2023-11-28T13:01:42.614790+0000","last_became_active":"2023-11-28T13:01:42.607116+0000","last_became_peered":"2023-11-28T13:01:42.607116+0000","last_unstale":"2023-11-28T13:01:42.614790+0000","last_undegraded":"2023-11-28T13:01:42.614790+0000","last_fullsized":"2023-11-28T13:01:42.614790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:47.081647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.618123+0000","last_change":"2023-11-28T13:01:42.613296+0000","last_active":"2023-11-28T13:01:42.618123+0000","last_peered":"2023-11-28T13:01:42.618123+0000","last_clean":"2023-11-28T13:01:42.618123+0000","last_became_active":"2023-11-28T13:01:42.612988+0000","last_became_peered":"2023-11-28T13:01:42.612988+0000","last_unstale":"2023-11-28T13:01:42.618123+0000","last_undegraded":"2023-11-28T13:01:42.618123+0000","last_fullsized":"2023-11-28T13:01:42.618123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:46.649699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603770+0000","last_change":"2023-11-28T13:01:42.624183+0000","last_active":"2023-11-28T13:01:43.603770+0000","last_peered":"2023-11-28T13:01:43.603770+0000","last_clean":"2023-11-28T13:01:43.603770+0000","last_became_active":"2023-11-28T13:01:42.623920+0000","last_became_peered":"2023-11-28T13:01:42.623920+0000","last_unstale":"2023-11-28T13:01:43.603770+0000","last_undegraded":"2023-11-28T13:01:43.603770+0000","last_fullsized":"2023-11-28T13:01:43.603770+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:47.081647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733461+0000","last_change":"2023-11-28T13:01:42.621240+0000","last_active":"2023-11-28T13:01:43.733461+0000","last_peered":"2023-11-28T13:01:43.733461+0000","last_clean":"2023-11-28T13:01:43.733461+0000","last_became_active":"2023-11-28T13:01:42.620939+0000","last_became_peered":"2023-11-28T13:01:42.620939+0000","last_unstale":"2023-11-28T13:01:43.733461+0000","last_undegraded":"2023-11-28T13:01:43.733461+0000","last_fullsized":"2023-11-28T13:01:43.733461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:03:39.615987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627773+0000","last_change":"2023-11-28T13:01:42.623446+0000","last_active":"2023-11-28T13:01:42.627773+0000","last_peered":"2023-11-28T13:01:42.627773+0000","last_clean":"2023-11-28T13:01:42.627773+0000","last_became_active":"2023-11-28T13:01:42.623114+0000","last_became_peered":"2023-11-28T13:01:42.623114+0000","last_unstale":"2023-11-28T13:01:42.627773+0000","last_undegraded":"2023-11-28T13:01:42.627773+0000","last_fullsized":"2023-11-28T13:01:42.627773+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:01:42.963433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.617151+0000","last_change":"2023-11-28T13:01:42.613236+0000","last_active":"2023-11-28T13:01:42.617151+0000","last_peered":"2023-11-28T13:01:42.617151+0000","last_clean":"2023-11-28T13:01:42.617151+0000","last_became_active":"2023-11-28T13:01:42.612892+0000","last_became_peered":"2023-11-28T13:01:42.612892+0000","last_unstale":"2023-11-28T13:01:42.617151+0000","last_undegraded":"2023-11-28T13:01:42.617151+0000","last_fullsized":"2023-11-28T13:01:42.617151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:41:17.278714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733295+0000","last_change":"2023-11-28T13:01:42.626247+0000","last_active":"2023-11-28T13:01:43.733295+0000","last_peered":"2023-11-28T13:01:43.733295+0000","last_clean":"2023-11-28T13:01:43.733295+0000","last_became_active":"2023-11-28T13:01:42.625971+0000","last_became_peered":"2023-11-28T13:01:42.625971+0000","last_unstale":"2023-11-28T13:01:43.733295+0000","last_undegraded":"2023-11-28T13:01:43.733295+0000","last_fullsized":"2023-11-28T13:01:43.733295+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:41:23.267974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602658+0000","last_change":"2023-11-28T13:01:42.625666+0000","last_active":"2023-11-28T13:01:43.602658+0000","last_peered":"2023-11-28T13:01:43.602658+0000","last_clean":"2023-11-28T13:01:43.602658+0000","last_became_active":"2023-11-28T13:01:42.625263+0000","last_became_peered":"2023-11-28T13:01:42.625263+0000","last_unstale":"2023-11-28T13:01:43.602658+0000","last_undegraded":"2023-11-28T13:01:43.602658+0000","last_fullsized":"2023-11-28T13:01:43.602658+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:08.655229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602796+0000","last_change":"2023-11-28T13:01:42.615901+0000","last_active":"2023-11-28T13:01:43.602796+0000","last_peered":"2023-11-28T13:01:43.602796+0000","last_clean":"2023-11-28T13:01:43.602796+0000","last_became_active":"2023-11-28T13:01:42.615634+0000","last_became_peered":"2023-11-28T13:01:42.615634+0000","last_unstale":"2023-11-28T13:01:43.602796+0000","last_undegraded":"2023-11-28T13:01:43.602796+0000","last_fullsized":"2023-11-28T13:01:43.602796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:05.525990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.615240+0000","last_change":"2023-11-28T13:01:42.612785+0000","last_active":"2023-11-28T13:01:42.615240+0000","last_peered":"2023-11-28T13:01:42.615240+0000","last_clean":"2023-11-28T13:01:42.615240+0000","last_became_active":"2023-11-28T13:01:42.612519+0000","last_became_peered":"2023-11-28T13:01:42.612519+0000","last_unstale":"2023-11-28T13:01:42.615240+0000","last_undegraded":"2023-11-28T13:01:42.615240+0000","last_fullsized":"2023-11-28T13:01:42.615240+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:11.090771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"39'4","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603585+0000","last_change":"2023-11-28T13:01:42.191064+0000","last_active":"2023-11-28T13:01:43.603585+0000","last_peered":"2023-11-28T13:01:43.603585+0000","last_clean":"2023-11-28T13:01:43.603585+0000","last_became_active":"2023-11-28T13:01:42.190617+0000","last_became_peered":"2023-11-28T13:01:42.190617+0000","last_unstale":"2023-11-28T13:01:43.603585+0000","last_undegraded":"2023-11-28T13:01:43.603585+0000","last_fullsized":"2023-11-28T13:01:43.603585+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:38:22.687980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627111+0000","last_change":"2023-11-28T13:01:42.618889+0000","last_active":"2023-11-28T13:01:42.627111+0000","last_peered":"2023-11-28T13:01:42.627111+0000","last_clean":"2023-11-28T13:01:42.627111+0000","last_became_active":"2023-11-28T13:01:42.618661+0000","last_became_peered":"2023-11-28T13:01:42.618661+0000","last_unstale":"2023-11-28T13:01:42.627111+0000","last_undegraded":"2023-11-28T13:01:42.627111+0000","last_fullsized":"2023-11-28T13:01:42.627111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:30.356407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733820+0000","last_change":"2023-11-28T13:01:42.196934+0000","last_active":"2023-11-28T13:01:43.733820+0000","last_peered":"2023-11-28T13:01:43.733820+0000","last_clean":"2023-11-28T13:01:43.733820+0000","last_became_active":"2023-11-28T13:01:42.196626+0000","last_became_peered":"2023-11-28T13:01:42.196626+0000","last_unstale":"2023-11-28T13:01:43.733820+0000","last_undegraded":"2023-11-28T13:01:43.733820+0000","last_fullsized":"2023-11-28T13:01:43.733820+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:10.421372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.629734+0000","last_change":"2023-11-28T13:01:42.627323+0000","last_active":"2023-11-28T13:01:42.629734+0000","last_peered":"2023-11-28T13:01:42.629734+0000","last_clean":"2023-11-28T13:01:42.629734+0000","last_became_active":"2023-11-28T13:01:42.627034+0000","last_became_peered":"2023-11-28T13:01:42.627034+0000","last_unstale":"2023-11-28T13:01:42.629734+0000","last_undegraded":"2023-11-28T13:01:42.629734+0000","last_fullsized":"2023-11-28T13:01:42.629734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:39:01.198843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.626279+0000","last_change":"2023-11-28T13:01:42.198461+0000","last_active":"2023-11-28T13:01:42.626279+0000","last_peered":"2023-11-28T13:01:42.626279+0000","last_clean":"2023-11-28T13:01:42.626279+0000","last_became_active":"2023-11-28T13:01:42.198213+0000","last_became_peered":"2023-11-28T13:01:42.198213+0000","last_unstale":"2023-11-28T13:01:42.626279+0000","last_undegraded":"2023-11-28T13:01:42.626279+0000","last_fullsized":"2023-11-28T13:01:42.626279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:31.551094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.625937+0000","last_change":"2023-11-28T13:01:42.616223+0000","last_active":"2023-11-28T13:01:42.625937+0000","last_peered":"2023-11-28T13:01:42.625937+0000","last_clean":"2023-11-28T13:01:42.625937+0000","last_became_active":"2023-11-28T13:01:42.615686+0000","last_became_peered":"2023-11-28T13:01:42.615686+0000","last_unstale":"2023-11-28T13:01:42.625937+0000","last_undegraded":"2023-11-28T13:01:42.625937+0000","last_fullsized":"2023-11-28T13:01:42.625937+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:29.410609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606004+0000","last_change":"2023-11-28T13:01:42.198227+0000","last_active":"2023-11-28T13:01:43.606004+0000","last_peered":"2023-11-28T13:01:43.606004+0000","last_clean":"2023-11-28T13:01:43.606004+0000","last_became_active":"2023-11-28T13:01:42.197941+0000","last_became_peered":"2023-11-28T13:01:42.197941+0000","last_unstale":"2023-11-28T13:01:43.606004+0000","last_undegraded":"2023-11-28T13:01:43.606004+0000","last_fullsized":"2023-11-28T13:01:43.606004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:45.645303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733324+0000","last_change":"2023-11-28T13:01:42.625191+0000","last_active":"2023-11-28T13:01:43.733324+0000","last_peered":"2023-11-28T13:01:43.733324+0000","last_clean":"2023-11-28T13:01:43.733324+0000","last_became_active":"2023-11-28T13:01:42.624441+0000","last_became_peered":"2023-11-28T13:01:42.624441+0000","last_unstale":"2023-11-28T13:01:43.733324+0000","last_undegraded":"2023-11-28T13:01:43.733324+0000","last_fullsized":"2023-11-28T13:01:43.733324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:35.576766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603113+0000","last_change":"2023-11-28T13:01:42.187433+0000","last_active":"2023-11-28T13:01:43.603113+0000","last_peered":"2023-11-28T13:01:43.603113+0000","last_clean":"2023-11-28T13:01:43.603113+0000","last_became_active":"2023-11-28T13:01:42.187145+0000","last_became_peered":"2023-11-28T13:01:42.187145+0000","last_unstale":"2023-11-28T13:01:43.603113+0000","last_undegraded":"2023-11-28T13:01:43.603113+0000","last_fullsized":"2023-11-28T13:01:43.603113+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:17.338814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652997+0000","last_change":"2023-11-28T13:01:42.616703+0000","last_active":"2023-11-28T13:01:43.652997+0000","last_peered":"2023-11-28T13:01:43.652997+0000","last_clean":"2023-11-28T13:01:43.652997+0000","last_became_active":"2023-11-28T13:01:42.615202+0000","last_became_peered":"2023-11-28T13:01:42.615202+0000","last_unstale":"2023-11-28T13:01:43.652997+0000","last_undegraded":"2023-11-28T13:01:43.652997+0000","last_fullsized":"2023-11-28T13:01:43.652997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:24.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733431+0000","last_change":"2023-11-28T13:01:42.199326+0000","last_active":"2023-11-28T13:01:43.733431+0000","last_peered":"2023-11-28T13:01:43.733431+0000","last_clean":"2023-11-28T13:01:43.733431+0000","last_became_active":"2023-11-28T13:01:42.198914+0000","last_became_peered":"2023-11-28T13:01:42.198914+0000","last_unstale":"2023-11-28T13:01:43.733431+0000","last_undegraded":"2023-11-28T13:01:43.733431+0000","last_fullsized":"2023-11-28T13:01:43.733431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:08.791421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652708+0000","last_change":"2023-11-28T13:01:42.609755+0000","last_active":"2023-11-28T13:01:43.652708+0000","last_peered":"2023-11-28T13:01:43.652708+0000","last_clean":"2023-11-28T13:01:43.652708+0000","last_became_active":"2023-11-28T13:01:42.609504+0000","last_became_peered":"2023-11-28T13:01:42.609504+0000","last_unstale":"2023-11-28T13:01:43.652708+0000","last_undegraded":"2023-11-28T13:01:43.652708+0000","last_fullsized":"2023-11-28T13:01:43.652708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:02:43.012209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.614390+0000","last_change":"2023-11-28T13:01:42.186635+0000","last_active":"2023-11-28T13:01:42.614390+0000","last_peered":"2023-11-28T13:01:42.614390+0000","last_clean":"2023-11-28T13:01:42.614390+0000","last_became_active":"2023-11-28T13:01:42.186328+0000","last_became_peered":"2023-11-28T13:01:42.186328+0000","last_unstale":"2023-11-28T13:01:42.614390+0000","last_undegraded":"2023-11-28T13:01:42.614390+0000","last_fullsized":"2023-11-28T13:01:42.614390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:29.352010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.614380+0000","last_change":"2023-11-28T13:01:42.607459+0000","last_active":"2023-11-28T13:01:42.614380+0000","last_peered":"2023-11-28T13:01:42.614380+0000","last_clean":"2023-11-28T13:01:42.614380+0000","last_became_active":"2023-11-28T13:01:42.607104+0000","last_became_peered":"2023-11-28T13:01:42.607104+0000","last_unstale":"2023-11-28T13:01:42.614380+0000","last_undegraded":"2023-11-28T13:01:42.614380+0000","last_fullsized":"2023-11-28T13:01:42.614380+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:34.061420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652645+0000","last_change":"2023-11-28T13:01:42.189399+0000","last_active":"2023-11-28T13:01:43.652645+0000","last_peered":"2023-11-28T13:01:43.652645+0000","last_clean":"2023-11-28T13:01:43.652645+0000","last_became_active":"2023-11-28T13:01:42.189135+0000","last_became_peered":"2023-11-28T13:01:42.189135+0000","last_unstale":"2023-11-28T13:01:43.652645+0000","last_undegraded":"2023-11-28T13:01:43.652645+0000","last_fullsized":"2023-11-28T13:01:43.652645+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:29.267568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733949+0000","last_change":"2023-11-28T13:01:42.621702+0000","last_active":"2023-11-28T13:01:43.733949+0000","last_peered":"2023-11-28T13:01:43.733949+0000","last_clean":"2023-11-28T13:01:43.733949+0000","last_became_active":"2023-11-28T13:01:42.621419+0000","last_became_peered":"2023-11-28T13:01:42.621419+0000","last_unstale":"2023-11-28T13:01:43.733949+0000","last_undegraded":"2023-11-28T13:01:43.733949+0000","last_fullsized":"2023-11-28T13:01:43.733949+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:19.749490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733983+0000","last_change":"2023-11-28T13:01:42.193953+0000","last_active":"2023-11-28T13:01:43.733983+0000","last_peered":"2023-11-28T13:01:43.733983+0000","last_clean":"2023-11-28T13:01:43.733983+0000","last_became_active":"2023-11-28T13:01:42.193687+0000","last_became_peered":"2023-11-28T13:01:42.193687+0000","last_unstale":"2023-11-28T13:01:43.733983+0000","last_undegraded":"2023-11-28T13:01:43.733983+0000","last_fullsized":"2023-11-28T13:01:43.733983+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:54.891996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606239+0000","last_change":"2023-11-28T13:01:42.621535+0000","last_active":"2023-11-28T13:01:43.606239+0000","last_peered":"2023-11-28T13:01:43.606239+0000","last_clean":"2023-11-28T13:01:43.606239+0000","last_became_active":"2023-11-28T13:01:42.621233+0000","last_became_peered":"2023-11-28T13:01:42.621233+0000","last_unstale":"2023-11-28T13:01:43.606239+0000","last_undegraded":"2023-11-28T13:01:43.606239+0000","last_fullsized":"2023-11-28T13:01:43.606239+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:45:49.367244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.653141+0000","last_change":"2023-11-28T13:01:42.191414+0000","last_active":"2023-11-28T13:01:43.653141+0000","last_peered":"2023-11-28T13:01:43.653141+0000","last_clean":"2023-11-28T13:01:43.653141+0000","last_became_active":"2023-11-28T13:01:42.191040+0000","last_became_peered":"2023-11-28T13:01:42.191040+0000","last_unstale":"2023-11-28T13:01:43.653141+0000","last_undegraded":"2023-11-28T13:01:43.653141+0000","last_fullsized":"2023-11-28T13:01:43.653141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:38.246885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603448+0000","last_change":"2023-11-28T13:01:42.625984+0000","last_active":"2023-11-28T13:01:43.603448+0000","last_peered":"2023-11-28T13:01:43.603448+0000","last_clean":"2023-11-28T13:01:43.603448+0000","last_became_active":"2023-11-28T13:01:42.625744+0000","last_became_peered":"2023-11-28T13:01:42.625744+0000","last_unstale":"2023-11-28T13:01:43.603448+0000","last_undegraded":"2023-11-28T13:01:43.603448+0000","last_fullsized":"2023-11-28T13:01:43.603448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:34:29.680397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"39'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606196+0000","last_change":"2023-11-28T13:01:42.193850+0000","last_active":"2023-11-28T13:01:43.606196+0000","last_peered":"2023-11-28T13:01:43.606196+0000","last_clean":"2023-11-28T13:01:43.606196+0000","last_became_active":"2023-11-28T13:01:42.193484+0000","last_became_peered":"2023-11-28T13:01:42.193484+0000","last_unstale":"2023-11-28T13:01:43.606196+0000","last_undegraded":"2023-11-28T13:01:43.606196+0000","last_fullsized":"2023-11-28T13:01:43.606196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:34.635440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606030+0000","last_change":"2023-11-28T13:01:42.617068+0000","last_active":"2023-11-28T13:01:43.606030+0000","last_peered":"2023-11-28T13:01:43.606030+0000","last_clean":"2023-11-28T13:01:43.606030+0000","last_became_active":"2023-11-28T13:01:42.616795+0000","last_became_peered":"2023-11-28T13:01:42.616795+0000","last_unstale":"2023-11-28T13:01:43.606030+0000","last_undegraded":"2023-11-28T13:01:43.606030+0000","last_fullsized":"2023-11-28T13:01:43.606030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:11.425769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"39'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603128+0000","last_change":"2023-11-28T13:01:42.194421+0000","last_active":"2023-11-28T13:01:43.603128+0000","last_peered":"2023-11-28T13:01:43.603128+0000","last_clean":"2023-11-28T13:01:43.603128+0000","last_became_active":"2023-11-28T13:01:42.194077+0000","last_became_peered":"2023-11-28T13:01:42.194077+0000","last_unstale":"2023-11-28T13:01:43.603128+0000","last_undegraded":"2023-11-28T13:01:43.603128+0000","last_fullsized":"2023-11-28T13:01:43.603128+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:27.262805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.616984+0000","last_change":"2023-11-28T13:01:42.610042+0000","last_active":"2023-11-28T13:01:42.616984+0000","last_peered":"2023-11-28T13:01:42.616984+0000","last_clean":"2023-11-28T13:01:42.616984+0000","last_became_active":"2023-11-28T13:01:42.609715+0000","last_became_peered":"2023-11-28T13:01:42.609715+0000","last_unstale":"2023-11-28T13:01:42.616984+0000","last_undegraded":"2023-11-28T13:01:42.616984+0000","last_fullsized":"2023-11-28T13:01:42.616984+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:56.807979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603295+0000","last_change":"2023-11-28T13:01:42.193660+0000","last_active":"2023-11-28T13:01:43.603295+0000","last_peered":"2023-11-28T13:01:43.603295+0000","last_clean":"2023-11-28T13:01:43.603295+0000","last_became_active":"2023-11-28T13:01:42.193379+0000","last_became_peered":"2023-11-28T13:01:42.193379+0000","last_unstale":"2023-11-28T13:01:43.603295+0000","last_undegraded":"2023-11-28T13:01:43.603295+0000","last_fullsized":"2023-11-28T13:01:43.603295+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:50:47.996723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.653034+0000","last_change":"2023-11-28T13:01:42.618330+0000","last_active":"2023-11-28T13:01:43.653034+0000","last_peered":"2023-11-28T13:01:43.653034+0000","last_clean":"2023-11-28T13:01:43.653034+0000","last_became_active":"2023-11-28T13:01:42.617153+0000","last_became_peered":"2023-11-28T13:01:42.617153+0000","last_unstale":"2023-11-28T13:01:43.653034+0000","last_undegraded":"2023-11-28T13:01:43.653034+0000","last_fullsized":"2023-11-28T13:01:43.653034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:16.224085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.605920+0000","last_change":"2023-11-28T13:01:42.185588+0000","last_active":"2023-11-28T13:01:43.605920+0000","last_peered":"2023-11-28T13:01:43.605920+0000","last_clean":"2023-11-28T13:01:43.605920+0000","last_became_active":"2023-11-28T13:01:42.185273+0000","last_became_peered":"2023-11-28T13:01:42.185273+0000","last_unstale":"2023-11-28T13:01:43.605920+0000","last_undegraded":"2023-11-28T13:01:43.605920+0000","last_fullsized":"2023-11-28T13:01:43.605920+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:50.497243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602548+0000","last_change":"2023-11-28T13:01:42.620480+0000","last_active":"2023-11-28T13:01:43.602548+0000","last_peered":"2023-11-28T13:01:43.602548+0000","last_clean":"2023-11-28T13:01:43.602548+0000","last_became_active":"2023-11-28T13:01:42.620164+0000","last_became_peered":"2023-11-28T13:01:42.620164+0000","last_unstale":"2023-11-28T13:01:43.602548+0000","last_undegraded":"2023-11-28T13:01:43.602548+0000","last_fullsized":"2023-11-28T13:01:43.602548+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:16.224085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602741+0000","last_change":"2023-11-28T13:01:42.193694+0000","last_active":"2023-11-28T13:01:43.602741+0000","last_peered":"2023-11-28T13:01:43.602741+0000","last_clean":"2023-11-28T13:01:43.602741+0000","last_became_active":"2023-11-28T13:01:42.193454+0000","last_became_peered":"2023-11-28T13:01:42.193454+0000","last_unstale":"2023-11-28T13:01:43.602741+0000","last_undegraded":"2023-11-28T13:01:43.602741+0000","last_fullsized":"2023-11-28T13:01:43.602741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:38:36.902601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.605918+0000","last_change":"2023-11-28T13:01:42.606829+0000","last_active":"2023-11-28T13:01:43.605918+0000","last_peered":"2023-11-28T13:01:43.605918+0000","last_clean":"2023-11-28T13:01:43.605918+0000","last_became_active":"2023-11-28T13:01:42.606524+0000","last_became_peered":"2023-11-28T13:01:42.606524+0000","last_unstale":"2023-11-28T13:01:43.605918+0000","last_undegraded":"2023-11-28T13:01:43.605918+0000","last_fullsized":"2023-11-28T13:01:43.605918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:55.896392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"39'8","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652750+0000","last_change":"2023-11-28T13:01:42.199301+0000","last_active":"2023-11-28T13:01:43.652750+0000","last_peered":"2023-11-28T13:01:43.652750+0000","last_clean":"2023-11-28T13:01:43.652750+0000","last_became_active":"2023-11-28T13:01:42.198993+0000","last_became_peered":"2023-11-28T13:01:42.198993+0000","last_unstale":"2023-11-28T13:01:43.652750+0000","last_undegraded":"2023-11-28T13:01:43.652750+0000","last_fullsized":"2023-11-28T13:01:43.652750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:31:22.563114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603823+0000","last_change":"2023-11-28T13:01:42.622012+0000","last_active":"2023-11-28T13:01:43.603823+0000","last_peered":"2023-11-28T13:01:43.603823+0000","last_clean":"2023-11-28T13:01:43.603823+0000","last_became_active":"2023-11-28T13:01:42.621799+0000","last_became_peered":"2023-11-28T13:01:42.621799+0000","last_unstale":"2023-11-28T13:01:43.603823+0000","last_undegraded":"2023-11-28T13:01:43.603823+0000","last_fullsized":"2023-11-28T13:01:43.603823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:25:13.025584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603281+0000","last_change":"2023-11-28T13:01:42.193639+0000","last_active":"2023-11-28T13:01:43.603281+0000","last_peered":"2023-11-28T13:01:43.603281+0000","last_clean":"2023-11-28T13:01:43.603281+0000","last_became_active":"2023-11-28T13:01:42.193379+0000","last_became_peered":"2023-11-28T13:01:42.193379+0000","last_unstale":"2023-11-28T13:01:43.603281+0000","last_undegraded":"2023-11-28T13:01:43.603281+0000","last_fullsized":"2023-11-28T13:01:43.603281+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:57:03.220754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652989+0000","last_change":"2023-11-28T13:01:42.612684+0000","last_active":"2023-11-28T13:01:43.652989+0000","last_peered":"2023-11-28T13:01:43.652989+0000","last_clean":"2023-11-28T13:01:43.652989+0000","last_became_active":"2023-11-28T13:01:42.611096+0000","last_became_peered":"2023-11-28T13:01:42.611096+0000","last_unstale":"2023-11-28T13:01:43.652989+0000","last_undegraded":"2023-11-28T13:01:43.652989+0000","last_fullsized":"2023-11-28T13:01:43.652989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:22.271937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.734000+0000","last_change":"2023-11-28T13:01:42.194635+0000","last_active":"2023-11-28T13:01:43.734000+0000","last_peered":"2023-11-28T13:01:43.734000+0000","last_clean":"2023-11-28T13:01:43.734000+0000","last_became_active":"2023-11-28T13:01:42.194326+0000","last_became_peered":"2023-11-28T13:01:42.194326+0000","last_unstale":"2023-11-28T13:01:43.734000+0000","last_undegraded":"2023-11-28T13:01:43.734000+0000","last_fullsized":"2023-11-28T13:01:43.734000+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:07.286796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.626910+0000","last_change":"2023-11-28T13:01:42.617512+0000","last_active":"2023-11-28T13:01:42.626910+0000","last_peered":"2023-11-28T13:01:42.626910+0000","last_clean":"2023-11-28T13:01:42.626910+0000","last_became_active":"2023-11-28T13:01:42.616736+0000","last_became_peered":"2023-11-28T13:01:42.616736+0000","last_unstale":"2023-11-28T13:01:42.626910+0000","last_undegraded":"2023-11-28T13:01:42.626910+0000","last_fullsized":"2023-11-28T13:01:42.626910+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:35.639836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"39'1","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.626828+0000","last_change":"2023-11-28T13:01:42.197100+0000","last_active":"2023-11-28T13:01:42.626828+0000","last_peered":"2023-11-28T13:01:42.626828+0000","last_clean":"2023-11-28T13:01:42.626828+0000","last_became_active":"2023-11-28T13:01:42.196854+0000","last_became_peered":"2023-11-28T13:01:42.196854+0000","last_unstale":"2023-11-28T13:01:42.626828+0000","last_undegraded":"2023-11-28T13:01:42.626828+0000","last_fullsized":"2023-11-28T13:01:42.626828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:35.106734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627192+0000","last_change":"2023-11-28T13:01:42.619671+0000","last_active":"2023-11-28T13:01:42.627192+0000","last_peered":"2023-11-28T13:01:42.627192+0000","last_clean":"2023-11-28T13:01:42.627192+0000","last_became_active":"2023-11-28T13:01:42.617084+0000","last_became_peered":"2023-11-28T13:01:42.617084+0000","last_unstale":"2023-11-28T13:01:42.627192+0000","last_undegraded":"2023-11-28T13:01:42.627192+0000","last_fullsized":"2023-11-28T13:01:42.627192+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:51.501639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603523+0000","last_change":"2023-11-28T13:01:42.189324+0000","last_active":"2023-11-28T13:01:43.603523+0000","last_peered":"2023-11-28T13:01:43.603523+0000","last_clean":"2023-11-28T13:01:43.603523+0000","last_became_active":"2023-11-28T13:01:42.189011+0000","last_became_peered":"2023-11-28T13:01:42.189011+0000","last_unstale":"2023-11-28T13:01:43.603523+0000","last_undegraded":"2023-11-28T13:01:43.603523+0000","last_fullsized":"2023-11-28T13:01:43.603523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:53.830121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602768+0000","last_change":"2023-11-28T13:01:42.620560+0000","last_active":"2023-11-28T13:01:43.602768+0000","last_peered":"2023-11-28T13:01:43.602768+0000","last_clean":"2023-11-28T13:01:43.602768+0000","last_became_active":"2023-11-28T13:01:42.620275+0000","last_became_peered":"2023-11-28T13:01:42.620275+0000","last_unstale":"2023-11-28T13:01:43.602768+0000","last_undegraded":"2023-11-28T13:01:43.602768+0000","last_fullsized":"2023-11-28T13:01:43.602768+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:03:45.927554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603347+0000","last_change":"2023-11-28T13:01:42.187236+0000","last_active":"2023-11-28T13:01:43.603347+0000","last_peered":"2023-11-28T13:01:43.603347+0000","last_clean":"2023-11-28T13:01:43.603347+0000","last_became_active":"2023-11-28T13:01:42.186927+0000","last_became_peered":"2023-11-28T13:01:42.186927+0000","last_unstale":"2023-11-28T13:01:43.603347+0000","last_undegraded":"2023-11-28T13:01:43.603347+0000","last_fullsized":"2023-11-28T13:01:43.603347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:16.716838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.626132+0000","last_change":"2023-11-28T13:01:42.621914+0000","last_active":"2023-11-28T13:01:42.626132+0000","last_peered":"2023-11-28T13:01:42.626132+0000","last_clean":"2023-11-28T13:01:42.626132+0000","last_became_active":"2023-11-28T13:01:42.621624+0000","last_became_peered":"2023-11-28T13:01:42.621624+0000","last_unstale":"2023-11-28T13:01:42.626132+0000","last_undegraded":"2023-11-28T13:01:42.626132+0000","last_fullsized":"2023-11-28T13:01:42.626132+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:08.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.612522+0000","last_change":"2023-11-28T13:01:42.196503+0000","last_active":"2023-11-28T13:01:42.612522+0000","last_peered":"2023-11-28T13:01:42.612522+0000","last_clean":"2023-11-28T13:01:42.612522+0000","last_became_active":"2023-11-28T13:01:42.196215+0000","last_became_peered":"2023-11-28T13:01:42.196215+0000","last_unstale":"2023-11-28T13:01:42.612522+0000","last_undegraded":"2023-11-28T13:01:42.612522+0000","last_fullsized":"2023-11-28T13:01:42.612522+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:49.909597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602898+0000","last_change":"2023-11-28T13:01:42.617055+0000","last_active":"2023-11-28T13:01:43.602898+0000","last_peered":"2023-11-28T13:01:43.602898+0000","last_clean":"2023-11-28T13:01:43.602898+0000","last_became_active":"2023-11-28T13:01:42.616908+0000","last_became_peered":"2023-11-28T13:01:42.616908+0000","last_unstale":"2023-11-28T13:01:43.602898+0000","last_undegraded":"2023-11-28T13:01:43.602898+0000","last_fullsized":"2023-11-28T13:01:43.602898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:56.456532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652606+0000","last_change":"2023-11-28T13:01:42.199791+0000","last_active":"2023-11-28T13:01:43.652606+0000","last_peered":"2023-11-28T13:01:43.652606+0000","last_clean":"2023-11-28T13:01:43.652606+0000","last_became_active":"2023-11-28T13:01:42.199486+0000","last_became_peered":"2023-11-28T13:01:42.199486+0000","last_unstale":"2023-11-28T13:01:43.652606+0000","last_undegraded":"2023-11-28T13:01:43.652606+0000","last_fullsized":"2023-11-28T13:01:43.652606+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:02.788585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652688+0000","last_change":"2023-11-28T13:01:42.619612+0000","last_active":"2023-11-28T13:01:43.652688+0000","last_peered":"2023-11-28T13:01:43.652688+0000","last_clean":"2023-11-28T13:01:43.652688+0000","last_became_active":"2023-11-28T13:01:42.619350+0000","last_became_peered":"2023-11-28T13:01:42.619350+0000","last_unstale":"2023-11-28T13:01:43.652688+0000","last_undegraded":"2023-11-28T13:01:43.652688+0000","last_fullsized":"2023-11-28T13:01:43.652688+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:37.363198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.605549+0000","last_change":"2023-11-28T13:01:42.185587+0000","last_active":"2023-11-28T13:01:43.605549+0000","last_peered":"2023-11-28T13:01:43.605549+0000","last_clean":"2023-11-28T13:01:43.605549+0000","last_became_active":"2023-11-28T13:01:42.185256+0000","last_became_peered":"2023-11-28T13:01:42.185256+0000","last_unstale":"2023-11-28T13:01:43.605549+0000","last_undegraded":"2023-11-28T13:01:43.605549+0000","last_fullsized":"2023-11-28T13:01:43.605549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:19.899215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603202+0000","last_change":"2023-11-28T13:01:42.621313+0000","last_active":"2023-11-28T13:01:43.603202+0000","last_peered":"2023-11-28T13:01:43.603202+0000","last_clean":"2023-11-28T13:01:43.603202+0000","last_became_active":"2023-11-28T13:01:42.621051+0000","last_became_peered":"2023-11-28T13:01:42.621051+0000","last_unstale":"2023-11-28T13:01:43.603202+0000","last_undegraded":"2023-11-28T13:01:43.603202+0000","last_fullsized":"2023-11-28T13:01:43.603202+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:25:04.329423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733491+0000","last_change":"2023-11-28T13:01:42.194697+0000","last_active":"2023-11-28T13:01:43.733491+0000","last_peered":"2023-11-28T13:01:43.733491+0000","last_clean":"2023-11-28T13:01:43.733491+0000","last_became_active":"2023-11-28T13:01:42.194387+0000","last_became_peered":"2023-11-28T13:01:42.194387+0000","last_unstale":"2023-11-28T13:01:43.733491+0000","last_undegraded":"2023-11-28T13:01:43.733491+0000","last_fullsized":"2023-11-28T13:01:43.733491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:39.879971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.610082+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:01:42.610082+0000","last_peered":"2023-11-28T13:01:42.610082+0000","last_clean":"2023-11-28T13:01:42.610082+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:01:42.610082+0000","last_undegraded":"2023-11-28T13:01:42.610082+0000","last_fullsized":"2023-11-28T13:01:42.610082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602873+0000","last_change":"2023-11-28T13:01:42.622392+0000","last_active":"2023-11-28T13:01:43.602873+0000","last_peered":"2023-11-28T13:01:43.602873+0000","last_clean":"2023-11-28T13:01:43.602873+0000","last_became_active":"2023-11-28T13:01:42.622121+0000","last_became_peered":"2023-11-28T13:01:42.622121+0000","last_unstale":"2023-11-28T13:01:43.602873+0000","last_undegraded":"2023-11-28T13:01:43.602873+0000","last_fullsized":"2023-11-28T13:01:43.602873+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:55.538689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606013+0000","last_change":"2023-11-28T13:01:42.190826+0000","last_active":"2023-11-28T13:01:43.606013+0000","last_peered":"2023-11-28T13:01:43.606013+0000","last_clean":"2023-11-28T13:01:43.606013+0000","last_became_active":"2023-11-28T13:01:42.190488+0000","last_became_peered":"2023-11-28T13:01:42.190488+0000","last_unstale":"2023-11-28T13:01:43.606013+0000","last_undegraded":"2023-11-28T13:01:43.606013+0000","last_fullsized":"2023-11-28T13:01:43.606013+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:25.334954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.600138+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:01:42.600138+0000","last_peered":"2023-11-28T13:01:42.600138+0000","last_clean":"2023-11-28T13:01:42.600138+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:01:42.600138+0000","last_undegraded":"2023-11-28T13:01:42.600138+0000","last_fullsized":"2023-11-28T13:01:42.600138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606054+0000","last_change":"2023-11-28T13:01:42.614706+0000","last_active":"2023-11-28T13:01:43.606054+0000","last_peered":"2023-11-28T13:01:43.606054+0000","last_clean":"2023-11-28T13:01:43.606054+0000","last_became_active":"2023-11-28T13:01:42.614405+0000","last_became_peered":"2023-11-28T13:01:42.614405+0000","last_unstale":"2023-11-28T13:01:43.606054+0000","last_undegraded":"2023-11-28T13:01:43.606054+0000","last_fullsized":"2023-11-28T13:01:43.606054+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:09.795817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.603400+0000","last_change":"2023-11-28T13:01:42.198137+0000","last_active":"2023-11-28T13:01:42.603400+0000","last_peered":"2023-11-28T13:01:42.603400+0000","last_clean":"2023-11-28T13:01:42.603400+0000","last_became_active":"2023-11-28T13:01:42.197851+0000","last_became_peered":"2023-11-28T13:01:42.197851+0000","last_unstale":"2023-11-28T13:01:42.603400+0000","last_undegraded":"2023-11-28T13:01:42.603400+0000","last_fullsized":"2023-11-28T13:01:42.603400+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:11.853093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.185877+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:01:42.185877+0000","last_peered":"2023-11-28T13:01:42.185877+0000","last_clean":"2023-11-28T13:01:42.185877+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:01:42.185877+0000","last_undegraded":"2023-11-28T13:01:42.185877+0000","last_fullsized":"2023-11-28T13:01:42.185877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":183,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.602550+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:01:42.602550+0000","last_peered":"2023-11-28T13:01:42.602550+0000","last_clean":"2023-11-28T13:01:42.602550+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:01:42.602550+0000","last_undegraded":"2023-11-28T13:01:42.602550+0000","last_fullsized":"2023-11-28T13:01:42.602550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652912+0000","last_change":"2023-11-28T13:01:42.613295+0000","last_active":"2023-11-28T13:01:43.652912+0000","last_peered":"2023-11-28T13:01:43.652912+0000","last_clean":"2023-11-28T13:01:43.652912+0000","last_became_active":"2023-11-28T13:01:42.612994+0000","last_became_peered":"2023-11-28T13:01:42.612994+0000","last_unstale":"2023-11-28T13:01:43.652912+0000","last_undegraded":"2023-11-28T13:01:43.652912+0000","last_fullsized":"2023-11-28T13:01:43.652912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:34:29.680397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606188+0000","last_change":"2023-11-28T13:01:42.190891+0000","last_active":"2023-11-28T13:01:43.606188+0000","last_peered":"2023-11-28T13:01:43.606188+0000","last_clean":"2023-11-28T13:01:43.606188+0000","last_became_active":"2023-11-28T13:01:42.190442+0000","last_became_peered":"2023-11-28T13:01:42.190442+0000","last_unstale":"2023-11-28T13:01:43.606188+0000","last_undegraded":"2023-11-28T13:01:43.606188+0000","last_fullsized":"2023-11-28T13:01:43.606188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:06.893878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":34,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.603181+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:01:42.603181+0000","last_peered":"2023-11-28T13:01:42.603181+0000","last_clean":"2023-11-28T13:01:42.603181+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:01:42.603181+0000","last_undegraded":"2023-11-28T13:01:42.603181+0000","last_fullsized":"2023-11-28T13:01:42.603181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.653138+0000","last_change":"2023-11-28T13:01:42.607948+0000","last_active":"2023-11-28T13:01:43.653138+0000","last_peered":"2023-11-28T13:01:43.653138+0000","last_clean":"2023-11-28T13:01:43.653138+0000","last_became_active":"2023-11-28T13:01:42.607542+0000","last_became_peered":"2023-11-28T13:01:42.607542+0000","last_unstale":"2023-11-28T13:01:43.653138+0000","last_undegraded":"2023-11-28T13:01:43.653138+0000","last_fullsized":"2023-11-28T13:01:43.653138+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:03:45.927554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.629922+0000","last_change":"2023-11-28T13:01:42.192294+0000","last_active":"2023-11-28T13:01:42.629922+0000","last_peered":"2023-11-28T13:01:42.629922+0000","last_clean":"2023-11-28T13:01:42.629922+0000","last_became_active":"2023-11-28T13:01:42.191964+0000","last_became_peered":"2023-11-28T13:01:42.191964+0000","last_unstale":"2023-11-28T13:01:42.629922+0000","last_undegraded":"2023-11-28T13:01:42.629922+0000","last_fullsized":"2023-11-28T13:01:42.629922+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:55.803583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.602436+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:01:42.602436+0000","last_peered":"2023-11-28T13:01:42.602436+0000","last_clean":"2023-11-28T13:01:42.602436+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:01:42.602436+0000","last_undegraded":"2023-11-28T13:01:42.602436+0000","last_fullsized":"2023-11-28T13:01:42.602436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.629661+0000","last_change":"2023-11-28T13:01:42.622794+0000","last_active":"2023-11-28T13:01:42.629661+0000","last_peered":"2023-11-28T13:01:42.629661+0000","last_clean":"2023-11-28T13:01:42.629661+0000","last_became_active":"2023-11-28T13:01:42.622530+0000","last_became_peered":"2023-11-28T13:01:42.622530+0000","last_unstale":"2023-11-28T13:01:42.629661+0000","last_undegraded":"2023-11-28T13:01:42.629661+0000","last_fullsized":"2023-11-28T13:01:42.629661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:26.339350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603690+0000","last_change":"2023-11-28T13:01:42.195785+0000","last_active":"2023-11-28T13:01:43.603690+0000","last_peered":"2023-11-28T13:01:43.603690+0000","last_clean":"2023-11-28T13:01:43.603690+0000","last_became_active":"2023-11-28T13:01:42.195490+0000","last_became_peered":"2023-11-28T13:01:42.195490+0000","last_unstale":"2023-11-28T13:01:43.603690+0000","last_undegraded":"2023-11-28T13:01:43.603690+0000","last_fullsized":"2023-11-28T13:01:43.603690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:35:54.450538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.180423+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:01:42.180423+0000","last_peered":"2023-11-28T13:01:42.180423+0000","last_clean":"2023-11-28T13:01:42.180423+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:01:42.180423+0000","last_undegraded":"2023-11-28T13:01:42.180423+0000","last_fullsized":"2023-11-28T13:01:42.180423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627939+0000","last_change":"2023-11-28T13:01:42.625683+0000","last_active":"2023-11-28T13:01:42.627939+0000","last_peered":"2023-11-28T13:01:42.627939+0000","last_clean":"2023-11-28T13:01:42.627939+0000","last_became_active":"2023-11-28T13:01:42.624735+0000","last_became_peered":"2023-11-28T13:01:42.624735+0000","last_unstale":"2023-11-28T13:01:42.627939+0000","last_undegraded":"2023-11-28T13:01:42.627939+0000","last_fullsized":"2023-11-28T13:01:42.627939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:41.092930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.603401+0000","last_change":"2023-11-28T13:01:42.195922+0000","last_active":"2023-11-28T13:01:42.603401+0000","last_peered":"2023-11-28T13:01:42.603401+0000","last_clean":"2023-11-28T13:01:42.603401+0000","last_became_active":"2023-11-28T13:01:42.195650+0000","last_became_peered":"2023-11-28T13:01:42.195650+0000","last_unstale":"2023-11-28T13:01:42.603401+0000","last_undegraded":"2023-11-28T13:01:42.603401+0000","last_fullsized":"2023-11-28T13:01:42.603401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:30:00.065910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.187751+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:01:42.187751+0000","last_peered":"2023-11-28T13:01:42.187751+0000","last_clean":"2023-11-28T13:01:42.187751+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:01:42.187751+0000","last_undegraded":"2023-11-28T13:01:42.187751+0000","last_fullsized":"2023-11-28T13:01:42.187751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733272+0000","last_change":"2023-11-28T13:01:42.624225+0000","last_active":"2023-11-28T13:01:43.733272+0000","last_peered":"2023-11-28T13:01:43.733272+0000","last_clean":"2023-11-28T13:01:43.733272+0000","last_became_active":"2023-11-28T13:01:42.623895+0000","last_became_peered":"2023-11-28T13:01:42.623895+0000","last_unstale":"2023-11-28T13:01:43.733272+0000","last_undegraded":"2023-11-28T13:01:43.733272+0000","last_fullsized":"2023-11-28T13:01:43.733272+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:26.241773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652809+0000","last_change":"2023-11-28T13:01:42.193673+0000","last_active":"2023-11-28T13:01:43.652809+0000","last_peered":"2023-11-28T13:01:43.652809+0000","last_clean":"2023-11-28T13:01:43.652809+0000","last_became_active":"2023-11-28T13:01:42.193390+0000","last_became_peered":"2023-11-28T13:01:42.193390+0000","last_unstale":"2023-11-28T13:01:43.652809+0000","last_undegraded":"2023-11-28T13:01:43.652809+0000","last_fullsized":"2023-11-28T13:01:43.652809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:10.086374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.187796+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:01:42.187796+0000","last_peered":"2023-11-28T13:01:42.187796+0000","last_clean":"2023-11-28T13:01:42.187796+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:01:42.187796+0000","last_undegraded":"2023-11-28T13:01:42.187796+0000","last_fullsized":"2023-11-28T13:01:42.187796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9812,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362028,"statfs":{"total":96636764160,"available":96626716672,"internally_reserved":0,"allocated":1060864,"data_stored":799095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.60899999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.624},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8584,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363256,"statfs":{"total":96636764160,"available":96627974144,"internally_reserved":0,"allocated":458752,"data_stored":202320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.44}]},{"osd":1,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.38600000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.496}]},{"osd":4,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.65400000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.62},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9668,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362172,"statfs":{"total":96636764160,"available":96626864128,"internally_reserved":0,"allocated":1044480,"data_stored":791097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.34899999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.52600000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.64200000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.623}]},{"osd":7,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":462848,"data_stored":207685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.252},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.40100000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.44}]},{"osd":4,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61899999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57899999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8528,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363312,"statfs":{"total":96636764160,"available":96628031488,"internally_reserved":0,"allocated":466944,"data_stored":203234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.34499999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.317},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.28999999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.35499999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64700000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69699999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":462848,"data_stored":202286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.318}]},{"osd":1,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35799999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":5.0540000000000003,"5min":5.0540000000000003,"15min":5.0540000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.72899999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56899999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8728,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363112,"statfs":{"total":96636764160,"available":96627826688,"internally_reserved":0,"allocated":475136,"data_stored":210185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.67200000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41999999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.499}]},{"osd":3,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.69899999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.627}]},{"osd":6,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.752},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.77800000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.81499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":201734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64800000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.63}]},{"osd":2,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73999999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.40899999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.42699999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.48899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:01:46.731 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:01:46.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.887+0000 7f7490b07700 1 Processor -- start 2023-11-28T13:01:46.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.887+0000 7f7490b07700 1 -- start start 2023-11-28T13:01:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.888+0000 7f7490b07700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c06e400 0x7f748c15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.888+0000 7f7490b07700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f748c0ce890 0x7f748c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.888+0000 7f7490b07700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c15e0b0 con 0x7f748c159590 2023-11-28T13:01:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.888+0000 7f7490b07700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c0cf250 con 0x7f748c0ce890 2023-11-28T13:01:46.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.888+0000 7f7490b07700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c15d120 con 0x7f748c06e400 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f7489d9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c06e400 0x7f748c15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748ad9d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f748c0ce890 0x7f748c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748ad9d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f748c0ce890 0x7f748c0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:35978/0 (socket says 172.21.15.6:35978) 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f7489d9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c06e400 0x7f748c15db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:50194/0 (socket says 172.21.15.6:50194) 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748ad9d700 1 -- 172.21.15.6:0/3348374491 learned_addr learned my addr 172.21.15.6:0/3348374491 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748ad9d700 1 -- 172.21.15.6:0/3348374491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c06e400 msgr2=0x7f748c15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748ad9d700 1 --2- 172.21.15.6:0/3348374491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c06e400 0x7f748c15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748ad9d700 1 -- 172.21.15.6:0/3348374491 >> v1:172.21.15.61:6789/0 conn(0x7f748c159590 legacy=0x7f748c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748ad9d700 1 -- 172.21.15.6:0/3348374491 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c15d6e0 con 0x7f748c0ce890 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748959a700 1 -- 172.21.15.6:0/3348374491 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129454050 0 0) 0x7f7480001110 con 0x7f748c159590 2023-11-28T13:01:46.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748ad9d700 1 --2- 172.21.15.6:0/3348374491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f748c0ce890 0x7f748c0ced10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7484009e50 tx=0x7f748400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21fe74b68947c66c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748959a700 1 -- 172.21.15.6:0/3348374491 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400e070 con 0x7f748c0ce890 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748959a700 1 -- 172.21.15.6:0/3348374491 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748400b910 con 0x7f748c0ce890 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.889+0000 7f748959a700 1 -- 172.21.15.6:0/3348374491 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400bcc0 con 0x7f748c0ce890 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.890+0000 7f7490b07700 1 -- 172.21.15.6:0/3348374491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f748c0ce890 msgr2=0x7f748c0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.890+0000 7f7490b07700 1 --2- 172.21.15.6:0/3348374491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f748c0ce890 0x7f748c0ced10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7484009e50 tx=0x7f748400b330 comp rx=0 tx=0).stop 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.890+0000 7f7490b07700 1 -- 172.21.15.6:0/3348374491 shutdown_connections 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.890+0000 7f7490b07700 1 --2- 172.21.15.6:0/3348374491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f748c0ce890 0x7f748c0ced10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.890+0000 7f7490b07700 1 --2- 172.21.15.6:0/3348374491 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c06e400 0x7f748c15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.890+0000 7f7490b07700 1 -- 172.21.15.6:0/3348374491 >> 172.21.15.6:0/3348374491 conn(0x7f748c0b1650 msgr2=0x7f748c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.890+0000 7f7490b07700 1 -- 172.21.15.6:0/3348374491 shutdown_connections 2023-11-28T13:01:46.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.890+0000 7f7490b07700 1 -- 172.21.15.6:0/3348374491 wait complete. 2023-11-28T13:01:46.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.891+0000 7f7490b07700 1 Processor -- start 2023-11-28T13:01:46.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.891+0000 7f7490b07700 1 -- start start 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f7490b07700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f748c06e400 0x7f748c154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f7490b07700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c0ce890 0x7f748c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f7490b07700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c159590 0x7f748c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f7490b07700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c154830 con 0x7f748c159590 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f7490b07700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c15aa50 con 0x7f748c06e400 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f7490b07700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c15ad30 con 0x7f748c0ce890 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f748ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c159590 0x7f748c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f748ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c159590 0x7f748c15b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60498/0 (socket says 172.21.15.6:60498) 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f748ad9d700 1 -- 172.21.15.6:0/3271070449 learned_addr learned my addr 172.21.15.6:0/3271070449 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f7489d9b700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c0ce890 0x7f748c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.892+0000 7f748a59c700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f748c06e400 0x7f748c154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f748ad9d700 1 -- 172.21.15.6:0/3271070449 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c0ce890 msgr2=0x7f748c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f748ad9d700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c0ce890 0x7f748c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f748ad9d700 1 -- 172.21.15.6:0/3271070449 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f748c06e400 msgr2=0x7f748c154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f748ad9d700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f748c06e400 0x7f748c154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f748ad9d700 1 -- 172.21.15.6:0/3271070449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484009b30 con 0x7f748c159590 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f748ad9d700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c159590 0x7f748c15b810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f748400b680 tx=0x7f7484004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f747affd700 1 -- 172.21.15.6:0/3271070449 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74840048b0 con 0x7f748c159590 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f747affd700 1 -- 172.21.15.6:0/3271070449 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74840092e0 con 0x7f748c159590 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f7490b07700 1 -- 172.21.15.6:0/3271070449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c0af240 con 0x7f748c159590 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f747affd700 1 -- 172.21.15.6:0/3271070449 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748401c660 con 0x7f748c159590 2023-11-28T13:01:46.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.893+0000 7f7490b07700 1 -- 172.21.15.6:0/3271070449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c0aef60 con 0x7f748c159590 2023-11-28T13:01:46.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.894+0000 7f7490b07700 1 -- 172.21.15.6:0/3271070449 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c15d370 con 0x7f748c159590 2023-11-28T13:01:46.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.894+0000 7f747affd700 1 -- 172.21.15.6:0/3271070449 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f748402c070 con 0x7f748c159590 2023-11-28T13:01:46.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.895+0000 7f747affd700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f746c0682f0 0x7f746c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:46.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.895+0000 7f747affd700 1 -- 172.21.15.6:0/3271070449 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7484087890 con 0x7f748c159590 2023-11-28T13:01:46.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.895+0000 7f748a59c700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f746c0682f0 0x7f746c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:46.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.895+0000 7f748a59c700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f746c0682f0 0x7f746c06a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7480000f60 tx=0x7f7480009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:46.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:46.899+0000 7f747affd700 1 -- 172.21.15.6:0/3271070449 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7484053fb0 con 0x7f748c159590 2023-11-28T13:01:47.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.087+0000 7f7490b07700 1 -- 172.21.15.6:0/3271070449 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f748c0675a0 con 0x7f746c0682f0 2023-11-28T13:01:47.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.095+0000 7f747affd700 1 -- 172.21.15.6:0/3271070449 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312009 (secure 0 0 0) 0x7f748c0675a0 con 0x7f746c0682f0 2023-11-28T13:01:47.100 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:47.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.098+0000 7f7490b07700 1 -- 172.21.15.6:0/3271070449 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f746c0682f0 msgr2=0x7f746c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.098+0000 7f7490b07700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f746c0682f0 0x7f746c06a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7480000f60 tx=0x7f7480009040 comp rx=0 tx=0).stop 2023-11-28T13:01:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.098+0000 7f7490b07700 1 -- 172.21.15.6:0/3271070449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c159590 msgr2=0x7f748c15b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.098+0000 7f7490b07700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c159590 0x7f748c15b810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f748400b680 tx=0x7f7484004ad0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.098+0000 7f7490b07700 1 -- 172.21.15.6:0/3271070449 shutdown_connections 2023-11-28T13:01:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.098+0000 7f7490b07700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f748c159590 0x7f748c15b810 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.098+0000 7f7490b07700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f748c06e400 0x7f748c154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.098+0000 7f7490b07700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f746c0682f0 0x7f746c06a7a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.098+0000 7f7490b07700 1 --2- 172.21.15.6:0/3271070449 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f748c0ce890 0x7f748c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.098+0000 7f7490b07700 1 -- 172.21.15.6:0/3271070449 >> 172.21.15.6:0/3271070449 conn(0x7f748c0b1650 msgr2=0x7f748c0afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:47.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.099+0000 7f7490b07700 1 -- 172.21.15.6:0/3271070449 shutdown_connections 2023-11-28T13:01:47.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.099+0000 7f7490b07700 1 -- 172.21.15.6:0/3271070449 wait complete. 2023-11-28T13:01:47.104 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:01:47.112 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-11-28T13:01:46.960398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71332,"kb_used_data":4772,"kb_used_omap":0,"kb_used_meta":66496,"kb_avail":754903388,"statfs":{"total":773094113280,"available":773021069312,"internally_reserved":0,"allocated":4886528,"data_stored":2817636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68091904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.793742"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606172+0000","last_change":"2023-11-28T13:01:42.620897+0000","last_active":"2023-11-28T13:01:43.606172+0000","last_peered":"2023-11-28T13:01:43.606172+0000","last_clean":"2023-11-28T13:01:43.606172+0000","last_became_active":"2023-11-28T13:01:42.620606+0000","last_became_peered":"2023-11-28T13:01:42.620606+0000","last_unstale":"2023-11-28T13:01:43.606172+0000","last_undegraded":"2023-11-28T13:01:43.606172+0000","last_fullsized":"2023-11-28T13:01:43.606172+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:08.291193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.632405+0000","last_change":"2023-11-28T13:01:42.627974+0000","last_active":"2023-11-28T13:01:42.632405+0000","last_peered":"2023-11-28T13:01:42.632405+0000","last_clean":"2023-11-28T13:01:42.632405+0000","last_became_active":"2023-11-28T13:01:42.627568+0000","last_became_peered":"2023-11-28T13:01:42.627568+0000","last_unstale":"2023-11-28T13:01:42.632405+0000","last_undegraded":"2023-11-28T13:01:42.632405+0000","last_fullsized":"2023-11-28T13:01:42.632405+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:59:55.382836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733463+0000","last_change":"2023-11-28T13:01:42.622185+0000","last_active":"2023-11-28T13:01:43.733463+0000","last_peered":"2023-11-28T13:01:43.733463+0000","last_clean":"2023-11-28T13:01:43.733463+0000","last_became_active":"2023-11-28T13:01:42.621835+0000","last_became_peered":"2023-11-28T13:01:42.621835+0000","last_unstale":"2023-11-28T13:01:43.733463+0000","last_undegraded":"2023-11-28T13:01:43.733463+0000","last_fullsized":"2023-11-28T13:01:43.733463+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:03.332763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652850+0000","last_change":"2023-11-28T13:01:42.620346+0000","last_active":"2023-11-28T13:01:43.652850+0000","last_peered":"2023-11-28T13:01:43.652850+0000","last_clean":"2023-11-28T13:01:43.652850+0000","last_became_active":"2023-11-28T13:01:42.620099+0000","last_became_peered":"2023-11-28T13:01:42.620099+0000","last_unstale":"2023-11-28T13:01:43.652850+0000","last_undegraded":"2023-11-28T13:01:43.652850+0000","last_fullsized":"2023-11-28T13:01:43.652850+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:08.054061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.629693+0000","last_change":"2023-11-28T13:01:42.626346+0000","last_active":"2023-11-28T13:01:42.629693+0000","last_peered":"2023-11-28T13:01:42.629693+0000","last_clean":"2023-11-28T13:01:42.629693+0000","last_became_active":"2023-11-28T13:01:42.625994+0000","last_became_peered":"2023-11-28T13:01:42.625994+0000","last_unstale":"2023-11-28T13:01:42.629693+0000","last_undegraded":"2023-11-28T13:01:42.629693+0000","last_fullsized":"2023-11-28T13:01:42.629693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:51:16.256643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602827+0000","last_change":"2023-11-28T13:01:42.610186+0000","last_active":"2023-11-28T13:01:43.602827+0000","last_peered":"2023-11-28T13:01:43.602827+0000","last_clean":"2023-11-28T13:01:43.602827+0000","last_became_active":"2023-11-28T13:01:42.609896+0000","last_became_peered":"2023-11-28T13:01:42.609896+0000","last_unstale":"2023-11-28T13:01:43.602827+0000","last_undegraded":"2023-11-28T13:01:43.602827+0000","last_fullsized":"2023-11-28T13:01:43.602827+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:02:43.012209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.605769+0000","last_change":"2023-11-28T13:01:42.613906+0000","last_active":"2023-11-28T13:01:43.605769+0000","last_peered":"2023-11-28T13:01:43.605769+0000","last_clean":"2023-11-28T13:01:43.605769+0000","last_became_active":"2023-11-28T13:01:42.613597+0000","last_became_peered":"2023-11-28T13:01:42.613597+0000","last_unstale":"2023-11-28T13:01:43.605769+0000","last_undegraded":"2023-11-28T13:01:43.605769+0000","last_fullsized":"2023-11-28T13:01:43.605769+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:40.884368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652550+0000","last_change":"2023-11-28T13:01:42.621574+0000","last_active":"2023-11-28T13:01:43.652550+0000","last_peered":"2023-11-28T13:01:43.652550+0000","last_clean":"2023-11-28T13:01:43.652550+0000","last_became_active":"2023-11-28T13:01:42.620644+0000","last_became_peered":"2023-11-28T13:01:42.620644+0000","last_unstale":"2023-11-28T13:01:43.652550+0000","last_undegraded":"2023-11-28T13:01:43.652550+0000","last_fullsized":"2023-11-28T13:01:43.652550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:58.604562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652933+0000","last_change":"2023-11-28T13:01:42.615811+0000","last_active":"2023-11-28T13:01:43.652933+0000","last_peered":"2023-11-28T13:01:43.652933+0000","last_clean":"2023-11-28T13:01:43.652933+0000","last_became_active":"2023-11-28T13:01:42.615553+0000","last_became_peered":"2023-11-28T13:01:42.615553+0000","last_unstale":"2023-11-28T13:01:43.652933+0000","last_undegraded":"2023-11-28T13:01:43.652933+0000","last_fullsized":"2023-11-28T13:01:43.652933+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:07:49.751548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.653097+0000","last_change":"2023-11-28T13:01:42.607788+0000","last_active":"2023-11-28T13:01:43.653097+0000","last_peered":"2023-11-28T13:01:43.653097+0000","last_clean":"2023-11-28T13:01:43.653097+0000","last_became_active":"2023-11-28T13:01:42.607188+0000","last_became_peered":"2023-11-28T13:01:42.607188+0000","last_unstale":"2023-11-28T13:01:43.653097+0000","last_undegraded":"2023-11-28T13:01:43.653097+0000","last_fullsized":"2023-11-28T13:01:43.653097+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:25:04.329423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652894+0000","last_change":"2023-11-28T13:01:42.619702+0000","last_active":"2023-11-28T13:01:43.652894+0000","last_peered":"2023-11-28T13:01:43.652894+0000","last_clean":"2023-11-28T13:01:43.652894+0000","last_became_active":"2023-11-28T13:01:42.619410+0000","last_became_peered":"2023-11-28T13:01:42.619410+0000","last_unstale":"2023-11-28T13:01:43.652894+0000","last_undegraded":"2023-11-28T13:01:43.652894+0000","last_fullsized":"2023-11-28T13:01:43.652894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:55.538689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627748+0000","last_change":"2023-11-28T13:01:42.623415+0000","last_active":"2023-11-28T13:01:42.627748+0000","last_peered":"2023-11-28T13:01:42.627748+0000","last_clean":"2023-11-28T13:01:42.627748+0000","last_became_active":"2023-11-28T13:01:42.623094+0000","last_became_peered":"2023-11-28T13:01:42.623094+0000","last_unstale":"2023-11-28T13:01:42.627748+0000","last_undegraded":"2023-11-28T13:01:42.627748+0000","last_fullsized":"2023-11-28T13:01:42.627748+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:40:43.658059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603224+0000","last_change":"2023-11-28T13:01:42.624892+0000","last_active":"2023-11-28T13:01:43.603224+0000","last_peered":"2023-11-28T13:01:43.603224+0000","last_clean":"2023-11-28T13:01:43.603224+0000","last_became_active":"2023-11-28T13:01:42.624667+0000","last_became_peered":"2023-11-28T13:01:42.624667+0000","last_unstale":"2023-11-28T13:01:43.603224+0000","last_undegraded":"2023-11-28T13:01:43.603224+0000","last_fullsized":"2023-11-28T13:01:43.603224+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:34.061420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652763+0000","last_change":"2023-11-28T13:01:42.619651+0000","last_active":"2023-11-28T13:01:43.652763+0000","last_peered":"2023-11-28T13:01:43.652763+0000","last_clean":"2023-11-28T13:01:43.652763+0000","last_became_active":"2023-11-28T13:01:42.619398+0000","last_became_peered":"2023-11-28T13:01:42.619398+0000","last_unstale":"2023-11-28T13:01:43.652763+0000","last_undegraded":"2023-11-28T13:01:43.652763+0000","last_fullsized":"2023-11-28T13:01:43.652763+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:08.655229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602628+0000","last_change":"2023-11-28T13:01:42.614356+0000","last_active":"2023-11-28T13:01:43.602628+0000","last_peered":"2023-11-28T13:01:43.602628+0000","last_clean":"2023-11-28T13:01:43.602628+0000","last_became_active":"2023-11-28T13:01:42.614064+0000","last_became_peered":"2023-11-28T13:01:42.614064+0000","last_unstale":"2023-11-28T13:01:43.602628+0000","last_undegraded":"2023-11-28T13:01:43.602628+0000","last_fullsized":"2023-11-28T13:01:43.602628+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:04:08.054061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627142+0000","last_change":"2023-11-28T13:01:42.618032+0000","last_active":"2023-11-28T13:01:42.627142+0000","last_peered":"2023-11-28T13:01:42.627142+0000","last_clean":"2023-11-28T13:01:42.627142+0000","last_became_active":"2023-11-28T13:01:42.617526+0000","last_became_peered":"2023-11-28T13:01:42.617526+0000","last_unstale":"2023-11-28T13:01:42.627142+0000","last_undegraded":"2023-11-28T13:01:42.627142+0000","last_fullsized":"2023-11-28T13:01:42.627142+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:07.898274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733909+0000","last_change":"2023-11-28T13:01:42.622381+0000","last_active":"2023-11-28T13:01:43.733909+0000","last_peered":"2023-11-28T13:01:43.733909+0000","last_clean":"2023-11-28T13:01:43.733909+0000","last_became_active":"2023-11-28T13:01:42.622014+0000","last_became_peered":"2023-11-28T13:01:42.622014+0000","last_unstale":"2023-11-28T13:01:43.733909+0000","last_undegraded":"2023-11-28T13:01:43.733909+0000","last_fullsized":"2023-11-28T13:01:43.733909+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:07.815817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733811+0000","last_change":"2023-11-28T13:01:42.625656+0000","last_active":"2023-11-28T13:01:43.733811+0000","last_peered":"2023-11-28T13:01:43.733811+0000","last_clean":"2023-11-28T13:01:43.733811+0000","last_became_active":"2023-11-28T13:01:42.625335+0000","last_became_peered":"2023-11-28T13:01:42.625335+0000","last_unstale":"2023-11-28T13:01:43.733811+0000","last_undegraded":"2023-11-28T13:01:43.733811+0000","last_fullsized":"2023-11-28T13:01:43.733811+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:33.547445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.616609+0000","last_change":"2023-11-28T13:01:42.610597+0000","last_active":"2023-11-28T13:01:42.616609+0000","last_peered":"2023-11-28T13:01:42.616609+0000","last_clean":"2023-11-28T13:01:42.616609+0000","last_became_active":"2023-11-28T13:01:42.610350+0000","last_became_peered":"2023-11-28T13:01:42.610350+0000","last_unstale":"2023-11-28T13:01:42.616609+0000","last_undegraded":"2023-11-28T13:01:42.616609+0000","last_fullsized":"2023-11-28T13:01:42.616609+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:20.903612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602964+0000","last_change":"2023-11-28T13:01:42.622745+0000","last_active":"2023-11-28T13:01:43.602964+0000","last_peered":"2023-11-28T13:01:43.602964+0000","last_clean":"2023-11-28T13:01:43.602964+0000","last_became_active":"2023-11-28T13:01:42.622386+0000","last_became_peered":"2023-11-28T13:01:42.622386+0000","last_unstale":"2023-11-28T13:01:43.602964+0000","last_undegraded":"2023-11-28T13:01:43.602964+0000","last_fullsized":"2023-11-28T13:01:43.602964+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:48:52.405752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603017+0000","last_change":"2023-11-28T13:01:42.610652+0000","last_active":"2023-11-28T13:01:43.603017+0000","last_peered":"2023-11-28T13:01:43.603017+0000","last_clean":"2023-11-28T13:01:43.603017+0000","last_became_active":"2023-11-28T13:01:42.610321+0000","last_became_peered":"2023-11-28T13:01:42.610321+0000","last_unstale":"2023-11-28T13:01:43.603017+0000","last_undegraded":"2023-11-28T13:01:43.603017+0000","last_fullsized":"2023-11-28T13:01:43.603017+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:37.363198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652753+0000","last_change":"2023-11-28T13:01:42.620156+0000","last_active":"2023-11-28T13:01:43.652753+0000","last_peered":"2023-11-28T13:01:43.652753+0000","last_clean":"2023-11-28T13:01:43.652753+0000","last_became_active":"2023-11-28T13:01:42.619906+0000","last_became_peered":"2023-11-28T13:01:42.619906+0000","last_unstale":"2023-11-28T13:01:43.652753+0000","last_undegraded":"2023-11-28T13:01:43.652753+0000","last_fullsized":"2023-11-28T13:01:43.652753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:12:40.189110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.614414+0000","last_change":"2023-11-28T13:01:42.607938+0000","last_active":"2023-11-28T13:01:42.614414+0000","last_peered":"2023-11-28T13:01:42.614414+0000","last_clean":"2023-11-28T13:01:42.614414+0000","last_became_active":"2023-11-28T13:01:42.607560+0000","last_became_peered":"2023-11-28T13:01:42.607560+0000","last_unstale":"2023-11-28T13:01:42.614414+0000","last_undegraded":"2023-11-28T13:01:42.614414+0000","last_fullsized":"2023-11-28T13:01:42.614414+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:27:15.550145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.614790+0000","last_change":"2023-11-28T13:01:42.607375+0000","last_active":"2023-11-28T13:01:42.614790+0000","last_peered":"2023-11-28T13:01:42.614790+0000","last_clean":"2023-11-28T13:01:42.614790+0000","last_became_active":"2023-11-28T13:01:42.607116+0000","last_became_peered":"2023-11-28T13:01:42.607116+0000","last_unstale":"2023-11-28T13:01:42.614790+0000","last_undegraded":"2023-11-28T13:01:42.614790+0000","last_fullsized":"2023-11-28T13:01:42.614790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:47.081647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.618123+0000","last_change":"2023-11-28T13:01:42.613296+0000","last_active":"2023-11-28T13:01:42.618123+0000","last_peered":"2023-11-28T13:01:42.618123+0000","last_clean":"2023-11-28T13:01:42.618123+0000","last_became_active":"2023-11-28T13:01:42.612988+0000","last_became_peered":"2023-11-28T13:01:42.612988+0000","last_unstale":"2023-11-28T13:01:42.618123+0000","last_undegraded":"2023-11-28T13:01:42.618123+0000","last_fullsized":"2023-11-28T13:01:42.618123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:46.649699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603770+0000","last_change":"2023-11-28T13:01:42.624183+0000","last_active":"2023-11-28T13:01:43.603770+0000","last_peered":"2023-11-28T13:01:43.603770+0000","last_clean":"2023-11-28T13:01:43.603770+0000","last_became_active":"2023-11-28T13:01:42.623920+0000","last_became_peered":"2023-11-28T13:01:42.623920+0000","last_unstale":"2023-11-28T13:01:43.603770+0000","last_undegraded":"2023-11-28T13:01:43.603770+0000","last_fullsized":"2023-11-28T13:01:43.603770+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:47.081647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733461+0000","last_change":"2023-11-28T13:01:42.621240+0000","last_active":"2023-11-28T13:01:43.733461+0000","last_peered":"2023-11-28T13:01:43.733461+0000","last_clean":"2023-11-28T13:01:43.733461+0000","last_became_active":"2023-11-28T13:01:42.620939+0000","last_became_peered":"2023-11-28T13:01:42.620939+0000","last_unstale":"2023-11-28T13:01:43.733461+0000","last_undegraded":"2023-11-28T13:01:43.733461+0000","last_fullsized":"2023-11-28T13:01:43.733461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:03:39.615987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627773+0000","last_change":"2023-11-28T13:01:42.623446+0000","last_active":"2023-11-28T13:01:42.627773+0000","last_peered":"2023-11-28T13:01:42.627773+0000","last_clean":"2023-11-28T13:01:42.627773+0000","last_became_active":"2023-11-28T13:01:42.623114+0000","last_became_peered":"2023-11-28T13:01:42.623114+0000","last_unstale":"2023-11-28T13:01:42.627773+0000","last_undegraded":"2023-11-28T13:01:42.627773+0000","last_fullsized":"2023-11-28T13:01:42.627773+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:01:42.963433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.617151+0000","last_change":"2023-11-28T13:01:42.613236+0000","last_active":"2023-11-28T13:01:42.617151+0000","last_peered":"2023-11-28T13:01:42.617151+0000","last_clean":"2023-11-28T13:01:42.617151+0000","last_became_active":"2023-11-28T13:01:42.612892+0000","last_became_peered":"2023-11-28T13:01:42.612892+0000","last_unstale":"2023-11-28T13:01:42.617151+0000","last_undegraded":"2023-11-28T13:01:42.617151+0000","last_fullsized":"2023-11-28T13:01:42.617151+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:41:17.278714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733295+0000","last_change":"2023-11-28T13:01:42.626247+0000","last_active":"2023-11-28T13:01:43.733295+0000","last_peered":"2023-11-28T13:01:43.733295+0000","last_clean":"2023-11-28T13:01:43.733295+0000","last_became_active":"2023-11-28T13:01:42.625971+0000","last_became_peered":"2023-11-28T13:01:42.625971+0000","last_unstale":"2023-11-28T13:01:43.733295+0000","last_undegraded":"2023-11-28T13:01:43.733295+0000","last_fullsized":"2023-11-28T13:01:43.733295+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:41:23.267974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602658+0000","last_change":"2023-11-28T13:01:42.625666+0000","last_active":"2023-11-28T13:01:43.602658+0000","last_peered":"2023-11-28T13:01:43.602658+0000","last_clean":"2023-11-28T13:01:43.602658+0000","last_became_active":"2023-11-28T13:01:42.625263+0000","last_became_peered":"2023-11-28T13:01:42.625263+0000","last_unstale":"2023-11-28T13:01:43.602658+0000","last_undegraded":"2023-11-28T13:01:43.602658+0000","last_fullsized":"2023-11-28T13:01:43.602658+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:36:08.655229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602796+0000","last_change":"2023-11-28T13:01:42.615901+0000","last_active":"2023-11-28T13:01:43.602796+0000","last_peered":"2023-11-28T13:01:43.602796+0000","last_clean":"2023-11-28T13:01:43.602796+0000","last_became_active":"2023-11-28T13:01:42.615634+0000","last_became_peered":"2023-11-28T13:01:42.615634+0000","last_unstale":"2023-11-28T13:01:43.602796+0000","last_undegraded":"2023-11-28T13:01:43.602796+0000","last_fullsized":"2023-11-28T13:01:43.602796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:14:05.525990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.615240+0000","last_change":"2023-11-28T13:01:42.612785+0000","last_active":"2023-11-28T13:01:42.615240+0000","last_peered":"2023-11-28T13:01:42.615240+0000","last_clean":"2023-11-28T13:01:42.615240+0000","last_became_active":"2023-11-28T13:01:42.612519+0000","last_became_peered":"2023-11-28T13:01:42.612519+0000","last_unstale":"2023-11-28T13:01:42.615240+0000","last_undegraded":"2023-11-28T13:01:42.615240+0000","last_fullsized":"2023-11-28T13:01:42.615240+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:11.090771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"39'4","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603585+0000","last_change":"2023-11-28T13:01:42.191064+0000","last_active":"2023-11-28T13:01:43.603585+0000","last_peered":"2023-11-28T13:01:43.603585+0000","last_clean":"2023-11-28T13:01:43.603585+0000","last_became_active":"2023-11-28T13:01:42.190617+0000","last_became_peered":"2023-11-28T13:01:42.190617+0000","last_unstale":"2023-11-28T13:01:43.603585+0000","last_undegraded":"2023-11-28T13:01:43.603585+0000","last_fullsized":"2023-11-28T13:01:43.603585+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:38:22.687980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627111+0000","last_change":"2023-11-28T13:01:42.618889+0000","last_active":"2023-11-28T13:01:42.627111+0000","last_peered":"2023-11-28T13:01:42.627111+0000","last_clean":"2023-11-28T13:01:42.627111+0000","last_became_active":"2023-11-28T13:01:42.618661+0000","last_became_peered":"2023-11-28T13:01:42.618661+0000","last_unstale":"2023-11-28T13:01:42.627111+0000","last_undegraded":"2023-11-28T13:01:42.627111+0000","last_fullsized":"2023-11-28T13:01:42.627111+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:30.356407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733820+0000","last_change":"2023-11-28T13:01:42.196934+0000","last_active":"2023-11-28T13:01:43.733820+0000","last_peered":"2023-11-28T13:01:43.733820+0000","last_clean":"2023-11-28T13:01:43.733820+0000","last_became_active":"2023-11-28T13:01:42.196626+0000","last_became_peered":"2023-11-28T13:01:42.196626+0000","last_unstale":"2023-11-28T13:01:43.733820+0000","last_undegraded":"2023-11-28T13:01:43.733820+0000","last_fullsized":"2023-11-28T13:01:43.733820+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:10.421372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.629734+0000","last_change":"2023-11-28T13:01:42.627323+0000","last_active":"2023-11-28T13:01:42.629734+0000","last_peered":"2023-11-28T13:01:42.629734+0000","last_clean":"2023-11-28T13:01:42.629734+0000","last_became_active":"2023-11-28T13:01:42.627034+0000","last_became_peered":"2023-11-28T13:01:42.627034+0000","last_unstale":"2023-11-28T13:01:42.629734+0000","last_undegraded":"2023-11-28T13:01:42.629734+0000","last_fullsized":"2023-11-28T13:01:42.629734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:39:01.198843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.626279+0000","last_change":"2023-11-28T13:01:42.198461+0000","last_active":"2023-11-28T13:01:42.626279+0000","last_peered":"2023-11-28T13:01:42.626279+0000","last_clean":"2023-11-28T13:01:42.626279+0000","last_became_active":"2023-11-28T13:01:42.198213+0000","last_became_peered":"2023-11-28T13:01:42.198213+0000","last_unstale":"2023-11-28T13:01:42.626279+0000","last_undegraded":"2023-11-28T13:01:42.626279+0000","last_fullsized":"2023-11-28T13:01:42.626279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:51:31.551094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.625937+0000","last_change":"2023-11-28T13:01:42.616223+0000","last_active":"2023-11-28T13:01:42.625937+0000","last_peered":"2023-11-28T13:01:42.625937+0000","last_clean":"2023-11-28T13:01:42.625937+0000","last_became_active":"2023-11-28T13:01:42.615686+0000","last_became_peered":"2023-11-28T13:01:42.615686+0000","last_unstale":"2023-11-28T13:01:42.625937+0000","last_undegraded":"2023-11-28T13:01:42.625937+0000","last_fullsized":"2023-11-28T13:01:42.625937+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:29.410609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606004+0000","last_change":"2023-11-28T13:01:42.198227+0000","last_active":"2023-11-28T13:01:43.606004+0000","last_peered":"2023-11-28T13:01:43.606004+0000","last_clean":"2023-11-28T13:01:43.606004+0000","last_became_active":"2023-11-28T13:01:42.197941+0000","last_became_peered":"2023-11-28T13:01:42.197941+0000","last_unstale":"2023-11-28T13:01:43.606004+0000","last_undegraded":"2023-11-28T13:01:43.606004+0000","last_fullsized":"2023-11-28T13:01:43.606004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:45.645303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733324+0000","last_change":"2023-11-28T13:01:42.625191+0000","last_active":"2023-11-28T13:01:43.733324+0000","last_peered":"2023-11-28T13:01:43.733324+0000","last_clean":"2023-11-28T13:01:43.733324+0000","last_became_active":"2023-11-28T13:01:42.624441+0000","last_became_peered":"2023-11-28T13:01:42.624441+0000","last_unstale":"2023-11-28T13:01:43.733324+0000","last_undegraded":"2023-11-28T13:01:43.733324+0000","last_fullsized":"2023-11-28T13:01:43.733324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:35.576766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603113+0000","last_change":"2023-11-28T13:01:42.187433+0000","last_active":"2023-11-28T13:01:43.603113+0000","last_peered":"2023-11-28T13:01:43.603113+0000","last_clean":"2023-11-28T13:01:43.603113+0000","last_became_active":"2023-11-28T13:01:42.187145+0000","last_became_peered":"2023-11-28T13:01:42.187145+0000","last_unstale":"2023-11-28T13:01:43.603113+0000","last_undegraded":"2023-11-28T13:01:43.603113+0000","last_fullsized":"2023-11-28T13:01:43.603113+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:10:17.338814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652997+0000","last_change":"2023-11-28T13:01:42.616703+0000","last_active":"2023-11-28T13:01:43.652997+0000","last_peered":"2023-11-28T13:01:43.652997+0000","last_clean":"2023-11-28T13:01:43.652997+0000","last_became_active":"2023-11-28T13:01:42.615202+0000","last_became_peered":"2023-11-28T13:01:42.615202+0000","last_unstale":"2023-11-28T13:01:43.652997+0000","last_undegraded":"2023-11-28T13:01:43.652997+0000","last_fullsized":"2023-11-28T13:01:43.652997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:24.091603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733431+0000","last_change":"2023-11-28T13:01:42.199326+0000","last_active":"2023-11-28T13:01:43.733431+0000","last_peered":"2023-11-28T13:01:43.733431+0000","last_clean":"2023-11-28T13:01:43.733431+0000","last_became_active":"2023-11-28T13:01:42.198914+0000","last_became_peered":"2023-11-28T13:01:42.198914+0000","last_unstale":"2023-11-28T13:01:43.733431+0000","last_undegraded":"2023-11-28T13:01:43.733431+0000","last_fullsized":"2023-11-28T13:01:43.733431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:08.791421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652708+0000","last_change":"2023-11-28T13:01:42.609755+0000","last_active":"2023-11-28T13:01:43.652708+0000","last_peered":"2023-11-28T13:01:43.652708+0000","last_clean":"2023-11-28T13:01:43.652708+0000","last_became_active":"2023-11-28T13:01:42.609504+0000","last_became_peered":"2023-11-28T13:01:42.609504+0000","last_unstale":"2023-11-28T13:01:43.652708+0000","last_undegraded":"2023-11-28T13:01:43.652708+0000","last_fullsized":"2023-11-28T13:01:43.652708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:02:43.012209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.614390+0000","last_change":"2023-11-28T13:01:42.186635+0000","last_active":"2023-11-28T13:01:42.614390+0000","last_peered":"2023-11-28T13:01:42.614390+0000","last_clean":"2023-11-28T13:01:42.614390+0000","last_became_active":"2023-11-28T13:01:42.186328+0000","last_became_peered":"2023-11-28T13:01:42.186328+0000","last_unstale":"2023-11-28T13:01:42.614390+0000","last_undegraded":"2023-11-28T13:01:42.614390+0000","last_fullsized":"2023-11-28T13:01:42.614390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:46:29.352010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.614380+0000","last_change":"2023-11-28T13:01:42.607459+0000","last_active":"2023-11-28T13:01:42.614380+0000","last_peered":"2023-11-28T13:01:42.614380+0000","last_clean":"2023-11-28T13:01:42.614380+0000","last_became_active":"2023-11-28T13:01:42.607104+0000","last_became_peered":"2023-11-28T13:01:42.607104+0000","last_unstale":"2023-11-28T13:01:42.614380+0000","last_undegraded":"2023-11-28T13:01:42.614380+0000","last_fullsized":"2023-11-28T13:01:42.614380+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:30:34.061420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652645+0000","last_change":"2023-11-28T13:01:42.189399+0000","last_active":"2023-11-28T13:01:43.652645+0000","last_peered":"2023-11-28T13:01:43.652645+0000","last_clean":"2023-11-28T13:01:43.652645+0000","last_became_active":"2023-11-28T13:01:42.189135+0000","last_became_peered":"2023-11-28T13:01:42.189135+0000","last_unstale":"2023-11-28T13:01:43.652645+0000","last_undegraded":"2023-11-28T13:01:43.652645+0000","last_fullsized":"2023-11-28T13:01:43.652645+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:29.267568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733949+0000","last_change":"2023-11-28T13:01:42.621702+0000","last_active":"2023-11-28T13:01:43.733949+0000","last_peered":"2023-11-28T13:01:43.733949+0000","last_clean":"2023-11-28T13:01:43.733949+0000","last_became_active":"2023-11-28T13:01:42.621419+0000","last_became_peered":"2023-11-28T13:01:42.621419+0000","last_unstale":"2023-11-28T13:01:43.733949+0000","last_undegraded":"2023-11-28T13:01:43.733949+0000","last_fullsized":"2023-11-28T13:01:43.733949+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:11:19.749490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733983+0000","last_change":"2023-11-28T13:01:42.193953+0000","last_active":"2023-11-28T13:01:43.733983+0000","last_peered":"2023-11-28T13:01:43.733983+0000","last_clean":"2023-11-28T13:01:43.733983+0000","last_became_active":"2023-11-28T13:01:42.193687+0000","last_became_peered":"2023-11-28T13:01:42.193687+0000","last_unstale":"2023-11-28T13:01:43.733983+0000","last_undegraded":"2023-11-28T13:01:43.733983+0000","last_fullsized":"2023-11-28T13:01:43.733983+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:54.891996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606239+0000","last_change":"2023-11-28T13:01:42.621535+0000","last_active":"2023-11-28T13:01:43.606239+0000","last_peered":"2023-11-28T13:01:43.606239+0000","last_clean":"2023-11-28T13:01:43.606239+0000","last_became_active":"2023-11-28T13:01:42.621233+0000","last_became_peered":"2023-11-28T13:01:42.621233+0000","last_unstale":"2023-11-28T13:01:43.606239+0000","last_undegraded":"2023-11-28T13:01:43.606239+0000","last_fullsized":"2023-11-28T13:01:43.606239+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:45:49.367244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.653141+0000","last_change":"2023-11-28T13:01:42.191414+0000","last_active":"2023-11-28T13:01:43.653141+0000","last_peered":"2023-11-28T13:01:43.653141+0000","last_clean":"2023-11-28T13:01:43.653141+0000","last_became_active":"2023-11-28T13:01:42.191040+0000","last_became_peered":"2023-11-28T13:01:42.191040+0000","last_unstale":"2023-11-28T13:01:43.653141+0000","last_undegraded":"2023-11-28T13:01:43.653141+0000","last_fullsized":"2023-11-28T13:01:43.653141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:00:38.246885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603448+0000","last_change":"2023-11-28T13:01:42.625984+0000","last_active":"2023-11-28T13:01:43.603448+0000","last_peered":"2023-11-28T13:01:43.603448+0000","last_clean":"2023-11-28T13:01:43.603448+0000","last_became_active":"2023-11-28T13:01:42.625744+0000","last_became_peered":"2023-11-28T13:01:42.625744+0000","last_unstale":"2023-11-28T13:01:43.603448+0000","last_undegraded":"2023-11-28T13:01:43.603448+0000","last_fullsized":"2023-11-28T13:01:43.603448+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:34:29.680397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"39'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606196+0000","last_change":"2023-11-28T13:01:42.193850+0000","last_active":"2023-11-28T13:01:43.606196+0000","last_peered":"2023-11-28T13:01:43.606196+0000","last_clean":"2023-11-28T13:01:43.606196+0000","last_became_active":"2023-11-28T13:01:42.193484+0000","last_became_peered":"2023-11-28T13:01:42.193484+0000","last_unstale":"2023-11-28T13:01:43.606196+0000","last_undegraded":"2023-11-28T13:01:43.606196+0000","last_fullsized":"2023-11-28T13:01:43.606196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:34.635440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606030+0000","last_change":"2023-11-28T13:01:42.617068+0000","last_active":"2023-11-28T13:01:43.606030+0000","last_peered":"2023-11-28T13:01:43.606030+0000","last_clean":"2023-11-28T13:01:43.606030+0000","last_became_active":"2023-11-28T13:01:42.616795+0000","last_became_peered":"2023-11-28T13:01:42.616795+0000","last_unstale":"2023-11-28T13:01:43.606030+0000","last_undegraded":"2023-11-28T13:01:43.606030+0000","last_fullsized":"2023-11-28T13:01:43.606030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:11.425769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"39'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603128+0000","last_change":"2023-11-28T13:01:42.194421+0000","last_active":"2023-11-28T13:01:43.603128+0000","last_peered":"2023-11-28T13:01:43.603128+0000","last_clean":"2023-11-28T13:01:43.603128+0000","last_became_active":"2023-11-28T13:01:42.194077+0000","last_became_peered":"2023-11-28T13:01:42.194077+0000","last_unstale":"2023-11-28T13:01:43.603128+0000","last_undegraded":"2023-11-28T13:01:43.603128+0000","last_fullsized":"2023-11-28T13:01:43.603128+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:27.262805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.616984+0000","last_change":"2023-11-28T13:01:42.610042+0000","last_active":"2023-11-28T13:01:42.616984+0000","last_peered":"2023-11-28T13:01:42.616984+0000","last_clean":"2023-11-28T13:01:42.616984+0000","last_became_active":"2023-11-28T13:01:42.609715+0000","last_became_peered":"2023-11-28T13:01:42.609715+0000","last_unstale":"2023-11-28T13:01:42.616984+0000","last_undegraded":"2023-11-28T13:01:42.616984+0000","last_fullsized":"2023-11-28T13:01:42.616984+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:56.807979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603295+0000","last_change":"2023-11-28T13:01:42.193660+0000","last_active":"2023-11-28T13:01:43.603295+0000","last_peered":"2023-11-28T13:01:43.603295+0000","last_clean":"2023-11-28T13:01:43.603295+0000","last_became_active":"2023-11-28T13:01:42.193379+0000","last_became_peered":"2023-11-28T13:01:42.193379+0000","last_unstale":"2023-11-28T13:01:43.603295+0000","last_undegraded":"2023-11-28T13:01:43.603295+0000","last_fullsized":"2023-11-28T13:01:43.603295+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:50:47.996723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.653034+0000","last_change":"2023-11-28T13:01:42.618330+0000","last_active":"2023-11-28T13:01:43.653034+0000","last_peered":"2023-11-28T13:01:43.653034+0000","last_clean":"2023-11-28T13:01:43.653034+0000","last_became_active":"2023-11-28T13:01:42.617153+0000","last_became_peered":"2023-11-28T13:01:42.617153+0000","last_unstale":"2023-11-28T13:01:43.653034+0000","last_undegraded":"2023-11-28T13:01:43.653034+0000","last_fullsized":"2023-11-28T13:01:43.653034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:16.224085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.605920+0000","last_change":"2023-11-28T13:01:42.185588+0000","last_active":"2023-11-28T13:01:43.605920+0000","last_peered":"2023-11-28T13:01:43.605920+0000","last_clean":"2023-11-28T13:01:43.605920+0000","last_became_active":"2023-11-28T13:01:42.185273+0000","last_became_peered":"2023-11-28T13:01:42.185273+0000","last_unstale":"2023-11-28T13:01:43.605920+0000","last_undegraded":"2023-11-28T13:01:43.605920+0000","last_fullsized":"2023-11-28T13:01:43.605920+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:50.497243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602548+0000","last_change":"2023-11-28T13:01:42.620480+0000","last_active":"2023-11-28T13:01:43.602548+0000","last_peered":"2023-11-28T13:01:43.602548+0000","last_clean":"2023-11-28T13:01:43.602548+0000","last_became_active":"2023-11-28T13:01:42.620164+0000","last_became_peered":"2023-11-28T13:01:42.620164+0000","last_unstale":"2023-11-28T13:01:43.602548+0000","last_undegraded":"2023-11-28T13:01:43.602548+0000","last_fullsized":"2023-11-28T13:01:43.602548+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:40:16.224085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602741+0000","last_change":"2023-11-28T13:01:42.193694+0000","last_active":"2023-11-28T13:01:43.602741+0000","last_peered":"2023-11-28T13:01:43.602741+0000","last_clean":"2023-11-28T13:01:43.602741+0000","last_became_active":"2023-11-28T13:01:42.193454+0000","last_became_peered":"2023-11-28T13:01:42.193454+0000","last_unstale":"2023-11-28T13:01:43.602741+0000","last_undegraded":"2023-11-28T13:01:43.602741+0000","last_fullsized":"2023-11-28T13:01:43.602741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:38:36.902601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.605918+0000","last_change":"2023-11-28T13:01:42.606829+0000","last_active":"2023-11-28T13:01:43.605918+0000","last_peered":"2023-11-28T13:01:43.605918+0000","last_clean":"2023-11-28T13:01:43.605918+0000","last_became_active":"2023-11-28T13:01:42.606524+0000","last_became_peered":"2023-11-28T13:01:42.606524+0000","last_unstale":"2023-11-28T13:01:43.605918+0000","last_undegraded":"2023-11-28T13:01:43.605918+0000","last_fullsized":"2023-11-28T13:01:43.605918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:03:55.896392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"39'8","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652750+0000","last_change":"2023-11-28T13:01:42.199301+0000","last_active":"2023-11-28T13:01:43.652750+0000","last_peered":"2023-11-28T13:01:43.652750+0000","last_clean":"2023-11-28T13:01:43.652750+0000","last_became_active":"2023-11-28T13:01:42.198993+0000","last_became_peered":"2023-11-28T13:01:42.198993+0000","last_unstale":"2023-11-28T13:01:43.652750+0000","last_undegraded":"2023-11-28T13:01:43.652750+0000","last_fullsized":"2023-11-28T13:01:43.652750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:31:22.563114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603823+0000","last_change":"2023-11-28T13:01:42.622012+0000","last_active":"2023-11-28T13:01:43.603823+0000","last_peered":"2023-11-28T13:01:43.603823+0000","last_clean":"2023-11-28T13:01:43.603823+0000","last_became_active":"2023-11-28T13:01:42.621799+0000","last_became_peered":"2023-11-28T13:01:42.621799+0000","last_unstale":"2023-11-28T13:01:43.603823+0000","last_undegraded":"2023-11-28T13:01:43.603823+0000","last_fullsized":"2023-11-28T13:01:43.603823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:25:13.025584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603281+0000","last_change":"2023-11-28T13:01:42.193639+0000","last_active":"2023-11-28T13:01:43.603281+0000","last_peered":"2023-11-28T13:01:43.603281+0000","last_clean":"2023-11-28T13:01:43.603281+0000","last_became_active":"2023-11-28T13:01:42.193379+0000","last_became_peered":"2023-11-28T13:01:42.193379+0000","last_unstale":"2023-11-28T13:01:43.603281+0000","last_undegraded":"2023-11-28T13:01:43.603281+0000","last_fullsized":"2023-11-28T13:01:43.603281+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:57:03.220754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652989+0000","last_change":"2023-11-28T13:01:42.612684+0000","last_active":"2023-11-28T13:01:43.652989+0000","last_peered":"2023-11-28T13:01:43.652989+0000","last_clean":"2023-11-28T13:01:43.652989+0000","last_became_active":"2023-11-28T13:01:42.611096+0000","last_became_peered":"2023-11-28T13:01:42.611096+0000","last_unstale":"2023-11-28T13:01:43.652989+0000","last_undegraded":"2023-11-28T13:01:43.652989+0000","last_fullsized":"2023-11-28T13:01:43.652989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:17:22.271937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.734000+0000","last_change":"2023-11-28T13:01:42.194635+0000","last_active":"2023-11-28T13:01:43.734000+0000","last_peered":"2023-11-28T13:01:43.734000+0000","last_clean":"2023-11-28T13:01:43.734000+0000","last_became_active":"2023-11-28T13:01:42.194326+0000","last_became_peered":"2023-11-28T13:01:42.194326+0000","last_unstale":"2023-11-28T13:01:43.734000+0000","last_undegraded":"2023-11-28T13:01:43.734000+0000","last_fullsized":"2023-11-28T13:01:43.734000+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:07.286796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.626910+0000","last_change":"2023-11-28T13:01:42.617512+0000","last_active":"2023-11-28T13:01:42.626910+0000","last_peered":"2023-11-28T13:01:42.626910+0000","last_clean":"2023-11-28T13:01:42.626910+0000","last_became_active":"2023-11-28T13:01:42.616736+0000","last_became_peered":"2023-11-28T13:01:42.616736+0000","last_unstale":"2023-11-28T13:01:42.626910+0000","last_undegraded":"2023-11-28T13:01:42.626910+0000","last_fullsized":"2023-11-28T13:01:42.626910+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:42:35.639836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"39'1","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.626828+0000","last_change":"2023-11-28T13:01:42.197100+0000","last_active":"2023-11-28T13:01:42.626828+0000","last_peered":"2023-11-28T13:01:42.626828+0000","last_clean":"2023-11-28T13:01:42.626828+0000","last_became_active":"2023-11-28T13:01:42.196854+0000","last_became_peered":"2023-11-28T13:01:42.196854+0000","last_unstale":"2023-11-28T13:01:42.626828+0000","last_undegraded":"2023-11-28T13:01:42.626828+0000","last_fullsized":"2023-11-28T13:01:42.626828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:26:35.106734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627192+0000","last_change":"2023-11-28T13:01:42.619671+0000","last_active":"2023-11-28T13:01:42.627192+0000","last_peered":"2023-11-28T13:01:42.627192+0000","last_clean":"2023-11-28T13:01:42.627192+0000","last_became_active":"2023-11-28T13:01:42.617084+0000","last_became_peered":"2023-11-28T13:01:42.617084+0000","last_unstale":"2023-11-28T13:01:42.627192+0000","last_undegraded":"2023-11-28T13:01:42.627192+0000","last_fullsized":"2023-11-28T13:01:42.627192+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:07:51.501639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603523+0000","last_change":"2023-11-28T13:01:42.189324+0000","last_active":"2023-11-28T13:01:43.603523+0000","last_peered":"2023-11-28T13:01:43.603523+0000","last_clean":"2023-11-28T13:01:43.603523+0000","last_became_active":"2023-11-28T13:01:42.189011+0000","last_became_peered":"2023-11-28T13:01:42.189011+0000","last_unstale":"2023-11-28T13:01:43.603523+0000","last_undegraded":"2023-11-28T13:01:43.603523+0000","last_fullsized":"2023-11-28T13:01:43.603523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:53.830121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602768+0000","last_change":"2023-11-28T13:01:42.620560+0000","last_active":"2023-11-28T13:01:43.602768+0000","last_peered":"2023-11-28T13:01:43.602768+0000","last_clean":"2023-11-28T13:01:43.602768+0000","last_became_active":"2023-11-28T13:01:42.620275+0000","last_became_peered":"2023-11-28T13:01:42.620275+0000","last_unstale":"2023-11-28T13:01:43.602768+0000","last_undegraded":"2023-11-28T13:01:43.602768+0000","last_fullsized":"2023-11-28T13:01:43.602768+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:03:45.927554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603347+0000","last_change":"2023-11-28T13:01:42.187236+0000","last_active":"2023-11-28T13:01:43.603347+0000","last_peered":"2023-11-28T13:01:43.603347+0000","last_clean":"2023-11-28T13:01:43.603347+0000","last_became_active":"2023-11-28T13:01:42.186927+0000","last_became_peered":"2023-11-28T13:01:42.186927+0000","last_unstale":"2023-11-28T13:01:43.603347+0000","last_undegraded":"2023-11-28T13:01:43.603347+0000","last_fullsized":"2023-11-28T13:01:43.603347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:16.716838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.626132+0000","last_change":"2023-11-28T13:01:42.621914+0000","last_active":"2023-11-28T13:01:42.626132+0000","last_peered":"2023-11-28T13:01:42.626132+0000","last_clean":"2023-11-28T13:01:42.626132+0000","last_became_active":"2023-11-28T13:01:42.621624+0000","last_became_peered":"2023-11-28T13:01:42.621624+0000","last_unstale":"2023-11-28T13:01:42.626132+0000","last_undegraded":"2023-11-28T13:01:42.626132+0000","last_fullsized":"2023-11-28T13:01:42.626132+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:56:08.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.612522+0000","last_change":"2023-11-28T13:01:42.196503+0000","last_active":"2023-11-28T13:01:42.612522+0000","last_peered":"2023-11-28T13:01:42.612522+0000","last_clean":"2023-11-28T13:01:42.612522+0000","last_became_active":"2023-11-28T13:01:42.196215+0000","last_became_peered":"2023-11-28T13:01:42.196215+0000","last_unstale":"2023-11-28T13:01:42.612522+0000","last_undegraded":"2023-11-28T13:01:42.612522+0000","last_fullsized":"2023-11-28T13:01:42.612522+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:25:49.909597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602898+0000","last_change":"2023-11-28T13:01:42.617055+0000","last_active":"2023-11-28T13:01:43.602898+0000","last_peered":"2023-11-28T13:01:43.602898+0000","last_clean":"2023-11-28T13:01:43.602898+0000","last_became_active":"2023-11-28T13:01:42.616908+0000","last_became_peered":"2023-11-28T13:01:42.616908+0000","last_unstale":"2023-11-28T13:01:43.602898+0000","last_undegraded":"2023-11-28T13:01:43.602898+0000","last_fullsized":"2023-11-28T13:01:43.602898+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:56.456532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652606+0000","last_change":"2023-11-28T13:01:42.199791+0000","last_active":"2023-11-28T13:01:43.652606+0000","last_peered":"2023-11-28T13:01:43.652606+0000","last_clean":"2023-11-28T13:01:43.652606+0000","last_became_active":"2023-11-28T13:01:42.199486+0000","last_became_peered":"2023-11-28T13:01:42.199486+0000","last_unstale":"2023-11-28T13:01:43.652606+0000","last_undegraded":"2023-11-28T13:01:43.652606+0000","last_fullsized":"2023-11-28T13:01:43.652606+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:02.788585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652688+0000","last_change":"2023-11-28T13:01:42.619612+0000","last_active":"2023-11-28T13:01:43.652688+0000","last_peered":"2023-11-28T13:01:43.652688+0000","last_clean":"2023-11-28T13:01:43.652688+0000","last_became_active":"2023-11-28T13:01:42.619350+0000","last_became_peered":"2023-11-28T13:01:42.619350+0000","last_unstale":"2023-11-28T13:01:43.652688+0000","last_undegraded":"2023-11-28T13:01:43.652688+0000","last_fullsized":"2023-11-28T13:01:43.652688+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:21:37.363198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.605549+0000","last_change":"2023-11-28T13:01:42.185587+0000","last_active":"2023-11-28T13:01:43.605549+0000","last_peered":"2023-11-28T13:01:43.605549+0000","last_clean":"2023-11-28T13:01:43.605549+0000","last_became_active":"2023-11-28T13:01:42.185256+0000","last_became_peered":"2023-11-28T13:01:42.185256+0000","last_unstale":"2023-11-28T13:01:43.605549+0000","last_undegraded":"2023-11-28T13:01:43.605549+0000","last_fullsized":"2023-11-28T13:01:43.605549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:19.899215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603202+0000","last_change":"2023-11-28T13:01:42.621313+0000","last_active":"2023-11-28T13:01:43.603202+0000","last_peered":"2023-11-28T13:01:43.603202+0000","last_clean":"2023-11-28T13:01:43.603202+0000","last_became_active":"2023-11-28T13:01:42.621051+0000","last_became_peered":"2023-11-28T13:01:42.621051+0000","last_unstale":"2023-11-28T13:01:43.603202+0000","last_undegraded":"2023-11-28T13:01:43.603202+0000","last_fullsized":"2023-11-28T13:01:43.603202+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:25:04.329423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733491+0000","last_change":"2023-11-28T13:01:42.194697+0000","last_active":"2023-11-28T13:01:43.733491+0000","last_peered":"2023-11-28T13:01:43.733491+0000","last_clean":"2023-11-28T13:01:43.733491+0000","last_became_active":"2023-11-28T13:01:42.194387+0000","last_became_peered":"2023-11-28T13:01:42.194387+0000","last_unstale":"2023-11-28T13:01:43.733491+0000","last_undegraded":"2023-11-28T13:01:43.733491+0000","last_fullsized":"2023-11-28T13:01:43.733491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:34:39.879971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":30,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.610082+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:01:42.610082+0000","last_peered":"2023-11-28T13:01:42.610082+0000","last_clean":"2023-11-28T13:01:42.610082+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:01:42.610082+0000","last_undegraded":"2023-11-28T13:01:42.610082+0000","last_fullsized":"2023-11-28T13:01:42.610082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.602873+0000","last_change":"2023-11-28T13:01:42.622392+0000","last_active":"2023-11-28T13:01:43.602873+0000","last_peered":"2023-11-28T13:01:43.602873+0000","last_clean":"2023-11-28T13:01:43.602873+0000","last_became_active":"2023-11-28T13:01:42.622121+0000","last_became_peered":"2023-11-28T13:01:42.622121+0000","last_unstale":"2023-11-28T13:01:43.602873+0000","last_undegraded":"2023-11-28T13:01:43.602873+0000","last_fullsized":"2023-11-28T13:01:43.602873+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:55.538689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606013+0000","last_change":"2023-11-28T13:01:42.190826+0000","last_active":"2023-11-28T13:01:43.606013+0000","last_peered":"2023-11-28T13:01:43.606013+0000","last_clean":"2023-11-28T13:01:43.606013+0000","last_became_active":"2023-11-28T13:01:42.190488+0000","last_became_peered":"2023-11-28T13:01:42.190488+0000","last_unstale":"2023-11-28T13:01:43.606013+0000","last_undegraded":"2023-11-28T13:01:43.606013+0000","last_fullsized":"2023-11-28T13:01:43.606013+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:25.334954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":31,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.600138+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:01:42.600138+0000","last_peered":"2023-11-28T13:01:42.600138+0000","last_clean":"2023-11-28T13:01:42.600138+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:01:42.600138+0000","last_undegraded":"2023-11-28T13:01:42.600138+0000","last_fullsized":"2023-11-28T13:01:42.600138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606054+0000","last_change":"2023-11-28T13:01:42.614706+0000","last_active":"2023-11-28T13:01:43.606054+0000","last_peered":"2023-11-28T13:01:43.606054+0000","last_clean":"2023-11-28T13:01:43.606054+0000","last_became_active":"2023-11-28T13:01:42.614405+0000","last_became_peered":"2023-11-28T13:01:42.614405+0000","last_unstale":"2023-11-28T13:01:43.606054+0000","last_undegraded":"2023-11-28T13:01:43.606054+0000","last_fullsized":"2023-11-28T13:01:43.606054+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:09.795817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.603400+0000","last_change":"2023-11-28T13:01:42.198137+0000","last_active":"2023-11-28T13:01:42.603400+0000","last_peered":"2023-11-28T13:01:42.603400+0000","last_clean":"2023-11-28T13:01:42.603400+0000","last_became_active":"2023-11-28T13:01:42.197851+0000","last_became_peered":"2023-11-28T13:01:42.197851+0000","last_unstale":"2023-11-28T13:01:42.603400+0000","last_undegraded":"2023-11-28T13:01:42.603400+0000","last_fullsized":"2023-11-28T13:01:42.603400+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:44:11.853093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.185877+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:01:42.185877+0000","last_peered":"2023-11-28T13:01:42.185877+0000","last_clean":"2023-11-28T13:01:42.185877+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:01:42.185877+0000","last_undegraded":"2023-11-28T13:01:42.185877+0000","last_fullsized":"2023-11-28T13:01:42.185877+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":183,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.602550+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:01:42.602550+0000","last_peered":"2023-11-28T13:01:42.602550+0000","last_clean":"2023-11-28T13:01:42.602550+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:01:42.602550+0000","last_undegraded":"2023-11-28T13:01:42.602550+0000","last_fullsized":"2023-11-28T13:01:42.602550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652912+0000","last_change":"2023-11-28T13:01:42.613295+0000","last_active":"2023-11-28T13:01:43.652912+0000","last_peered":"2023-11-28T13:01:43.652912+0000","last_clean":"2023-11-28T13:01:43.652912+0000","last_became_active":"2023-11-28T13:01:42.612994+0000","last_became_peered":"2023-11-28T13:01:42.612994+0000","last_unstale":"2023-11-28T13:01:43.652912+0000","last_undegraded":"2023-11-28T13:01:43.652912+0000","last_fullsized":"2023-11-28T13:01:43.652912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:34:29.680397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.606188+0000","last_change":"2023-11-28T13:01:42.190891+0000","last_active":"2023-11-28T13:01:43.606188+0000","last_peered":"2023-11-28T13:01:43.606188+0000","last_clean":"2023-11-28T13:01:43.606188+0000","last_became_active":"2023-11-28T13:01:42.190442+0000","last_became_peered":"2023-11-28T13:01:42.190442+0000","last_unstale":"2023-11-28T13:01:43.606188+0000","last_undegraded":"2023-11-28T13:01:43.606188+0000","last_fullsized":"2023-11-28T13:01:43.606188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:02:06.893878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":34,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.603181+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:01:42.603181+0000","last_peered":"2023-11-28T13:01:42.603181+0000","last_clean":"2023-11-28T13:01:42.603181+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:01:42.603181+0000","last_undegraded":"2023-11-28T13:01:42.603181+0000","last_fullsized":"2023-11-28T13:01:42.603181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.653138+0000","last_change":"2023-11-28T13:01:42.607948+0000","last_active":"2023-11-28T13:01:43.653138+0000","last_peered":"2023-11-28T13:01:43.653138+0000","last_clean":"2023-11-28T13:01:43.653138+0000","last_became_active":"2023-11-28T13:01:42.607542+0000","last_became_peered":"2023-11-28T13:01:42.607542+0000","last_unstale":"2023-11-28T13:01:43.653138+0000","last_undegraded":"2023-11-28T13:01:43.653138+0000","last_fullsized":"2023-11-28T13:01:43.653138+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:03:45.927554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"39'2","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.629922+0000","last_change":"2023-11-28T13:01:42.192294+0000","last_active":"2023-11-28T13:01:42.629922+0000","last_peered":"2023-11-28T13:01:42.629922+0000","last_clean":"2023-11-28T13:01:42.629922+0000","last_became_active":"2023-11-28T13:01:42.191964+0000","last_became_peered":"2023-11-28T13:01:42.191964+0000","last_unstale":"2023-11-28T13:01:42.629922+0000","last_undegraded":"2023-11-28T13:01:42.629922+0000","last_fullsized":"2023-11-28T13:01:42.629922+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:55.803583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.602436+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:01:42.602436+0000","last_peered":"2023-11-28T13:01:42.602436+0000","last_clean":"2023-11-28T13:01:42.602436+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:01:42.602436+0000","last_undegraded":"2023-11-28T13:01:42.602436+0000","last_fullsized":"2023-11-28T13:01:42.602436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.629661+0000","last_change":"2023-11-28T13:01:42.622794+0000","last_active":"2023-11-28T13:01:42.629661+0000","last_peered":"2023-11-28T13:01:42.629661+0000","last_clean":"2023-11-28T13:01:42.629661+0000","last_became_active":"2023-11-28T13:01:42.622530+0000","last_became_peered":"2023-11-28T13:01:42.622530+0000","last_unstale":"2023-11-28T13:01:42.629661+0000","last_undegraded":"2023-11-28T13:01:42.629661+0000","last_fullsized":"2023-11-28T13:01:42.629661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:26.339350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.603690+0000","last_change":"2023-11-28T13:01:42.195785+0000","last_active":"2023-11-28T13:01:43.603690+0000","last_peered":"2023-11-28T13:01:43.603690+0000","last_clean":"2023-11-28T13:01:43.603690+0000","last_became_active":"2023-11-28T13:01:42.195490+0000","last_became_peered":"2023-11-28T13:01:42.195490+0000","last_unstale":"2023-11-28T13:01:43.603690+0000","last_undegraded":"2023-11-28T13:01:43.603690+0000","last_fullsized":"2023-11-28T13:01:43.603690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:35:54.450538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":30,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.180423+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:01:42.180423+0000","last_peered":"2023-11-28T13:01:42.180423+0000","last_clean":"2023-11-28T13:01:42.180423+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:01:42.180423+0000","last_undegraded":"2023-11-28T13:01:42.180423+0000","last_fullsized":"2023-11-28T13:01:42.180423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.627939+0000","last_change":"2023-11-28T13:01:42.625683+0000","last_active":"2023-11-28T13:01:42.627939+0000","last_peered":"2023-11-28T13:01:42.627939+0000","last_clean":"2023-11-28T13:01:42.627939+0000","last_became_active":"2023-11-28T13:01:42.624735+0000","last_became_peered":"2023-11-28T13:01:42.624735+0000","last_unstale":"2023-11-28T13:01:42.627939+0000","last_undegraded":"2023-11-28T13:01:42.627939+0000","last_fullsized":"2023-11-28T13:01:42.627939+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:15:41.092930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.603401+0000","last_change":"2023-11-28T13:01:42.195922+0000","last_active":"2023-11-28T13:01:42.603401+0000","last_peered":"2023-11-28T13:01:42.603401+0000","last_clean":"2023-11-28T13:01:42.603401+0000","last_became_active":"2023-11-28T13:01:42.195650+0000","last_became_peered":"2023-11-28T13:01:42.195650+0000","last_unstale":"2023-11-28T13:01:42.603401+0000","last_undegraded":"2023-11-28T13:01:42.603401+0000","last_fullsized":"2023-11-28T13:01:42.603401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:30:00.065910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.187751+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:01:42.187751+0000","last_peered":"2023-11-28T13:01:42.187751+0000","last_clean":"2023-11-28T13:01:42.187751+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:01:42.187751+0000","last_undegraded":"2023-11-28T13:01:42.187751+0000","last_fullsized":"2023-11-28T13:01:42.187751+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.733272+0000","last_change":"2023-11-28T13:01:42.624225+0000","last_active":"2023-11-28T13:01:43.733272+0000","last_peered":"2023-11-28T13:01:43.733272+0000","last_clean":"2023-11-28T13:01:43.733272+0000","last_became_active":"2023-11-28T13:01:42.623895+0000","last_became_peered":"2023-11-28T13:01:42.623895+0000","last_unstale":"2023-11-28T13:01:43.733272+0000","last_undegraded":"2023-11-28T13:01:43.733272+0000","last_fullsized":"2023-11-28T13:01:43.733272+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:42.166626+0000","last_clean_scrub_stamp":"2023-11-28T13:01:42.166626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:26.241773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T13:01:43.652809+0000","last_change":"2023-11-28T13:01:42.193673+0000","last_active":"2023-11-28T13:01:43.652809+0000","last_peered":"2023-11-28T13:01:43.652809+0000","last_clean":"2023-11-28T13:01:43.652809+0000","last_became_active":"2023-11-28T13:01:42.193390+0000","last_became_peered":"2023-11-28T13:01:42.193390+0000","last_unstale":"2023-11-28T13:01:43.652809+0000","last_undegraded":"2023-11-28T13:01:43.652809+0000","last_fullsized":"2023-11-28T13:01:43.652809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:01:41.162230+0000","last_clean_scrub_stamp":"2023-11-28T13:01:41.162230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:41:10.086374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-28T13:01:42.187796+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:01:42.187796+0000","last_peered":"2023-11-28T13:01:42.187796+0000","last_clean":"2023-11-28T13:01:42.187796+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:01:42.187796+0000","last_undegraded":"2023-11-28T13:01:42.187796+0000","last_fullsized":"2023-11-28T13:01:42.187796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9812,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362028,"statfs":{"total":96636764160,"available":96626716672,"internally_reserved":0,"allocated":1060864,"data_stored":799095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.49199999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.60899999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.624},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8584,"kb_used_data":448,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363256,"statfs":{"total":96636764160,"available":96627974144,"internally_reserved":0,"allocated":458752,"data_stored":202320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.44}]},{"osd":1,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.53900000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.38600000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.496}]},{"osd":4,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.65400000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.62},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9668,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362172,"statfs":{"total":96636764160,"available":96626864128,"internally_reserved":0,"allocated":1044480,"data_stored":791097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.308,"15min":0.308},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.34899999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.52600000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.64200000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.623}]},{"osd":7,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.56000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":462848,"data_stored":207685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.252},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.40100000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.44}]},{"osd":4,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61899999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57899999999999996}]},{"osd":6,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79100000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:01:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8528,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363312,"statfs":{"total":96636764160,"available":96628031488,"internally_reserved":0,"allocated":466944,"data_stored":203234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.34499999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.317},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.28999999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.35499999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64700000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69699999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8716,"kb_used_data":452,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363124,"statfs":{"total":96636764160,"available":96627838976,"internally_reserved":0,"allocated":462848,"data_stored":202286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.318}]},{"osd":1,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.35799999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":5.0540000000000003,"5min":5.0540000000000003,"15min":5.0540000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.72899999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56899999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.73699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8728,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363112,"statfs":{"total":96636764160,"available":96627826688,"internally_reserved":0,"allocated":475136,"data_stored":210185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.67200000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.41999999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.499}]},{"osd":3,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.69899999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.627}]},{"osd":6,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.752},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.77800000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:01:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.81499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8580,"kb_used_data":444,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363260,"statfs":{"total":96636764160,"available":96627978240,"internally_reserved":0,"allocated":454656,"data_stored":201734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.64800000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.63}]},{"osd":2,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.68999999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73999999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.40899999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.42699999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:01:14 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.48899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:01:47.119 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:01:47.119 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T13:01:47.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.282+0000 7fa76f7c8700 1 Processor -- start 2023-11-28T13:01:47.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.283+0000 7fa76f7c8700 1 -- start start 2023-11-28T13:01:47.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.283+0000 7fa76f7c8700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa7680cf5f0 0x7fa7680caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.283+0000 7fa76f7c8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680cb3e0 0x7fa76806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.283+0000 7fa76f7c8700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7680ce550 con 0x7fa7680b1630 2023-11-28T13:01:47.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.283+0000 7fa76f7c8700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7680ce830 con 0x7fa7680cf5f0 2023-11-28T13:01:47.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.283+0000 7fa76f7c8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680ceb10 con 0x7fa7680cb3e0 2023-11-28T13:01:47.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.283+0000 7fa76d564700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fa7680b1630 0x7fa7680b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47822/0 (socket says 172.21.15.6:47822) 2023-11-28T13:01:47.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.283+0000 7fa76d564700 1 -- 172.21.15.6:0/3845601526 learned_addr learned my addr 172.21.15.6:0/3845601526 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:47.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa76cd63700 1 --2- 172.21.15.6:0/3845601526 >> v2:172.21.15.61:3300/0 conn(0x7fa7680cf5f0 0x7fa7680caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709391842 0 0) 0x7fa7680ce550 con 0x7fa7680b1630 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa76dd65700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680cb3e0 0x7fa76806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7540035b0 con 0x7fa7680b1630 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1491729485 0 0) 0x7fa7540035b0 con 0x7fa7680b1630 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680cb3e0 msgr2=0x7fa76806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680cb3e0 0x7fa76806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 >> v2:172.21.15.61:3300/0 conn(0x7fa7680cf5f0 msgr2=0x7fa7680caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 --2- 172.21.15.6:0/3845601526 >> v2:172.21.15.61:3300/0 conn(0x7fa7680cf5f0 0x7fa7680caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7680cedf0 con 0x7fa7680b1630 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fa758001670 con 0x7fa7680b1630 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 >> v1:172.21.15.6:6789/0 conn(0x7fa7680b1630 legacy=0x7fa7680b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa754003a00 0x7fa754003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680cb3e0 0x7fa754004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa754004d90 0x7fa754005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680ce830 con 0x7fa754003a00 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680ceb10 con 0x7fa754004d90 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75400ba10 con 0x7fa7680cb3e0 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa76d564700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa754003a00 0x7fa754003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa76cd63700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680cb3e0 0x7fa754004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.284+0000 7fa76dd65700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa754004d90 0x7fa754005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76d564700 1 -- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680cb3e0 msgr2=0x7fa754004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76d564700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680cb3e0 0x7fa754004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76d564700 1 -- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa754004d90 msgr2=0x7fa754005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76d564700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa754004d90 0x7fa754005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76d564700 1 -- 172.21.15.6:0/3845601526 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7540035b0 con 0x7fa754003a00 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76dd65700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa754004d90 0x7fa754005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76d564700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa754003a00 0x7fa754003e10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa75800c240 tx=0x7fa754005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bec7f169767ed22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758007c10 con 0x7fa754003a00 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75800ce00 con 0x7fa754003a00 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa75ffff700 1 -- 172.21.15.6:0/3845601526 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758005ba0 con 0x7fa754003a00 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76f7c8700 1 -- 172.21.15.6:0/3845601526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa754003a00 msgr2=0x7fa754003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76f7c8700 1 --2- 172.21.15.6:0/3845601526 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa754003a00 0x7fa754003e10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa75800c240 tx=0x7fa754005880 comp rx=0 tx=0).stop 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76dd65700 1 -- 172.21.15.6:0/3845601526 reap_dead start 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76f7c8700 1 -- 172.21.15.6:0/3845601526 shutdown_connections 2023-11-28T13:01:47.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.285+0000 7fa76f7c8700 1 -- 172.21.15.6:0/3845601526 >> 172.21.15.6:0/3845601526 conn(0x7fa7680cc630 msgr2=0x7fa7680cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:47.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.286+0000 7fa76f7c8700 1 -- 172.21.15.6:0/3845601526 shutdown_connections 2023-11-28T13:01:47.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.286+0000 7fa76f7c8700 1 -- 172.21.15.6:0/3845601526 wait complete. 2023-11-28T13:01:47.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.287+0000 7fa76f7c8700 1 Processor -- start 2023-11-28T13:01:47.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.287+0000 7fa76f7c8700 1 -- start start 2023-11-28T13:01:47.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.287+0000 7fa76f7c8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa754003a00 0x7fa7680b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76f7c8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680c7d20 0x7fa76815bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76f7c8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7680b3e40 0x7fa7680c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76f7c8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680b3a20 con 0x7fa7680b3e40 2023-11-28T13:01:47.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76f7c8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680b2da0 con 0x7fa754003a00 2023-11-28T13:01:47.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76f7c8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680b3080 con 0x7fa7680c7d20 2023-11-28T13:01:47.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76dd65700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7680b3e40 0x7fa7680c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76dd65700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7680b3e40 0x7fa7680c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60518/0 (socket says 172.21.15.6:60518) 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76dd65700 1 -- 172.21.15.6:0/1091924447 learned_addr learned my addr 172.21.15.6:0/1091924447 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76d564700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa754003a00 0x7fa7680b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76cd63700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680c7d20 0x7fa76815bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76dd65700 1 -- 172.21.15.6:0/1091924447 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680c7d20 msgr2=0x7fa76815bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76dd65700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680c7d20 0x7fa76815bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76dd65700 1 -- 172.21.15.6:0/1091924447 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa754003a00 msgr2=0x7fa7680b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76dd65700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa754003a00 0x7fa7680b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76dd65700 1 -- 172.21.15.6:0/1091924447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7680ce140 con 0x7fa7680b3e40 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76dd65700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7680b3e40 0x7fa7680c8640 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa768066a20 tx=0x7fa7600104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa75dffb700 1 -- 172.21.15.6:0/1091924447 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76001b070 con 0x7fa7680b3e40 2023-11-28T13:01:47.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa75dffb700 1 -- 172.21.15.6:0/1091924447 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76000f040 con 0x7fa7680b3e40 2023-11-28T13:01:47.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa76f7c8700 1 -- 172.21.15.6:0/1091924447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7680cf0d0 con 0x7fa7680b3e40 2023-11-28T13:01:47.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.288+0000 7fa75dffb700 1 -- 172.21.15.6:0/1091924447 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76000e050 con 0x7fa7680b3e40 2023-11-28T13:01:47.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.289+0000 7fa76f7c8700 1 -- 172.21.15.6:0/1091924447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7680ce2d0 con 0x7fa7680b3e40 2023-11-28T13:01:47.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.289+0000 7fa76f7c8700 1 -- 172.21.15.6:0/1091924447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7680b3b60 con 0x7fa7680b3e40 2023-11-28T13:01:47.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.290+0000 7fa75dffb700 1 -- 172.21.15.6:0/1091924447 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa760015440 con 0x7fa7680b3e40 2023-11-28T13:01:47.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.290+0000 7fa75dffb700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa74c068320 0x7fa74c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.291+0000 7fa75dffb700 1 -- 172.21.15.6:0/1091924447 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa7600860a0 con 0x7fa7680b3e40 2023-11-28T13:01:47.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.291+0000 7fa76d564700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa74c068320 0x7fa74c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.291+0000 7fa76d564700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa74c068320 0x7fa74c06a7d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa758000c00 tx=0x7fa758007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:47.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.295+0000 7fa75dffb700 1 -- 172.21.15.6:0/1091924447 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7680b3b60 con 0x7fa7680b3e40 2023-11-28T13:01:47.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.573+0000 7fa76f7c8700 1 -- 172.21.15.6:0/1091924447 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fa7680b3b60 con 0x7fa7680b3e40 2023-11-28T13:01:47.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.577+0000 7fa75dffb700 1 -- 172.21.15.6:0/1091924447 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa7680b3b60 con 0x7fa7680b3e40 2023-11-28T13:01:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 -- 172.21.15.6:0/1091924447 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa74c068320 msgr2=0x7fa74c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa74c068320 0x7fa74c06a7d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa758000c00 tx=0x7fa758007000 comp rx=0 tx=0).stop 2023-11-28T13:01:47.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 -- 172.21.15.6:0/1091924447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7680b3e40 msgr2=0x7fa7680c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7680b3e40 0x7fa7680c8640 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa768066a20 tx=0x7fa7600104a0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 -- 172.21.15.6:0/1091924447 shutdown_connections 2023-11-28T13:01:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7680b3e40 0x7fa7680c8640 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa754003a00 0x7fa7680b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa74c068320 0x7fa74c06a7d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 --2- 172.21.15.6:0/1091924447 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7680c7d20 0x7fa76815bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 -- 172.21.15.6:0/1091924447 >> 172.21.15.6:0/1091924447 conn(0x7fa7680cc630 msgr2=0x7fa7680b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 -- 172.21.15.6:0/1091924447 shutdown_connections 2023-11-28T13:01:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.579+0000 7fa76f7c8700 1 -- 172.21.15.6:0/1091924447 wait complete. 2023-11-28T13:01:47.583 INFO:teuthology.orchestra.run.smithi006.stderr:updated caps for client.0 2023-11-28T13:01:47.593 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:01:47.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.716+0000 7fc4eef96700 1 Processor -- start 2023-11-28T13:01:47.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.717+0000 7fc4eef96700 1 -- start start 2023-11-28T13:01:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.717+0000 7fc4eef96700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4e80b2660 0x7fc4e80ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.717+0000 7fc4eef96700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc4e80cecf0 0x7fc4e806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.717+0000 7fc4eef96700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e80b30b0 con 0x7fc4e815c560 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.717+0000 7fc4eef96700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e806c020 con 0x7fc4e80cecf0 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.717+0000 7fc4eef96700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80cf130 con 0x7fc4e80b2660 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.717+0000 7fc4ed533700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc4e80cecf0 0x7fc4e806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4ed533700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc4e80cecf0 0x7fc4e806c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60526/0 (socket says 172.21.15.6:60526) 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4ecd32700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fc4e815c560 0x7fc4e806f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47842/0 (socket says 172.21.15.6:47842) 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4ed533700 1 -- 172.21.15.6:0/1693589101 learned_addr learned my addr 172.21.15.6:0/1693589101 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4e7fff700 1 --2- 172.21.15.6:0/1693589101 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4e80b2660 0x7fc4e80ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4ed533700 1 -- 172.21.15.6:0/1693589101 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4e80b2660 msgr2=0x7fc4e80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4ed533700 1 --2- 172.21.15.6:0/1693589101 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4e80b2660 0x7fc4e80ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4ed533700 1 -- 172.21.15.6:0/1693589101 >> v1:172.21.15.6:6789/0 conn(0x7fc4e815c560 legacy=0x7fc4e806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4e77fe700 1 -- 172.21.15.6:0/1693589101 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806945106 0 0) 0x7fc4e80b30b0 con 0x7fc4e815c560 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4ed533700 1 -- 172.21.15.6:0/1693589101 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80d09c0 con 0x7fc4e80cecf0 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4ed533700 1 --2- 172.21.15.6:0/1693589101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80cecf0 0x7fc4e806c4b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009e50 tx=0x7fc4e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165726930deeb80f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4e77fe700 1 -- 172.21.15.6:0/1693589101 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e000e070 con 0x7fc4e80cecf0 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4e77fe700 1 -- 172.21.15.6:0/1693589101 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e000b910 con 0x7fc4e80cecf0 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.718+0000 7fc4e77fe700 1 -- 172.21.15.6:0/1693589101 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e000bcc0 con 0x7fc4e80cecf0 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.719+0000 7fc4eef96700 1 -- 172.21.15.6:0/1693589101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80cecf0 msgr2=0x7fc4e806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.719+0000 7fc4eef96700 1 --2- 172.21.15.6:0/1693589101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80cecf0 0x7fc4e806c4b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009e50 tx=0x7fc4e000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.719+0000 7fc4eef96700 1 -- 172.21.15.6:0/1693589101 shutdown_connections 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.719+0000 7fc4eef96700 1 --2- 172.21.15.6:0/1693589101 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80cecf0 0x7fc4e806c4b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.719+0000 7fc4eef96700 1 --2- 172.21.15.6:0/1693589101 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4e80b2660 0x7fc4e80ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.719+0000 7fc4eef96700 1 -- 172.21.15.6:0/1693589101 >> 172.21.15.6:0/1693589101 conn(0x7fc4e80cd660 msgr2=0x7fc4e80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:47.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.719+0000 7fc4eef96700 1 -- 172.21.15.6:0/1693589101 shutdown_connections 2023-11-28T13:01:47.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.719+0000 7fc4eef96700 1 -- 172.21.15.6:0/1693589101 wait complete. 2023-11-28T13:01:47.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.720+0000 7fc4eef96700 1 Processor -- start 2023-11-28T13:01:47.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.720+0000 7fc4eef96700 1 -- start start 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4eef96700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80b2660 0x7fc4e8158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4eef96700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4e80cecf0 0x7fc4e81f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4eef96700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4e815c560 0x7fc4e8155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4eef96700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e81f1db0 con 0x7fc4e80b2660 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4eef96700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e8156610 con 0x7fc4e815c560 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4eef96700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e81568c0 con 0x7fc4e80cecf0 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4ecd32700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80b2660 0x7fc4e8158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4ecd32700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80b2660 0x7fc4e8158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60528/0 (socket says 172.21.15.6:60528) 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4ecd32700 1 -- 172.21.15.6:0/3755148241 learned_addr learned my addr 172.21.15.6:0/3755148241 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4e7fff700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4e80cecf0 0x7fc4e81f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4ed533700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4e815c560 0x7fc4e8155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4ecd32700 1 -- 172.21.15.6:0/3755148241 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4e80cecf0 msgr2=0x7fc4e81f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4ecd32700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4e80cecf0 0x7fc4e81f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4ecd32700 1 -- 172.21.15.6:0/3755148241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4e815c560 msgr2=0x7fc4e8155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4ecd32700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4e815c560 0x7fc4e8155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4ecd32700 1 -- 172.21.15.6:0/3755148241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e0009b30 con 0x7fc4e80b2660 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.721+0000 7fc4ed533700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4e815c560 0x7fc4e8155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.722+0000 7fc4ecd32700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80b2660 0x7fc4e8158c50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0011e0 tx=0x7fc4dc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.722+0000 7fc4e57fa700 1 -- 172.21.15.6:0/3755148241 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc004c80 con 0x7fc4e80b2660 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.722+0000 7fc4e57fa700 1 -- 172.21.15.6:0/3755148241 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc004de0 con 0x7fc4e80b2660 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.722+0000 7fc4e57fa700 1 -- 172.21.15.6:0/3755148241 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc007310 con 0x7fc4e80b2660 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.722+0000 7fc4eef96700 1 -- 172.21.15.6:0/3755148241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e80b2dd0 con 0x7fc4e80b2660 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.722+0000 7fc4eef96700 1 -- 172.21.15.6:0/3755148241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e80b2af0 con 0x7fc4e80b2660 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.723+0000 7fc4eef96700 1 -- 172.21.15.6:0/3755148241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e806bcb0 con 0x7fc4e80b2660 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.723+0000 7fc4e57fa700 1 -- 172.21.15.6:0/3755148241 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc4dc01a020 con 0x7fc4e80b2660 2023-11-28T13:01:47.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.723+0000 7fc4e57fa700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4cc0682f0 0x7fc4cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:47.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.724+0000 7fc4e57fa700 1 -- 172.21.15.6:0/3755148241 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc4dc085990 con 0x7fc4e80b2660 2023-11-28T13:01:47.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.724+0000 7fc4e7fff700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4cc0682f0 0x7fc4cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:47.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.724+0000 7fc4e7fff700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4cc0682f0 0x7fc4cc06a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009b40 tx=0x7fc4d4009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:47.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.728+0000 7fc4e57fa700 1 -- 172.21.15.6:0/3755148241 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc4dc0520b0 con 0x7fc4e80b2660 2023-11-28T13:01:47.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.965+0000 7fc4eef96700 1 -- 172.21.15.6:0/3755148241 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4e8155090 con 0x7fc4e80b2660 2023-11-28T13:01:47.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.966+0000 7fc4e57fa700 1 -- 172.21.15.6:0/3755148241 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4642 (secure 0 0 0) 0x7fc4dc0556d0 con 0x7fc4e80b2660 2023-11-28T13:01:47.967 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:47.967 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4026920958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4026920958},{"type":"v1","addr":"172.21.15.61:6833","nonce":4026920958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2212733196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2212733196},{"type":"v1","addr":"172.21.15.61:6835","nonce":2212733196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1674709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1674709721},{"type":"v1","addr":"172.21.15.6:6836","nonce":1674709721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:42.595217+0000","modified":"2023-11-28T13:01:45.602537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/2789212407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":2789212407},{"type":"v1","addr":"172.21.15.6:6837","nonce":2789212407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:01:47.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 -- 172.21.15.6:0/3755148241 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4cc0682f0 msgr2=0x7fc4cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4cc0682f0 0x7fc4cc06a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009b40 tx=0x7fc4d4009210 comp rx=0 tx=0).stop 2023-11-28T13:01:47.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 -- 172.21.15.6:0/3755148241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80b2660 msgr2=0x7fc4e8158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:47.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80b2660 0x7fc4e8158c50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0011e0 tx=0x7fc4dc00ca30 comp rx=0 tx=0).stop 2023-11-28T13:01:47.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 -- 172.21.15.6:0/3755148241 shutdown_connections 2023-11-28T13:01:47.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4e815c560 0x7fc4e8155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc4e80b2660 0x7fc4e8158c50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc4cc0682f0 0x7fc4cc06a7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 --2- 172.21.15.6:0/3755148241 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc4e80cecf0 0x7fc4e81f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:47.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 -- 172.21.15.6:0/3755148241 >> 172.21.15.6:0/3755148241 conn(0x7fc4e80cd660 msgr2=0x7fc4e815b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:47.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 -- 172.21.15.6:0/3755148241 shutdown_connections 2023-11-28T13:01:47.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:47.968+0000 7fc4eef96700 1 -- 172.21.15.6:0/3755148241 wait complete. 2023-11-28T13:01:47.970 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2023-11-28T13:01:48.384 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:01:42.595217+0000', 'modified': '2023-11-28T13:01:45.602537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5505': {'gid': 5505, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6837/2789212407', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6835', 'nonce': 2789212407}, {'type': 'v1', 'addr': '172.21.15.6:6837', 'nonce': 2789212407}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:01:48.384 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:01:48.384 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T13:01:48.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.517+0000 7f6fdb275700 1 Processor -- start 2023-11-28T13:01:48.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.517+0000 7f6fdb275700 1 -- start start 2023-11-28T13:01:48.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.518+0000 7f6fdb275700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6fd4159590 0x7f6fd406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.518+0000 7f6fdb275700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6fd415db70 0x7f6fd40ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.518+0000 7f6fdb275700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd415d120 con 0x7f6fd40ce890 2023-11-28T13:01:48.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.518+0000 7f6fdb275700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd415d6e0 con 0x7f6fd4159590 2023-11-28T13:01:48.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.518+0000 7f6fdb275700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd40aef60 con 0x7f6fd415db70 2023-11-28T13:01:48.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.518+0000 7f6fd9011700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6fd4159590 0x7f6fd406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.518+0000 7f6fd9812700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6fd40ce890 0x7f6fd40caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47860/0 (socket says 172.21.15.6:47860) 2023-11-28T13:01:48.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.518+0000 7f6fd9812700 1 -- 172.21.15.6:0/7509127 learned_addr learned my addr 172.21.15.6:0/7509127 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:48.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.518+0000 7f6fd8810700 1 --2- 172.21.15.6:0/7509127 >> v2:172.21.15.61:3300/0 conn(0x7f6fd415db70 0x7f6fd40ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fd9011700 1 -- 172.21.15.6:0/7509127 >> v2:172.21.15.61:3300/0 conn(0x7f6fd415db70 msgr2=0x7f6fd40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fd9011700 1 --2- 172.21.15.6:0/7509127 >> v2:172.21.15.61:3300/0 conn(0x7f6fd415db70 0x7f6fd40ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fd9011700 1 -- 172.21.15.6:0/7509127 >> v1:172.21.15.6:6789/0 conn(0x7f6fd40ce890 legacy=0x7f6fd40caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fcbfff700 1 -- 172.21.15.6:0/7509127 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747292735 0 0) 0x7f6fd415d120 con 0x7f6fd40ce890 2023-11-28T13:01:48.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fd9011700 1 -- 172.21.15.6:0/7509127 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd40af240 con 0x7f6fd4159590 2023-11-28T13:01:48.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fd9011700 1 --2- 172.21.15.6:0/7509127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 0x7f6fd406e400 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6fd415d080 tx=0x7f6fc4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1f774a91271cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fcbfff700 1 -- 172.21.15.6:0/7509127 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4011830 con 0x7f6fd4159590 2023-11-28T13:01:48.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fcbfff700 1 -- 172.21.15.6:0/7509127 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc4011990 con 0x7f6fd4159590 2023-11-28T13:01:48.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fcbfff700 1 -- 172.21.15.6:0/7509127 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4011d20 con 0x7f6fd4159590 2023-11-28T13:01:48.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fdb275700 1 -- 172.21.15.6:0/7509127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 msgr2=0x7f6fd406e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.519+0000 7f6fdb275700 1 --2- 172.21.15.6:0/7509127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 0x7f6fd406e400 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6fd415d080 tx=0x7f6fc4009b20 comp rx=0 tx=0).stop 2023-11-28T13:01:48.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.520+0000 7f6fdb275700 1 -- 172.21.15.6:0/7509127 shutdown_connections 2023-11-28T13:01:48.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.520+0000 7f6fdb275700 1 --2- 172.21.15.6:0/7509127 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 0x7f6fd406e400 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.520+0000 7f6fdb275700 1 --2- 172.21.15.6:0/7509127 >> v2:172.21.15.61:3300/0 conn(0x7f6fd415db70 0x7f6fd40ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.520+0000 7f6fdb275700 1 -- 172.21.15.6:0/7509127 >> 172.21.15.6:0/7509127 conn(0x7f6fd40b1650 msgr2=0x7f6fd40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.520+0000 7f6fdb275700 1 -- 172.21.15.6:0/7509127 shutdown_connections 2023-11-28T13:01:48.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.520+0000 7f6fdb275700 1 -- 172.21.15.6:0/7509127 wait complete. 2023-11-28T13:01:48.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.521+0000 7f6fdb275700 1 Processor -- start 2023-11-28T13:01:48.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.521+0000 7f6fdb275700 1 -- start start 2023-11-28T13:01:48.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fdb275700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fd40ce890 0x7f6fd41523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fdb275700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 0x7f6fd40b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fdb275700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fd415db70 0x7f6fd40b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fdb275700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd40b4d40 con 0x7f6fd4159590 2023-11-28T13:01:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fdb275700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd40b33b0 con 0x7f6fd415db70 2023-11-28T13:01:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fdb275700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd415aa30 con 0x7f6fd40ce890 2023-11-28T13:01:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd8810700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 0x7f6fd40b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd8810700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 0x7f6fd40b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60544/0 (socket says 172.21.15.6:60544) 2023-11-28T13:01:48.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd8810700 1 -- 172.21.15.6:0/3324759552 learned_addr learned my addr 172.21.15.6:0/3324759552 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:48.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd9011700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fd40ce890 0x7f6fd41523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd9812700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fd415db70 0x7f6fd40b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd8810700 1 -- 172.21.15.6:0/3324759552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fd40ce890 msgr2=0x7f6fd41523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd8810700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fd40ce890 0x7f6fd41523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd8810700 1 -- 172.21.15.6:0/3324759552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fd415db70 msgr2=0x7f6fd40b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd8810700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fd415db70 0x7f6fd40b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd8810700 1 -- 172.21.15.6:0/3324759552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc40094a0 con 0x7f6fd4159590 2023-11-28T13:01:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd9011700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fd40ce890 0x7f6fd41523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd9812700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fd415db70 0x7f6fd40b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fd8810700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 0x7f6fd40b3840 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000e870 tx=0x7f6fd000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fc9ffb700 1 -- 172.21.15.6:0/3324759552 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0009590 con 0x7f6fd4159590 2023-11-28T13:01:48.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fc9ffb700 1 -- 172.21.15.6:0/3324759552 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd000ee70 con 0x7f6fd4159590 2023-11-28T13:01:48.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fdb275700 1 -- 172.21.15.6:0/3324759552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd415e010 con 0x7f6fd4159590 2023-11-28T13:01:48.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.522+0000 7f6fc9ffb700 1 -- 172.21.15.6:0/3324759552 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000ca50 con 0x7f6fd4159590 2023-11-28T13:01:48.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.523+0000 7f6fdb275700 1 -- 172.21.15.6:0/3324759552 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd415d370 con 0x7f6fd4159590 2023-11-28T13:01:48.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.524+0000 7f6fc9ffb700 1 -- 172.21.15.6:0/3324759552 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6fd0019030 con 0x7f6fd4159590 2023-11-28T13:01:48.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.525+0000 7f6fc9ffb700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6fb80682f0 0x7f6fb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.525+0000 7f6fd9011700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6fb80682f0 0x7f6fb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.525+0000 7f6fc9ffb700 1 -- 172.21.15.6:0/3324759552 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6fd0085d10 con 0x7f6fd4159590 2023-11-28T13:01:48.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.525+0000 7f6fd9011700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6fb80682f0 0x7f6fb806a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40090b0 tx=0x7f6fc4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.527+0000 7f6fe73feb80 1 Processor -- start 2023-11-28T13:01:48.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.527+0000 7f6fe73feb80 1 -- start start 2023-11-28T13:01:48.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d795a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7bd130 0x55558d7bfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7c0120 0x55558d7db770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558d67fbc0 con 0x55558d7bf810 2023-11-28T13:01:48.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558d6ee350 con 0x55558d7c0120 2023-11-28T13:01:48.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55558d6e7880 con 0x55558d7bd130 2023-11-28T13:01:48.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd9011700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d795a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd9011700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d795a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60550/0 (socket says 172.21.15.6:60550) 2023-11-28T13:01:48.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd9011700 1 -- 172.21.15.6:0/2964423077 learned_addr learned my addr 172.21.15.6:0/2964423077 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:48.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd8810700 1 --2- 172.21.15.6:0/2964423077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7bd130 0x55558d7bfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd9812700 1 --2- 172.21.15.6:0/2964423077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7c0120 0x55558d7db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd9011700 1 -- 172.21.15.6:0/2964423077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7bd130 msgr2=0x55558d7bfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd9011700 1 --2- 172.21.15.6:0/2964423077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7bd130 0x55558d7bfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd9011700 1 -- 172.21.15.6:0/2964423077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7c0120 msgr2=0x55558d7db770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd9011700 1 --2- 172.21.15.6:0/2964423077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7c0120 0x55558d7db770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd9011700 1 -- 172.21.15.6:0/2964423077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55558d5695e0 con 0x55558d7bf810 2023-11-28T13:01:48.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fd9011700 1 --2- 172.21.15.6:0/2964423077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d795a20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009e70 tx=0x7f6fc4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d87624970925cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.528+0000 7f6fbeffd700 1 -- 172.21.15.6:0/2964423077 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc402b070 con 0x55558d7bf810 2023-11-28T13:01:48.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fbeffd700 1 -- 172.21.15.6:0/2964423077 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc40037e0 con 0x55558d7bf810 2023-11-28T13:01:48.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fbeffd700 1 -- 172.21.15.6:0/2964423077 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc401f070 con 0x55558d7bf810 2023-11-28T13:01:48.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fe73feb80 1 -- 172.21.15.6:0/2964423077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 msgr2=0x55558d795a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/2964423077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d795a20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009e70 tx=0x7f6fc4010000 comp rx=0 tx=0).stop 2023-11-28T13:01:48.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fe73feb80 1 -- 172.21.15.6:0/2964423077 shutdown_connections 2023-11-28T13:01:48.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/2964423077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7c0120 0x55558d7db770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/2964423077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d795a20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/2964423077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7bd130 0x55558d7bfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fe73feb80 1 -- 172.21.15.6:0/2964423077 >> 172.21.15.6:0/2964423077 conn(0x55558d792d60 msgr2=0x55558d7936f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fe73feb80 1 -- 172.21.15.6:0/2964423077 shutdown_connections 2023-11-28T13:01:48.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.529+0000 7f6fe73feb80 1 -- 172.21.15.6:0/2964423077 wait complete. 2023-11-28T13:01:48.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.530+0000 7f6fe73feb80 1 -- start start 2023-11-28T13:01:48.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.530+0000 7f6fe73feb80 1 Processor -- start 2023-11-28T13:01:48.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d7bea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bf810 0x55558d7d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558d56b8d0 con 0x55558d7bd130 2023-11-28T13:01:48.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558d7c0ee0 con 0x55558d7bf810 2023-11-28T13:01:48.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55558d7af830 con 0x55558d7c0120 2023-11-28T13:01:48.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fe73feb80 10 client.0 _ll_register_callbacks cb 0x55558d58d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:01:48.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd9011700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d7bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd9011700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d7bea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60556/0 (socket says 172.21.15.6:60556) 2023-11-28T13:01:48.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd9011700 1 -- 172.21.15.6:0/4144084113 learned_addr learned my addr 172.21.15.6:0/4144084113 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:48.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd8810700 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bf810 0x55558d7d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd9812700 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd9011700 1 -- 172.21.15.6:0/4144084113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 msgr2=0x55558d7d3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd9011700 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd9011700 1 -- 172.21.15.6:0/4144084113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bf810 msgr2=0x55558d7d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd9011700 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bf810 0x55558d7d3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd9011700 1 -- 172.21.15.6:0/4144084113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd40af240 con 0x55558d7bd130 2023-11-28T13:01:48.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fd9011700 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d7bea30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4033040 tx=0x7f6fc40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf50badd3cf88006 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.531+0000 7f6fb3fff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:01:48.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fb3fff700 1 -- 172.21.15.6:0/4144084113 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc402b040 con 0x55558d7bd130 2023-11-28T13:01:48.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fb3fff700 1 -- 172.21.15.6:0/4144084113 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6fc4003e20 con 0x55558d7bd130 2023-11-28T13:01:48.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fe73feb80 1 -- 172.21.15.6:0/4144084113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55558d6b5140 con 0x55558d7bd130 2023-11-28T13:01:48.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fb3fff700 1 -- 172.21.15.6:0/4144084113 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fc4006ca0 con 0x55558d7bd130 2023-11-28T13:01:48.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fe73feb80 10 client.5694 fetch_fsmap learned FSMap version 16 2023-11-28T13:01:48.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fe73feb80 1 -- 172.21.15.6:0/4144084113 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55558d515d90 con 0x55558d7bd130 2023-11-28T13:01:48.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fb3fff700 1 -- 172.21.15.6:0/4144084113 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55558d6b5140 con 0x55558d7bd130 2023-11-28T13:01:48.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fb3fff700 1 -- 172.21.15.6:0/4144084113 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6fc4037030 con 0x55558d7bd130 2023-11-28T13:01:48.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fe73feb80 10 client.5694 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T13:01:48.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fe73feb80 10 client.5694 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:01:48.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fe73feb80 20 client.5694 populate_metadata read hostname 'smithi006' 2023-11-28T13:01:48.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.6:6835/2789212407,v1:172.21.15.6:6837/2789212407] conn(0x55558d7d1720 0x55558d7d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fe73feb80 4 client.5694 mds_command: new command op to 5505 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:01:48.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fe73feb80 1 -- 172.21.15.6:0/4144084113 --> [v2:172.21.15.6:6835/2789212407,v1:172.21.15.6:6837/2789212407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55558d56bf10 con 0x55558d7d1720 2023-11-28T13:01:48.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fd8810700 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.6:6835/2789212407,v1:172.21.15.6:6837/2789212407] conn(0x55558d7d1720 0x55558d7d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fb3fff700 10 client.5694 ms_handle_connect on v2:172.21.15.6:6835/2789212407 2023-11-28T13:01:48.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.532+0000 7f6fd8810700 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.6:6835/2789212407,v1:172.21.15.6:6837/2789212407] conn(0x55558d7d1720 0x55558d7d1af0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfaaea7ddd8e01a server_cookie=67efc4323cb5ff4f in_seq=0 out_seq=0 2023-11-28T13:01:48.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fb3fff700 1 -- 172.21.15.6:0/4144084113 <== mds.0 v2:172.21.15.6:6835/2789212407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55558d56bf10 con 0x55558d7d1720 2023-11-28T13:01:48.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fb3fff700 10 client.5694 handle_command_reply: tid=0 2023-11-28T13:01:48.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 client.5694 shutdown 2023-11-28T13:01:48.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 -- 172.21.15.6:0/4144084113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 msgr2=0x55558d7bea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d7bea30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4033040 tx=0x7f6fc40047c0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 -- 172.21.15.6:0/4144084113 shutdown_connections 2023-11-28T13:01:48.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d7bea30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.6:6835/2789212407,v1:172.21.15.6:6837/2789212407] conn(0x55558d7d1720 0x55558d7d1af0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/4144084113 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bf810 0x55558d7d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fb3fff700 0 client.5694 ms_handle_reset on v2:172.21.15.6:6835/2789212407 2023-11-28T13:01:48.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 -- 172.21.15.6:0/4144084113 >> 172.21.15.6:0/4144084113 conn(0x55558d792d60 msgr2=0x55558d794230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 -- 172.21.15.6:0/4144084113 shutdown_connections 2023-11-28T13:01:48.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.533+0000 7f6fe73feb80 1 -- 172.21.15.6:0/4144084113 wait complete. 2023-11-28T13:01:48.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.534+0000 7f6fe73feb80 20 client.5694 trim_cache size 0 max 16384 2023-11-28T13:01:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.551+0000 7f6fe73feb80 1 Processor -- start 2023-11-28T13:01:48.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.551+0000 7f6fe73feb80 1 -- start start 2023-11-28T13:01:48.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bd130 0x55558d7d4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d7bda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558d7d2dd0 con 0x55558d7bf810 2023-11-28T13:01:48.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558d7ae9a0 con 0x55558d7bd130 2023-11-28T13:01:48.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55558d7bcfa0 con 0x55558d7c0120 2023-11-28T13:01:48.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd8810700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d7bda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd8810700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d7bda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60566/0 (socket says 172.21.15.6:60566) 2023-11-28T13:01:48.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd8810700 1 -- 172.21.15.6:0/728898092 learned_addr learned my addr 172.21.15.6:0/728898092 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:48.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd9011700 1 --2- 172.21.15.6:0/728898092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bd130 0x55558d7d4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd9812700 1 --2- 172.21.15.6:0/728898092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd8810700 1 -- 172.21.15.6:0/728898092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 msgr2=0x55558d7d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd8810700 1 --2- 172.21.15.6:0/728898092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd8810700 1 -- 172.21.15.6:0/728898092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bd130 msgr2=0x55558d7d4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd8810700 1 --2- 172.21.15.6:0/728898092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bd130 0x55558d7d4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd8810700 1 -- 172.21.15.6:0/728898092 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55558d7d3620 con 0x55558d7bf810 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd9011700 1 --2- 172.21.15.6:0/728898092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bd130 0x55558d7d4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.552+0000 7f6fd8810700 1 --2- 172.21.15.6:0/728898092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d7bda50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6fd005dba0 tx=0x7f6fd0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992a5da9f42f79cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fbf7fe700 1 -- 172.21.15.6:0/728898092 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0055530 con 0x55558d7bf810 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fbf7fe700 1 -- 172.21.15.6:0/728898092 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd0055690 con 0x55558d7bf810 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fbf7fe700 1 -- 172.21.15.6:0/728898092 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0055800 con 0x55558d7bf810 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fe73feb80 1 -- 172.21.15.6:0/728898092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 msgr2=0x55558d7bda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/728898092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d7bda50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6fd005dba0 tx=0x7f6fd0000bc0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fe73feb80 1 -- 172.21.15.6:0/728898092 shutdown_connections 2023-11-28T13:01:48.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/728898092 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/728898092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bd130 0x55558d7d4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/728898092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bf810 0x55558d7bda50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fe73feb80 1 -- 172.21.15.6:0/728898092 >> 172.21.15.6:0/728898092 conn(0x55558d792d60 msgr2=0x55558d794230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fe73feb80 1 -- 172.21.15.6:0/728898092 shutdown_connections 2023-11-28T13:01:48.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.553+0000 7f6fe73feb80 1 -- 172.21.15.6:0/728898092 wait complete. 2023-11-28T13:01:48.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.554+0000 7f6fe73feb80 1 -- start start 2023-11-28T13:01:48.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.554+0000 7f6fe73feb80 1 Processor -- start 2023-11-28T13:01:48.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d83edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bf810 0x55558d7d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fe73feb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558d7be180 con 0x55558d7bd130 2023-11-28T13:01:48.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55558d7de3f0 con 0x55558d7bf810 2023-11-28T13:01:48.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fe73feb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55558d7ad7e0 con 0x55558d7c0120 2023-11-28T13:01:48.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fe73feb80 10 client.0 _ll_register_callbacks cb 0x55558d58d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:01:48.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fd9011700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d83edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fd9011700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d83edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60570/0 (socket says 172.21.15.6:60570) 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fd9011700 1 -- 172.21.15.6:0/1782703033 learned_addr learned my addr 172.21.15.6:0/1782703033 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fd9812700 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fd9011700 1 -- 172.21.15.6:0/1782703033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 msgr2=0x55558d7d3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fd9011700 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fd9011700 1 -- 172.21.15.6:0/1782703033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bf810 msgr2=0x55558d7d37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fd9011700 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bf810 0x55558d7d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fd9011700 1 -- 172.21.15.6:0/1782703033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd415d370 con 0x55558d7bd130 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fd9011700 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d83edc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4030d60 tx=0x7f6fc4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9292632d4abf71 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fb3fff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fb3fff700 1 -- 172.21.15.6:0/1782703033 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc402b070 con 0x55558d7bd130 2023-11-28T13:01:48.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fb3fff700 1 -- 172.21.15.6:0/1782703033 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6fc404dd80 con 0x55558d7bd130 2023-11-28T13:01:48.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fe73feb80 1 -- 172.21.15.6:0/1782703033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55558d6a5160 con 0x55558d7bd130 2023-11-28T13:01:48.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fb3fff700 1 -- 172.21.15.6:0/1782703033 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fc4003890 con 0x55558d7bd130 2023-11-28T13:01:48.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fe73feb80 10 client.5700 fetch_fsmap learned FSMap version 16 2023-11-28T13:01:48.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.555+0000 7f6fe73feb80 1 -- 172.21.15.6:0/1782703033 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55558d4fec10 con 0x55558d7bd130 2023-11-28T13:01:48.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fb3fff700 1 -- 172.21.15.6:0/1782703033 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55558d6a5160 con 0x55558d7bd130 2023-11-28T13:01:48.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fb3fff700 1 -- 172.21.15.6:0/1782703033 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 16) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f6fc40043d0 con 0x55558d7bd130 2023-11-28T13:01:48.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fe73feb80 10 client.5700 fetch_fsmap finished waiting for FSMap version 16 2023-11-28T13:01:48.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fe73feb80 10 client.5700 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T13:01:48.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fe73feb80 20 client.5700 populate_metadata read hostname 'smithi006' 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.6:6835/2789212407,v1:172.21.15.6:6837/2789212407] conn(0x55558d78e450 0x55558d78e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fe73feb80 4 client.5700 mds_command: new command op to 5505 tid=0[{"prefix": "status"}] 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fe73feb80 1 -- 172.21.15.6:0/1782703033 --> [v2:172.21.15.6:6835/2789212407,v1:172.21.15.6:6837/2789212407] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55558d68f170 con 0x55558d78e450 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fd8810700 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.6:6835/2789212407,v1:172.21.15.6:6837/2789212407] conn(0x55558d78e450 0x55558d78e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fb3fff700 10 client.5700 ms_handle_connect on v2:172.21.15.6:6835/2789212407 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fd8810700 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.6:6835/2789212407,v1:172.21.15.6:6837/2789212407] conn(0x55558d78e450 0x55558d78e820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69c09bac21587d7 server_cookie=32210ecbe82161b0 in_seq=0 out_seq=0 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fb3fff700 1 -- 172.21.15.6:0/1782703033 <== mds.0 v2:172.21.15.6:6835/2789212407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55558d68f170 con 0x55558d78e450 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fb3fff700 10 client.5700 handle_command_reply: tid=0 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.556+0000 7f6fe73feb80 1 client.5700 shutdown 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 1 -- 172.21.15.6:0/1782703033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 msgr2=0x55558d83edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d83edc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4030d60 tx=0x7f6fc4011000 comp rx=0 tx=0).stop 2023-11-28T13:01:48.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 1 -- 172.21.15.6:0/1782703033 shutdown_connections 2023-11-28T13:01:48.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55558d7c0120 0x55558d7d3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55558d7bd130 0x55558d83edc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.6:6835/2789212407,v1:172.21.15.6:6837/2789212407] conn(0x55558d78e450 0x55558d78e820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 1 --2- 172.21.15.6:0/1782703033 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55558d7bf810 0x55558d7d37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fb3fff700 0 client.5700 ms_handle_reset on v2:172.21.15.6:6835/2789212407 2023-11-28T13:01:48.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 1 -- 172.21.15.6:0/1782703033 >> 172.21.15.6:0/1782703033 conn(0x55558d792d60 msgr2=0x55558d791020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 1 -- 172.21.15.6:0/1782703033 shutdown_connections 2023-11-28T13:01:48.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 1 -- 172.21.15.6:0/1782703033 wait complete. 2023-11-28T13:01:48.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fe73feb80 20 client.5700 trim_cache size 0 max 16384 2023-11-28T13:01:48.570 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:01:48.570 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "dc643718-e974-4d8a-b829-ff6ced554687", 2023-11-28T13:01:48.570 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2023-11-28T13:01:48.570 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 5505, 2023-11-28T13:01:48.570 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2023-11-28T13:01:48.571 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2023-11-28T13:01:48.571 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2023-11-28T13:01:48.571 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 5.9610233929999996, 2023-11-28T13:01:48.571 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 16, 2023-11-28T13:01:48.571 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 40, 2023-11-28T13:01:48.571 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 40, 2023-11-28T13:01:48.571 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 13.236772802000001 2023-11-28T13:01:48.571 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:01:48.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fdb275700 1 -- 172.21.15.6:0/3324759552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6fb80682f0 msgr2=0x7f6fb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fdb275700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6fb80682f0 0x7f6fb806a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6fc40090b0 tx=0x7f6fc4008030 comp rx=0 tx=0).stop 2023-11-28T13:01:48.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fdb275700 1 -- 172.21.15.6:0/3324759552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 msgr2=0x7f6fd40b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fdb275700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 0x7f6fd40b3840 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000e870 tx=0x7f6fd000ec30 comp rx=0 tx=0).stop 2023-11-28T13:01:48.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fdb275700 1 -- 172.21.15.6:0/3324759552 shutdown_connections 2023-11-28T13:01:48.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fdb275700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6fd415db70 0x7f6fd40b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fdb275700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6fb80682f0 0x7f6fb806a7a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fdb275700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6fd40ce890 0x7f6fd41523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fdb275700 1 --2- 172.21.15.6:0/3324759552 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fd4159590 0x7f6fd40b3840 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.557+0000 7f6fdb275700 1 -- 172.21.15.6:0/3324759552 >> 172.21.15.6:0/3324759552 conn(0x7f6fd40b1650 msgr2=0x7f6fd415bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.558+0000 7f6fdb275700 1 -- 172.21.15.6:0/3324759552 shutdown_connections 2023-11-28T13:01:48.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.558+0000 7f6fdb275700 1 -- 172.21.15.6:0/3324759552 wait complete. 2023-11-28T13:01:48.574 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:01:48.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.697+0000 7fd957986700 1 Processor -- start 2023-11-28T13:01:48.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.698+0000 7fd957986700 1 -- start start 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.698+0000 7fd957986700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd95015c990 0x7fd9500cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.698+0000 7fd957986700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd950161090 0x7fd9500ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.698+0000 7fd957986700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd9500caeb0 0x7fd9500cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.698+0000 7fd957986700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9500ce340 con 0x7fd95015c990 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.698+0000 7fd957986700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9500cebe0 con 0x7fd9500caeb0 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.698+0000 7fd957986700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500cb3f0 con 0x7fd950161090 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955722700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd95015c990 0x7fd9500cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955722700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd95015c990 0x7fd9500cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60578/0 (socket says 172.21.15.6:60578) 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955722700 1 -- 172.21.15.6:0/3054649280 learned_addr learned my addr 172.21.15.6:0/3054649280 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955f23700 1 --2- 172.21.15.6:0/3054649280 >> v2:172.21.15.61:3300/0 conn(0x7fd9500caeb0 0x7fd9500cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd954f21700 1 --2- 172.21.15.6:0/3054649280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd950161090 0x7fd9500ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955722700 1 -- 172.21.15.6:0/3054649280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd950161090 msgr2=0x7fd9500ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955722700 1 --2- 172.21.15.6:0/3054649280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd950161090 0x7fd9500ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955722700 1 -- 172.21.15.6:0/3054649280 >> v2:172.21.15.61:3300/0 conn(0x7fd9500caeb0 msgr2=0x7fd9500cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955722700 1 --2- 172.21.15.6:0/3054649280 >> v2:172.21.15.61:3300/0 conn(0x7fd9500caeb0 0x7fd9500cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955722700 1 -- 172.21.15.6:0/3054649280 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9500ce590 con 0x7fd95015c990 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955f23700 1 --2- 172.21.15.6:0/3054649280 >> v2:172.21.15.61:3300/0 conn(0x7fd9500caeb0 0x7fd9500cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd955722700 1 --2- 172.21.15.6:0/3054649280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 0x7fd9500cf070 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd9500ca080 tx=0x7fd94c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc06134875a3811b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd943fff700 1 -- 172.21.15.6:0/3054649280 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00b870 con 0x7fd95015c990 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd943fff700 1 -- 172.21.15.6:0/3054649280 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c00b9d0 con 0x7fd95015c990 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.699+0000 7fd943fff700 1 -- 172.21.15.6:0/3054649280 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00bd60 con 0x7fd95015c990 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.700+0000 7fd957986700 1 -- 172.21.15.6:0/3054649280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 msgr2=0x7fd9500cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.700+0000 7fd957986700 1 --2- 172.21.15.6:0/3054649280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 0x7fd9500cf070 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd9500ca080 tx=0x7fd94c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.700+0000 7fd957986700 1 -- 172.21.15.6:0/3054649280 shutdown_connections 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.700+0000 7fd957986700 1 --2- 172.21.15.6:0/3054649280 >> v2:172.21.15.61:3300/0 conn(0x7fd9500caeb0 0x7fd9500cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.700+0000 7fd957986700 1 --2- 172.21.15.6:0/3054649280 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 0x7fd9500cf070 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.700+0000 7fd957986700 1 --2- 172.21.15.6:0/3054649280 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd950161090 0x7fd9500ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.700+0000 7fd957986700 1 -- 172.21.15.6:0/3054649280 >> 172.21.15.6:0/3054649280 conn(0x7fd9500b1650 msgr2=0x7fd9500b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.700+0000 7fd957986700 1 -- 172.21.15.6:0/3054649280 shutdown_connections 2023-11-28T13:01:48.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.700+0000 7fd957986700 1 -- 172.21.15.6:0/3054649280 wait complete. 2023-11-28T13:01:48.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.701+0000 7fd957986700 1 Processor -- start 2023-11-28T13:01:48.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.701+0000 7fd957986700 1 -- start start 2023-11-28T13:01:48.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd957986700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9500caeb0 0x7fd9500b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd957986700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 0x7fd9500abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd957986700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd950161090 0x7fd9500ac340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd957986700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500b4410 con 0x7fd95015c990 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd957986700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500c37a0 con 0x7fd950161090 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd957986700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500b3dc0 con 0x7fd9500caeb0 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd954f21700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 0x7fd9500abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd954f21700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 0x7fd9500abe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60586/0 (socket says 172.21.15.6:60586) 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd954f21700 1 -- 172.21.15.6:0/301528830 learned_addr learned my addr 172.21.15.6:0/301528830 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd955722700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9500caeb0 0x7fd9500b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd955f23700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd950161090 0x7fd9500ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd954f21700 1 -- 172.21.15.6:0/301528830 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9500caeb0 msgr2=0x7fd9500b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd954f21700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9500caeb0 0x7fd9500b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd954f21700 1 -- 172.21.15.6:0/301528830 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd950161090 msgr2=0x7fd9500ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd954f21700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd950161090 0x7fd9500ac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.702+0000 7fd954f21700 1 -- 172.21.15.6:0/301528830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c009550 con 0x7fd95015c990 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.703+0000 7fd954f21700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 0x7fd9500abe00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd94400d800 tx=0x7fd94400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.703+0000 7fd941ffb700 1 -- 172.21.15.6:0/301528830 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9440126b0 con 0x7fd95015c990 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.703+0000 7fd941ffb700 1 -- 172.21.15.6:0/301528830 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd944012810 con 0x7fd95015c990 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.703+0000 7fd957986700 1 -- 172.21.15.6:0/301528830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9500cf5b0 con 0x7fd95015c990 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.703+0000 7fd941ffb700 1 -- 172.21.15.6:0/301528830 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944012ba0 con 0x7fd95015c990 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.703+0000 7fd957986700 1 -- 172.21.15.6:0/301528830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9500ce870 con 0x7fd95015c990 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.704+0000 7fd957986700 1 -- 172.21.15.6:0/301528830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9500cf740 con 0x7fd95015c990 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.704+0000 7fd941ffb700 1 -- 172.21.15.6:0/301528830 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd944012d00 con 0x7fd95015c990 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.704+0000 7fd941ffb700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9340682f0 0x7fd93406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.705+0000 7fd941ffb700 1 -- 172.21.15.6:0/301528830 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd944085fa0 con 0x7fd95015c990 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.705+0000 7fd955722700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9340682f0 0x7fd93406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:48.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.705+0000 7fd955722700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9340682f0 0x7fd93406a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd94c000c00 tx=0x7fd94c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:48.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.709+0000 7fd941ffb700 1 -- 172.21.15.6:0/301528830 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd9440526c0 con 0x7fd95015c990 2023-11-28T13:01:48.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.946+0000 7fd957986700 1 -- 172.21.15.6:0/301528830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd9500675a0 con 0x7fd95015c990 2023-11-28T13:01:48.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.947+0000 7fd941ffb700 1 -- 172.21.15.6:0/301528830 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd944055ce0 con 0x7fd95015c990 2023-11-28T13:01:48.948 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:01:48.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 -- 172.21.15.6:0/301528830 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9340682f0 msgr2=0x7fd93406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9340682f0 0x7fd93406a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd94c000c00 tx=0x7fd94c008030 comp rx=0 tx=0).stop 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 -- 172.21.15.6:0/301528830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 msgr2=0x7fd9500abe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 0x7fd9500abe00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd94400d800 tx=0x7fd94400db10 comp rx=0 tx=0).stop 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 -- 172.21.15.6:0/301528830 shutdown_connections 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd950161090 0x7fd9500ac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd9340682f0 0x7fd93406a7a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd9500caeb0 0x7fd9500b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 --2- 172.21.15.6:0/301528830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd95015c990 0x7fd9500abe00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 -- 172.21.15.6:0/301528830 >> 172.21.15.6:0/301528830 conn(0x7fd9500b1650 msgr2=0x7fd95006c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 -- 172.21.15.6:0/301528830 shutdown_connections 2023-11-28T13:01:48.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:48.949+0000 7fd957986700 1 -- 172.21.15.6:0/301528830 wait complete. 2023-11-28T13:01:48.962 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:01:48.963 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:01:48.963 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-28T13:01:48.963 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:01:48.963 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:48.963 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:01:48.963 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T13:01:48.963 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:01:48.963 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:01:48.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:48.963 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-28T13:01:48.982 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:01:48.983 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42254sec preferred_lft 42254sec 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 8e:02:21:fe:ce:1c brd ff:ff:ff:ff:ff:ff 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::68d7:25ff:fe50:263/64 scope link 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:48.984 INFO:teuthology.orchestra.run.smithi006.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:01:48.985 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether ba:07:72:c8:31:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:01:48.985 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::b807:72ff:fec8:3118/64 scope link 2023-11-28T13:01:48.985 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:48.985 INFO:teuthology.orchestra.run.smithi006.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:01:48.985 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 8e:02:21:fe:ce:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:01:48.985 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::8c02:21ff:fefe:ce1c/64 scope link 2023-11-28T13:01:48.985 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:01:48.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:48.985 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:01:49.049 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-28T13:01:49.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:49.051 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-28T13:01:49.111 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-28T13:01:49.111 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:01:49.111 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T13:01:49.111 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T13:01:49.111 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:01:49.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:49.112 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:01:49.112 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:01:49.112 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T13:01:49.112 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T13:01:49.112 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T13:01:49.112 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:01:49.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:49.218 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-28T13:01:49.235 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:01:49.235 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:01:49.235 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:01:49.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:49.236 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2023-11-28T13:01:49.302 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T13:01:49.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:01:49.303 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:49.359 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:01:49.359 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:01:49.359 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:49.416 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:01:49.484 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:01:49.484 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:01:49.552 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:01:49.553 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:01:49.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:49.553 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:01:49.611 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T13:01:49.611 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T13:01:49.653 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:01:49.680 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:01:49.680 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:01:49.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:2023-11-28T13:01:49.741+0000 7fe749c1e5c0 -1 init, newargv = 0x55c792cd3a90 newargc=15 2023-11-28T13:01:49.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[122161]: starting ceph client 2023-11-28T13:01:49.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[122161]: starting fuse 2023-11-28T13:01:49.749 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:01:49.751 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:01:49.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:49.751 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:01:49.809 INFO:teuthology.orchestra.run.smithi006.stdout:47 2023-11-28T13:01:49.809 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T13:01:49.809 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:49.809 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2023-11-28T13:01:49.809 DEBUG:teuthology.orchestra.run.smithi006:> import re 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2023-11-28T13:01:49.810 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> return f 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.0")) 2023-11-28T13:01:49.811 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:01:49.933 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.0.122161.asok 2023-11-28T13:01:50.116 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.122161.asok 2023-11-28T13:01:50.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:50.117 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.122161.asok status 2023-11-28T13:01:50.284 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:01:50.284 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2023-11-28T13:01:50.284 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T13:01:50.284 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T13:01:50.284 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "0", 2023-11-28T13:01:50.284 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2023-11-28T13:01:50.284 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T13:01:50.284 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "122161", 2023-11-28T13:01:50.284 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2023-11-28T13:01:50.285 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:50.285 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2023-11-28T13:01:50.285 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2023-11-28T13:01:50.285 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 5712, 2023-11-28T13:01:50.285 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2023-11-28T13:01:50.285 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2023-11-28T13:01:50.285 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2023-11-28T13:01:50.285 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 5712 2023-11-28T13:01:50.285 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:50.286 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:01:50.286 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:01:50.286 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:01:50.286 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3201363014 2023-11-28T13:01:50.286 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:01:50.286 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:50.286 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:01:50.286 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:01:50.286 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:01:50.286 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3201363014 2023-11-28T13:01:50.287 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:01:50.287 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.5712 192.168.0.1:0/3201363014", 2023-11-28T13:01:50.287 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.1:0/3201363014", 2023-11-28T13:01:50.287 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2023-11-28T13:01:50.287 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 16, 2023-11-28T13:01:50.287 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 40, 2023-11-28T13:01:50.287 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2023-11-28T13:01:50.287 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2023-11-28T13:01:50.287 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2023-11-28T13:01:50.287 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:01:50.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:50.294 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2023-11-28T13:01:50.294 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:50.312 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2023-11-28T13:01:50.313 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:50.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:50.313 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:50.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:50.382 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-28T13:01:50.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:50.446 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-28T13:01:50.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:50.506 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-28T13:01:50.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:50.565 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-28T13:01:50.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:50.628 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-28T13:01:50.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:50.687 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-28T13:01:50.753 INFO:teuthology.orchestra.run.smithi006.stdout:./ 2023-11-28T13:01:50.753 INFO:teuthology.orchestra.run.smithi006.stdout:./rfile 2023-11-28T13:01:50.754 INFO:teuthology.orchestra.run.smithi006.stdout:./parentdir 2023-11-28T13:01:50.754 INFO:teuthology.orchestra.run.smithi006.stdout:./parentdir/pfile 2023-11-28T13:01:50.754 INFO:teuthology.orchestra.run.smithi006.stdout:./parentdir/childdir 2023-11-28T13:01:50.754 INFO:teuthology.orchestra.run.smithi006.stdout:./parentdir/childdir/cfile 2023-11-28T13:01:50.754 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:50.754 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:01:50.755 DEBUG:teuthology.orchestra.run.smithi006:> import stat 2023-11-28T13:01:50.755 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:01:50.755 DEBUG:teuthology.orchestra.run.smithi006:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-28T13:01:50.755 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:01:50.862 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-28T13:01:51.051 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:51.051 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:01:51.051 DEBUG:teuthology.orchestra.run.smithi006:> import stat 2023-11-28T13:01:51.051 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:01:51.051 DEBUG:teuthology.orchestra.run.smithi006:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-28T13:01:51.051 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:01:51.160 INFO:teuthology.orchestra.run.smithi006.stdout:1099511627778 2023-11-28T13:01:51.349 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:51.349 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:01:51.349 DEBUG:teuthology.orchestra.run.smithi006:> import stat 2023-11-28T13:01:51.349 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:01:51.350 DEBUG:teuthology.orchestra.run.smithi006:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-28T13:01:51.350 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:01:51.458 INFO:teuthology.orchestra.run.smithi006.stdout:1099511627776 2023-11-28T13:01:51.647 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:51.647 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:01:51.647 DEBUG:teuthology.orchestra.run.smithi006:> import stat 2023-11-28T13:01:51.647 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:01:51.647 DEBUG:teuthology.orchestra.run.smithi006:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-28T13:01:51.647 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:01:51.757 INFO:teuthology.orchestra.run.smithi006.stdout:1099511628279 2023-11-28T13:01:51.945 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:51.945 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:01:51.945 DEBUG:teuthology.orchestra.run.smithi006:> import stat 2023-11-28T13:01:51.945 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:01:51.946 DEBUG:teuthology.orchestra.run.smithi006:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-28T13:01:51.946 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:01:52.054 INFO:teuthology.orchestra.run.smithi006.stdout:1099511627777 2023-11-28T13:01:52.243 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:01:52.243 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:01:52.243 DEBUG:teuthology.orchestra.run.smithi006:> import stat 2023-11-28T13:01:52.243 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:01:52.243 DEBUG:teuthology.orchestra.run.smithi006:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-28T13:01:52.243 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:01:52.352 INFO:teuthology.orchestra.run.smithi006.stdout:1099511628280 2023-11-28T13:01:52.541 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:01:52.541 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:52.600 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:52.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.765+0000 7f4d230af700 1 Processor -- start 2023-11-28T13:01:52.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.765+0000 7f4d230af700 1 -- start start 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.766+0000 7f4d230af700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4d1c15c990 0x7f4d1c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.766+0000 7f4d230af700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c0ce340 con 0x7f4d1c161090 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.766+0000 7f4d230af700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c0cebe0 con 0x7f4d1c0cc310 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.766+0000 7f4d230af700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c0cb3f0 con 0x7f4d1c15c990 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.766+0000 7f4d20e4b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4d1c15c990 0x7f4d1c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.766+0000 7f4d20e4b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4d1c15c990 0x7f4d1c0cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60606/0 (socket says 172.21.15.6:60606) 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.766+0000 7f4d20e4b700 1 -- 172.21.15.6:0/1302734396 learned_addr learned my addr 172.21.15.6:0/1302734396 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.766+0000 7f4d1bfff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f4d1c161090 0x7f4d1c0ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47926/0 (socket says 172.21.15.6:47926) 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d20e4b700 1 -- 172.21.15.6:0/1302734396 >> v1:172.21.15.61:6789/0 conn(0x7f4d1c0cc310 legacy=0x7f4d1c0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d20e4b700 1 -- 172.21.15.6:0/1302734396 >> v1:172.21.15.6:6789/0 conn(0x7f4d1c161090 legacy=0x7f4d1c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d20e4b700 1 -- 172.21.15.6:0/1302734396 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0ce590 con 0x7f4d1c15c990 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d1b7fe700 1 -- 172.21.15.6:0/1302734396 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667789030 0 0) 0x7f4d1c0ce340 con 0x7f4d1c161090 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d20e4b700 1 --2- 172.21.15.6:0/1302734396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 0x7f4d1c0cf070 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4d100098f0 tx=0x7f4d10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2e46c83d9c933c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d1b7fe700 1 -- 172.21.15.6:0/1302734396 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10004230 con 0x7f4d1c15c990 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d1b7fe700 1 -- 172.21.15.6:0/1302734396 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d10004390 con 0x7f4d1c15c990 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d1b7fe700 1 -- 172.21.15.6:0/1302734396 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d100036a0 con 0x7f4d1c15c990 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d230af700 1 -- 172.21.15.6:0/1302734396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 msgr2=0x7f4d1c0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d230af700 1 --2- 172.21.15.6:0/1302734396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 0x7f4d1c0cf070 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4d100098f0 tx=0x7f4d10009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d230af700 1 -- 172.21.15.6:0/1302734396 shutdown_connections 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d230af700 1 --2- 172.21.15.6:0/1302734396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 0x7f4d1c0cf070 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.767+0000 7f4d230af700 1 -- 172.21.15.6:0/1302734396 >> 172.21.15.6:0/1302734396 conn(0x7f4d1c0b1650 msgr2=0x7f4d1c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.768+0000 7f4d230af700 1 -- 172.21.15.6:0/1302734396 shutdown_connections 2023-11-28T13:01:52.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.768+0000 7f4d230af700 1 -- 172.21.15.6:0/1302734396 wait complete. 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.769+0000 7f4d230af700 1 Processor -- start 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.769+0000 7f4d230af700 1 -- start start 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d230af700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d1c0cc310 0x7f4d1c14af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d230af700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 0x7f4d1c1e7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d230af700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d1c161090 0x7f4d1c1e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d230af700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c15e5c0 con 0x7f4d1c15c990 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d230af700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c14b4a0 con 0x7f4d1c0cc310 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d230af700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1e7500 con 0x7f4d1c161090 2023-11-28T13:01:52.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d1bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 0x7f4d1c1e7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d1bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 0x7f4d1c1e7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60612/0 (socket says 172.21.15.6:60612) 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d1bfff700 1 -- 172.21.15.6:0/2627841613 learned_addr learned my addr 172.21.15.6:0/2627841613 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d20e4b700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d1c0cc310 0x7f4d1c14af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d1bfff700 1 -- 172.21.15.6:0/2627841613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d1c161090 msgr2=0x7f4d1c1e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d2164c700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d1c161090 0x7f4d1c1e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d1bfff700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d1c161090 0x7f4d1c1e81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d1bfff700 1 -- 172.21.15.6:0/2627841613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d1c0cc310 msgr2=0x7f4d1c14af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d1bfff700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d1c0cc310 0x7f4d1c14af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d1bfff700 1 -- 172.21.15.6:0/2627841613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d10009550 con 0x7f4d1c15c990 2023-11-28T13:01:52.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d20e4b700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d1c0cc310 0x7f4d1c14af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d1bfff700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 0x7f4d1c1e7cb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800c550 tx=0x7f4d0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.770+0000 7f4d197fa700 1 -- 172.21.15.6:0/2627841613 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08004c80 con 0x7f4d1c15c990 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.771+0000 7f4d230af700 1 -- 172.21.15.6:0/2627841613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c0cb080 con 0x7f4d1c15c990 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.771+0000 7f4d197fa700 1 -- 172.21.15.6:0/2627841613 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d08004e00 con 0x7f4d1c15c990 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.771+0000 7f4d197fa700 1 -- 172.21.15.6:0/2627841613 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08007310 con 0x7f4d1c15c990 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.771+0000 7f4d230af700 1 -- 172.21.15.6:0/2627841613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c0ce870 con 0x7f4d1c15c990 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.771+0000 7f4d230af700 1 -- 172.21.15.6:0/2627841613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c0cf5b0 con 0x7f4d1c15c990 2023-11-28T13:01:52.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.772+0000 7f4d197fa700 1 -- 172.21.15.6:0/2627841613 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d0801a020 con 0x7f4d1c15c990 2023-11-28T13:01:52.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.772+0000 7f4d197fa700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 0x7f4d0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:52.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.772+0000 7f4d197fa700 1 -- 172.21.15.6:0/2627841613 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4d08085a50 con 0x7f4d1c15c990 2023-11-28T13:01:52.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.772+0000 7f4d20e4b700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 0x7f4d0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:52.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.773+0000 7f4d20e4b700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 0x7f4d0006a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009c40 tx=0x7f4d10008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:52.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.777+0000 7f4d197fa700 1 -- 172.21.15.6:0/2627841613 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d080520f0 con 0x7f4d1c15c990 2023-11-28T13:01:52.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.972+0000 7f4d230af700 1 -- 172.21.15.6:0/2627841613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d1c0cdc50 con 0x7f4d1c15c990 2023-11-28T13:01:52.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.974+0000 7f4d197fa700 1 -- 172.21.15.6:0/2627841613 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17974 (secure 0 0 0) 0x7f4d08055710 con 0x7f4d1c15c990 2023-11-28T13:01:52.975 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:52.975 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":40,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:43.596467+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:41.144348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:41.554568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:52.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 -- 172.21.15.6:0/2627841613 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 msgr2=0x7f4d0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 0x7f4d0006a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009c40 tx=0x7f4d10008030 comp rx=0 tx=0).stop 2023-11-28T13:01:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 -- 172.21.15.6:0/2627841613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 msgr2=0x7f4d1c1e7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 0x7f4d1c1e7cb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800c550 tx=0x7f4d0800c910 comp rx=0 tx=0).stop 2023-11-28T13:01:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 -- 172.21.15.6:0/2627841613 shutdown_connections 2023-11-28T13:01:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 0x7f4d0006a7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d1c161090 0x7f4d1c1e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d1c0cc310 0x7f4d1c14af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 --2- 172.21.15.6:0/2627841613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d1c15c990 0x7f4d1c1e7cb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 -- 172.21.15.6:0/2627841613 >> 172.21.15.6:0/2627841613 conn(0x7f4d1c0b1650 msgr2=0x7f4d1c0b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 -- 172.21.15.6:0/2627841613 shutdown_connections 2023-11-28T13:01:52.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:52.976+0000 7f4d230af700 1 -- 172.21.15.6:0/2627841613 wait complete. 2023-11-28T13:01:52.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:01:41.144348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:01:41.554568+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:01:52.990 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:53.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.116+0000 7fb62c92c700 1 Processor -- start 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.116+0000 7fb62c92c700 1 -- start start 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb62c92c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 0x7fb6280caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb62c92c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb6280cb3e0 0x7fb62806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb62c92c700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6280ce550 con 0x7fb6280b1630 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb62c92c700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6280cfa60 con 0x7fb6280cb3e0 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb62c92c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280ce830 con 0x7fb6280cf5f0 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb626d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb6280cb3e0 0x7fb62806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb62659c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb6280b1630 0x7fb6280b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47938/0 (socket says 172.21.15.6:47938) 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb62659c700 1 -- 172.21.15.6:0/1985955922 learned_addr learned my addr 172.21.15.6:0/1985955922 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb625d9b700 1 --2- 172.21.15.6:0/1985955922 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 0x7fb6280caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb626d9d700 1 -- 172.21.15.6:0/1985955922 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 msgr2=0x7fb6280caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:53.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb626d9d700 1 --2- 172.21.15.6:0/1985955922 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 0x7fb6280caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb626d9d700 1 -- 172.21.15.6:0/1985955922 >> v1:172.21.15.6:6789/0 conn(0x7fb6280b1630 legacy=0x7fb6280b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb626d9d700 1 -- 172.21.15.6:0/1985955922 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6280ceb10 con 0x7fb6280cb3e0 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb625d9b700 1 --2- 172.21.15.6:0/1985955922 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 0x7fb6280caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb626d9d700 1 --2- 172.21.15.6:0/1985955922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280cb3e0 0x7fb62806afc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb620009d90 tx=0x7fb62000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd8b6a7a6204906 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.117+0000 7fb62559a700 1 -- 172.21.15.6:0/1985955922 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62000e070 con 0x7fb6280cb3e0 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.118+0000 7fb62559a700 1 -- 172.21.15.6:0/1985955922 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62000bbd0 con 0x7fb6280cb3e0 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.118+0000 7fb62559a700 1 -- 172.21.15.6:0/1985955922 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620012430 con 0x7fb6280cb3e0 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.118+0000 7fb62c92c700 1 -- 172.21.15.6:0/1985955922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280cb3e0 msgr2=0x7fb62806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.118+0000 7fb62c92c700 1 --2- 172.21.15.6:0/1985955922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280cb3e0 0x7fb62806afc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb620009d90 tx=0x7fb62000b330 comp rx=0 tx=0).stop 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.118+0000 7fb62c92c700 1 -- 172.21.15.6:0/1985955922 shutdown_connections 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.118+0000 7fb62c92c700 1 --2- 172.21.15.6:0/1985955922 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280cb3e0 0x7fb62806afc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.118+0000 7fb62c92c700 1 --2- 172.21.15.6:0/1985955922 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 0x7fb6280caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.118+0000 7fb62c92c700 1 -- 172.21.15.6:0/1985955922 >> 172.21.15.6:0/1985955922 conn(0x7fb6280cc630 msgr2=0x7fb6280cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.118+0000 7fb62c92c700 1 -- 172.21.15.6:0/1985955922 shutdown_connections 2023-11-28T13:01:53.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.118+0000 7fb62c92c700 1 -- 172.21.15.6:0/1985955922 wait complete. 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.120+0000 7fb62c92c700 1 Processor -- start 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.120+0000 7fb62c92c700 1 -- start start 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.120+0000 7fb62c92c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280b1630 0x7fb6281f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.120+0000 7fb62c92c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6280cb3e0 0x7fb6280b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.120+0000 7fb62c92c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 0x7fb6281f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.120+0000 7fb62c92c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281f6590 con 0x7fb6280b1630 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.120+0000 7fb62c92c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62815e060 con 0x7fb6280cb3e0 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.120+0000 7fb62c92c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62815e340 con 0x7fb6280cf5f0 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.120+0000 7fb62659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280b1630 0x7fb6281f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb62659c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280b1630 0x7fb6281f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60624/0 (socket says 172.21.15.6:60624) 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb62659c700 1 -- 172.21.15.6:0/1262668216 learned_addr learned my addr 172.21.15.6:0/1262668216 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:53.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb625d9b700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6280cb3e0 0x7fb6280b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb626d9d700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 0x7fb6281f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb62659c700 1 -- 172.21.15.6:0/1262668216 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 msgr2=0x7fb6281f1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb62659c700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 0x7fb6281f1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb62659c700 1 -- 172.21.15.6:0/1262668216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6280cb3e0 msgr2=0x7fb6280b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb62659c700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6280cb3e0 0x7fb6280b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb62659c700 1 -- 172.21.15.6:0/1262668216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb620009a70 con 0x7fb6280b1630 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb62659c700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280b1630 0x7fb6281f6050 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb61800c520 tx=0x7fb61800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb616ffd700 1 -- 172.21.15.6:0/1262668216 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618005d70 con 0x7fb6280b1630 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb616ffd700 1 -- 172.21.15.6:0/1262668216 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6180070d0 con 0x7fb6280b1630 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb62c92c700 1 -- 172.21.15.6:0/1262668216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6280cf0d0 con 0x7fb6280b1630 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb616ffd700 1 -- 172.21.15.6:0/1262668216 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618007440 con 0x7fb6280b1630 2023-11-28T13:01:53.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.121+0000 7fb62c92c700 1 -- 172.21.15.6:0/1262668216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6280cedf0 con 0x7fb6280b1630 2023-11-28T13:01:53.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.122+0000 7fb62c92c700 1 -- 172.21.15.6:0/1262668216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6280ce140 con 0x7fb6280b1630 2023-11-28T13:01:53.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.123+0000 7fb616ffd700 1 -- 172.21.15.6:0/1262668216 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb618019030 con 0x7fb6280b1630 2023-11-28T13:01:53.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.123+0000 7fb616ffd700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6080682f0 0x7fb60806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.123+0000 7fb616ffd700 1 -- 172.21.15.6:0/1262668216 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb6180850a0 con 0x7fb6280b1630 2023-11-28T13:01:53.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.123+0000 7fb625d9b700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6080682f0 0x7fb60806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.124+0000 7fb625d9b700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6080682f0 0x7fb60806a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb61c006fd0 tx=0x7fb61c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.127+0000 7fb616ffd700 1 -- 172.21.15.6:0/1262668216 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb61804b5e0 con 0x7fb6280b1630 2023-11-28T13:01:53.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.319+0000 7fb62c92c700 1 -- 172.21.15.6:0/1262668216 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb6280739f0 con 0x7fb6280b1630 2023-11-28T13:01:53.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.320+0000 7fb616ffd700 1 -- 172.21.15.6:0/1262668216 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb6180170a0 con 0x7fb6280b1630 2023-11-28T13:01:53.321 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:53.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 -- 172.21.15.6:0/1262668216 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6080682f0 msgr2=0x7fb60806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6080682f0 0x7fb60806a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb61c006fd0 tx=0x7fb61c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:53.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 -- 172.21.15.6:0/1262668216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280b1630 msgr2=0x7fb6281f6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280b1630 0x7fb6281f6050 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb61800c520 tx=0x7fb61800c8e0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 -- 172.21.15.6:0/1262668216 shutdown_connections 2023-11-28T13:01:53.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb6080682f0 0x7fb60806a7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb6280cf5f0 0x7fb6281f1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6280b1630 0x7fb6281f6050 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 --2- 172.21.15.6:0/1262668216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6280cb3e0 0x7fb6280b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 -- 172.21.15.6:0/1262668216 >> 172.21.15.6:0/1262668216 conn(0x7fb6280cc630 msgr2=0x7fb6280b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 -- 172.21.15.6:0/1262668216 shutdown_connections 2023-11-28T13:01:53.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.322+0000 7fb62c92c700 1 -- 172.21.15.6:0/1262668216 wait complete. 2023-11-28T13:01:53.335 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:53.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.461+0000 7f4857204700 1 Processor -- start 2023-11-28T13:01:53.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.461+0000 7f4857204700 1 -- start start 2023-11-28T13:01:53.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f4857204700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f48500b1d40 0x7f48500cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f4857204700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48500ceff0 con 0x7f485015dbf0 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f4857204700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48500cf2d0 con 0x7f4850159500 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f4857204700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485015e090 con 0x7f48500b1d40 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f48557a1700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f485015dbf0 0x7f485006bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47956/0 (socket says 172.21.15.6:47956) 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f48557a1700 1 -- 172.21.15.6:0/3482830527 learned_addr learned my addr 172.21.15.6:0/3482830527 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f484ffff700 1 --2- 172.21.15.6:0/3482830527 >> v2:172.21.15.61:3300/0 conn(0x7f48500b1d40 0x7f48500cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508732162 0 0) 0x7f48500ceff0 con 0x7f485015dbf0 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48400035b0 con 0x7f485015dbf0 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1889907189 0 0) 0x7f48400035b0 con 0x7f485015dbf0 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 >> v2:172.21.15.61:3300/0 conn(0x7f48500b1d40 msgr2=0x7f48500cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f484f7fe700 1 --2- 172.21.15.6:0/3482830527 >> v2:172.21.15.61:3300/0 conn(0x7f48500b1d40 0x7f48500cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.462+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 >> v1:172.21.15.61:6789/0 conn(0x7f4850159500 legacy=0x7f48500b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485015d110 con 0x7f485015dbf0 2023-11-28T13:01:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f48480019f0 con 0x7f485015dbf0 2023-11-28T13:01:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 >> v1:172.21.15.6:6789/0 conn(0x7f485015dbf0 legacy=0x7f485006bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4840003a00 0x7f4840003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4840004850 0x7f4840004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4840005200 0x7f484000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485015e090 con 0x7f4840003a00 2023-11-28T13:01:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484000c060 con 0x7f4840004850 2023-11-28T13:01:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484000c220 con 0x7f4840005200 2023-11-28T13:01:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f4854fa0700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4840003a00 0x7f4840003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484ffff700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4840004850 0x7f4840004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f48557a1700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4840005200 0x7f484000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f4854fa0700 1 -- 172.21.15.6:0/3482830527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4840005200 msgr2=0x7f484000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f4854fa0700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4840005200 0x7f484000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f4854fa0700 1 -- 172.21.15.6:0/3482830527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4840004850 msgr2=0x7f4840004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f4854fa0700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4840004850 0x7f4840004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f48557a1700 1 -- 172.21.15.6:0/3482830527 reap_dead start 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f4854fa0700 1 -- 172.21.15.6:0/3482830527 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400035b0 con 0x7f4840003a00 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484ffff700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4840004850 0x7f4840004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f4854fa0700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4840003a00 0x7f4840003e10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f48440017c0 tx=0x7f4840005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eaf78ba2981bd22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48440052d0 con 0x7f4840003a00 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4844005450 con 0x7f4840003a00 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.463+0000 7f484f7fe700 1 -- 172.21.15.6:0/3482830527 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48440038c0 con 0x7f4840003a00 2023-11-28T13:01:53.469 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.464+0000 7f4857204700 1 -- 172.21.15.6:0/3482830527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4840003a00 msgr2=0x7f4840003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.464+0000 7f4857204700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4840003a00 0x7f4840003e10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f48440017c0 tx=0x7f4840005880 comp rx=0 tx=0).stop 2023-11-28T13:01:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.464+0000 7f4857204700 1 -- 172.21.15.6:0/3482830527 shutdown_connections 2023-11-28T13:01:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.464+0000 7f4857204700 1 --2- 172.21.15.6:0/3482830527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4840003a00 0x7f4840003e10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.464+0000 7f4857204700 1 -- 172.21.15.6:0/3482830527 >> 172.21.15.6:0/3482830527 conn(0x7f48500cc630 msgr2=0x7f48500cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.464+0000 7f4857204700 1 -- 172.21.15.6:0/3482830527 shutdown_connections 2023-11-28T13:01:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.464+0000 7f4857204700 1 -- 172.21.15.6:0/3482830527 wait complete. 2023-11-28T13:01:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.465+0000 7f4857204700 1 Processor -- start 2023-11-28T13:01:53.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.465+0000 7f4857204700 1 -- start start 2023-11-28T13:01:53.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f4857204700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4840003a00 0x7f4850154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f4857204700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48500b29e0 0x7f48500b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f4857204700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48500b32f0 0x7f48500b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f4857204700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500ac480 con 0x7f48500b29e0 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f4857204700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500b58f0 con 0x7f4840003a00 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f4857204700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4850155190 con 0x7f48500b32f0 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f484ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48500b29e0 0x7f48500b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f484ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48500b29e0 0x7f48500b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60646/0 (socket says 172.21.15.6:60646) 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f484ffff700 1 -- 172.21.15.6:0/2073010378 learned_addr learned my addr 172.21.15.6:0/2073010378 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f4854fa0700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4840003a00 0x7f4850154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f48557a1700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48500b32f0 0x7f48500b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f484ffff700 1 -- 172.21.15.6:0/2073010378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48500b32f0 msgr2=0x7f48500b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f484ffff700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48500b32f0 0x7f48500b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f484ffff700 1 -- 172.21.15.6:0/2073010378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4840003a00 msgr2=0x7f4850154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f484ffff700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4840003a00 0x7f4850154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.466+0000 7f484ffff700 1 -- 172.21.15.6:0/2073010378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485015d6d0 con 0x7f48500b29e0 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.467+0000 7f484ffff700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48500b29e0 0x7f48500b4e70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f483c011540 tx=0x7f483c011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.467+0000 7f484d7fa700 1 -- 172.21.15.6:0/2073010378 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c004440 con 0x7f48500b29e0 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.467+0000 7f484d7fa700 1 -- 172.21.15.6:0/2073010378 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c0045a0 con 0x7f48500b29e0 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.467+0000 7f484d7fa700 1 -- 172.21.15.6:0/2073010378 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c0038e0 con 0x7f48500b29e0 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.467+0000 7f4857204700 1 -- 172.21.15.6:0/2073010378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485015d3f0 con 0x7f48500b29e0 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.467+0000 7f4857204700 1 -- 172.21.15.6:0/2073010378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48501598d0 con 0x7f48500b29e0 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.468+0000 7f4857204700 1 -- 172.21.15.6:0/2073010378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48500ac630 con 0x7f48500b29e0 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.468+0000 7f484d7fa700 1 -- 172.21.15.6:0/2073010378 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f483c00f070 con 0x7f48500b29e0 2023-11-28T13:01:53.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.469+0000 7f484d7fa700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48340682f0 0x7f483406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.469+0000 7f4854fa0700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48340682f0 0x7f483406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.469+0000 7f484d7fa700 1 -- 172.21.15.6:0/2073010378 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f483c0860b0 con 0x7f48500b29e0 2023-11-28T13:01:53.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.469+0000 7f4854fa0700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48340682f0 0x7f483406a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4844000c00 tx=0x7f4844009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.473+0000 7f484d7fa700 1 -- 172.21.15.6:0/2073010378 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f48500ac630 con 0x7f48500b29e0 2023-11-28T13:01:53.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.666+0000 7f4857204700 1 -- 172.21.15.6:0/2073010378 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48500739f0 con 0x7f48500b29e0 2023-11-28T13:01:53.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.667+0000 7f484d7fa700 1 -- 172.21.15.6:0/2073010378 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f483c052770 con 0x7f48500b29e0 2023-11-28T13:01:53.669 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:53.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 -- 172.21.15.6:0/2073010378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48340682f0 msgr2=0x7f483406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48340682f0 0x7f483406a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4844000c00 tx=0x7f4844009030 comp rx=0 tx=0).stop 2023-11-28T13:01:53.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 -- 172.21.15.6:0/2073010378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48500b29e0 msgr2=0x7f48500b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48500b29e0 0x7f48500b4e70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f483c011540 tx=0x7f483c011850 comp rx=0 tx=0).stop 2023-11-28T13:01:53.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 -- 172.21.15.6:0/2073010378 shutdown_connections 2023-11-28T13:01:53.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48340682f0 0x7f483406a7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48500b32f0 0x7f48500b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4840003a00 0x7f4850154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 --2- 172.21.15.6:0/2073010378 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48500b29e0 0x7f48500b4e70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.672 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 -- 172.21.15.6:0/2073010378 >> 172.21.15.6:0/2073010378 conn(0x7f48500cc630 msgr2=0x7f48500cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 -- 172.21.15.6:0/2073010378 shutdown_connections 2023-11-28T13:01:53.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.669+0000 7f4857204700 1 -- 172.21.15.6:0/2073010378 wait complete. 2023-11-28T13:01:53.682 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:53.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.808+0000 7f71bcaa9700 1 Processor -- start 2023-11-28T13:01:53.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.808+0000 7f71bcaa9700 1 -- start start 2023-11-28T13:01:53.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.809+0000 7f71bcaa9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f71b80b1630 0x7f71b80cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.809+0000 7f71bcaa9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f71b806afc0 0x7f71b806b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.809+0000 7f71bcaa9700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b80ce550 con 0x7f71b80b1a00 2023-11-28T13:01:53.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.809+0000 7f71bcaa9700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b80ce830 con 0x7f71b80b1630 2023-11-28T13:01:53.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.809+0000 7f71bcaa9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b80ceb10 con 0x7f71b806afc0 2023-11-28T13:01:53.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.809+0000 7f71b659c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f71b80b1630 0x7f71b80cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.809+0000 7f71b5d9b700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f71b80b1a00 0x7f71b80caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47972/0 (socket says 172.21.15.6:47972) 2023-11-28T13:01:53.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.809+0000 7f71b5d9b700 1 -- 172.21.15.6:0/3669914368 learned_addr learned my addr 172.21.15.6:0/3669914368 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:53.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.809+0000 7f71b6d9d700 1 --2- 172.21.15.6:0/3669914368 >> v2:172.21.15.61:3300/0 conn(0x7f71b806afc0 0x7f71b806b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.812 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71b659c700 1 -- 172.21.15.6:0/3669914368 >> v2:172.21.15.61:3300/0 conn(0x7f71b806afc0 msgr2=0x7f71b806b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71b659c700 1 --2- 172.21.15.6:0/3669914368 >> v2:172.21.15.61:3300/0 conn(0x7f71b806afc0 0x7f71b806b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71b659c700 1 -- 172.21.15.6:0/3669914368 >> v1:172.21.15.6:6789/0 conn(0x7f71b80b1a00 legacy=0x7f71b80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71b659c700 1 -- 172.21.15.6:0/3669914368 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80cedf0 con 0x7f71b80b1630 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71b559a700 1 -- 172.21.15.6:0/3669914368 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667111719 0 0) 0x7f71b80ce550 con 0x7f71b80b1a00 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71b659c700 1 --2- 172.21.15.6:0/3669914368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 0x7f71b80cf5f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009840 tx=0x7f71ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09dbc876514ea16 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71b559a700 1 -- 172.21.15.6:0/3669914368 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac011830 con 0x7f71b80b1630 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71b559a700 1 -- 172.21.15.6:0/3669914368 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac011990 con 0x7f71b80b1630 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71b559a700 1 -- 172.21.15.6:0/3669914368 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac011d20 con 0x7f71b80b1630 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71bcaa9700 1 -- 172.21.15.6:0/3669914368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 msgr2=0x7f71b80cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.810+0000 7f71bcaa9700 1 --2- 172.21.15.6:0/3669914368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 0x7f71b80cf5f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009840 tx=0x7f71ac009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.811+0000 7f71bcaa9700 1 -- 172.21.15.6:0/3669914368 shutdown_connections 2023-11-28T13:01:53.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.811+0000 7f71bcaa9700 1 --2- 172.21.15.6:0/3669914368 >> v2:172.21.15.61:3300/0 conn(0x7f71b806afc0 0x7f71b806b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.811+0000 7f71bcaa9700 1 --2- 172.21.15.6:0/3669914368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 0x7f71b80cf5f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.811+0000 7f71bcaa9700 1 -- 172.21.15.6:0/3669914368 >> 172.21.15.6:0/3669914368 conn(0x7f71b80cc630 msgr2=0x7f71b80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:53.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.811+0000 7f71bcaa9700 1 -- 172.21.15.6:0/3669914368 shutdown_connections 2023-11-28T13:01:53.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.811+0000 7f71bcaa9700 1 -- 172.21.15.6:0/3669914368 wait complete. 2023-11-28T13:01:53.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.812+0000 7f71bcaa9700 1 Processor -- start 2023-11-28T13:01:53.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.812+0000 7f71bcaa9700 1 -- start start 2023-11-28T13:01:53.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71bcaa9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71b806afc0 0x7f71b815b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71bcaa9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 0x7f71b81f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71bcaa9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71b80b1a00 0x7f71b80b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71bcaa9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b81e5fb0 con 0x7f71b80b1630 2023-11-28T13:01:53.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71bcaa9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80b5630 con 0x7f71b80b1a00 2023-11-28T13:01:53.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71bcaa9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80b5910 con 0x7f71b806afc0 2023-11-28T13:01:53.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 0x7f71b81f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b5d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 0x7f71b81f2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60660/0 (socket says 172.21.15.6:60660) 2023-11-28T13:01:53.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b5d9b700 1 -- 172.21.15.6:0/2515958530 learned_addr learned my addr 172.21.15.6:0/2515958530 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:53.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b6d9d700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71b80b1a00 0x7f71b80b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b659c700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71b806afc0 0x7f71b815b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b5d9b700 1 -- 172.21.15.6:0/2515958530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71b806afc0 msgr2=0x7f71b815b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b5d9b700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71b806afc0 0x7f71b815b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b5d9b700 1 -- 172.21.15.6:0/2515958530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71b80b1a00 msgr2=0x7f71b80b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b5d9b700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71b80b1a00 0x7f71b80b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b5d9b700 1 -- 172.21.15.6:0/2515958530 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0094a0 con 0x7f71b80b1630 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71b5d9b700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 0x7f71b81f2300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f71a800c550 tx=0x7f71a800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.813+0000 7f71a6ffd700 1 -- 172.21.15.6:0/2515958530 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8004c80 con 0x7f71b80b1630 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.814+0000 7f71a6ffd700 1 -- 172.21.15.6:0/2515958530 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a8004e00 con 0x7f71b80b1630 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.814+0000 7f71bcaa9700 1 -- 172.21.15.6:0/2515958530 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b80b1ea0 con 0x7f71b80b1630 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.814+0000 7f71a6ffd700 1 -- 172.21.15.6:0/2515958530 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a8007310 con 0x7f71b80b1630 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.814+0000 7f71bcaa9700 1 -- 172.21.15.6:0/2515958530 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b80cf0d0 con 0x7f71b80b1630 2023-11-28T13:01:53.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.814+0000 7f71bcaa9700 1 -- 172.21.15.6:0/2515958530 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b80cb5b0 con 0x7f71b80b1630 2023-11-28T13:01:53.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.815+0000 7f71a6ffd700 1 -- 172.21.15.6:0/2515958530 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f71a801a020 con 0x7f71b80b1630 2023-11-28T13:01:53.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.815+0000 7f71a6ffd700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71980682f0 0x7f719806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:53.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.815+0000 7f71a6ffd700 1 -- 172.21.15.6:0/2515958530 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f71a8085800 con 0x7f71b80b1630 2023-11-28T13:01:53.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.815+0000 7f71b659c700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71980682f0 0x7f719806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:53.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.816+0000 7f71b659c700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71980682f0 0x7f719806a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0090b0 tx=0x7f71ac01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:53.820 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:53.820+0000 7f71a6ffd700 1 -- 172.21.15.6:0/2515958530 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f71a8051f20 con 0x7f71b80b1630 2023-11-28T13:01:54.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.015+0000 7f71bcaa9700 1 -- 172.21.15.6:0/2515958530 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71b80739f0 con 0x7f71b80b1630 2023-11-28T13:01:54.017 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.016+0000 7f71a6ffd700 1 -- 172.21.15.6:0/2515958530 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f71a8055540 con 0x7f71b80b1630 2023-11-28T13:01:54.017 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:54.019 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 -- 172.21.15.6:0/2515958530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71980682f0 msgr2=0x7f719806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71980682f0 0x7f719806a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0090b0 tx=0x7f71ac01c040 comp rx=0 tx=0).stop 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 -- 172.21.15.6:0/2515958530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 msgr2=0x7f71b81f2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 0x7f71b81f2300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f71a800c550 tx=0x7f71a800c860 comp rx=0 tx=0).stop 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 -- 172.21.15.6:0/2515958530 shutdown_connections 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71b80b1a00 0x7f71b80b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71980682f0 0x7f719806a7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f71b806afc0 0x7f71b815b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 --2- 172.21.15.6:0/2515958530 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f71b80b1630 0x7f71b81f2300 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 -- 172.21.15.6:0/2515958530 >> 172.21.15.6:0/2515958530 conn(0x7f71b80cc630 msgr2=0x7f71b806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 -- 172.21.15.6:0/2515958530 shutdown_connections 2023-11-28T13:01:54.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.018+0000 7f71bcaa9700 1 -- 172.21.15.6:0/2515958530 wait complete. 2023-11-28T13:01:54.031 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:54.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.158+0000 7f6f90565700 1 Processor -- start 2023-11-28T13:01:54.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.158+0000 7f6f90565700 1 -- start start 2023-11-28T13:01:54.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.159+0000 7f6f90565700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6f88159500 0x7f6f880cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.159+0000 7f6f90565700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6f8815dbf0 0x7f6f880ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.159+0000 7f6f90565700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8815e040 con 0x7f6f880b1630 2023-11-28T13:01:54.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.159+0000 7f6f90565700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f880cede0 con 0x7f6f88159500 2023-11-28T13:01:54.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.159+0000 7f6f90565700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f880ceff0 con 0x7f6f8815dbf0 2023-11-28T13:01:54.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.159+0000 7f6f8e301700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6f88159500 0x7f6f880cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.159+0000 7f6f8eb02700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6f880b1630 0x7f6f8806bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:47990/0 (socket says 172.21.15.6:47990) 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.159+0000 7f6f8eb02700 1 -- 172.21.15.6:0/2205795296 learned_addr learned my addr 172.21.15.6:0/2205795296 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.159+0000 7f6f8db00700 1 --2- 172.21.15.6:0/2205795296 >> v2:172.21.15.61:3300/0 conn(0x7f6f8815dbf0 0x7f6f880ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f8e301700 1 -- 172.21.15.6:0/2205795296 >> v2:172.21.15.61:3300/0 conn(0x7f6f8815dbf0 msgr2=0x7f6f880ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f8e301700 1 --2- 172.21.15.6:0/2205795296 >> v2:172.21.15.61:3300/0 conn(0x7f6f8815dbf0 0x7f6f880ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f8e301700 1 -- 172.21.15.6:0/2205795296 >> v1:172.21.15.6:6789/0 conn(0x7f6f880b1630 legacy=0x7f6f8806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f8d2ff700 1 -- 172.21.15.6:0/2205795296 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987474617 0 0) 0x7f6f8815e040 con 0x7f6f880b1630 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f8e301700 1 -- 172.21.15.6:0/2205795296 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f880cf2d0 con 0x7f6f88159500 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f8e301700 1 --2- 172.21.15.6:0/2205795296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f88159500 0x7f6f880cd140 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6f880d0c50 tx=0x7f6f78009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5a2e4ac1c232da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f8d2ff700 1 -- 172.21.15.6:0/2205795296 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78011830 con 0x7f6f88159500 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f8d2ff700 1 -- 172.21.15.6:0/2205795296 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f78011990 con 0x7f6f88159500 2023-11-28T13:01:54.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f8d2ff700 1 -- 172.21.15.6:0/2205795296 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78011d20 con 0x7f6f88159500 2023-11-28T13:01:54.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f90565700 1 -- 172.21.15.6:0/2205795296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f88159500 msgr2=0x7f6f880cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f90565700 1 --2- 172.21.15.6:0/2205795296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f88159500 0x7f6f880cd140 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6f880d0c50 tx=0x7f6f78009b20 comp rx=0 tx=0).stop 2023-11-28T13:01:54.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f90565700 1 -- 172.21.15.6:0/2205795296 shutdown_connections 2023-11-28T13:01:54.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f90565700 1 --2- 172.21.15.6:0/2205795296 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f88159500 0x7f6f880cd140 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f90565700 1 --2- 172.21.15.6:0/2205795296 >> v2:172.21.15.61:3300/0 conn(0x7f6f8815dbf0 0x7f6f880ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.160+0000 7f6f90565700 1 -- 172.21.15.6:0/2205795296 >> 172.21.15.6:0/2205795296 conn(0x7f6f880cc630 msgr2=0x7f6f880cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:54.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.161+0000 7f6f90565700 1 -- 172.21.15.6:0/2205795296 shutdown_connections 2023-11-28T13:01:54.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.161+0000 7f6f90565700 1 -- 172.21.15.6:0/2205795296 wait complete. 2023-11-28T13:01:54.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.162+0000 7f6f90565700 1 Processor -- start 2023-11-28T13:01:54.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.162+0000 7f6f90565700 1 -- start start 2023-11-28T13:01:54.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.162+0000 7f6f90565700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f880b1630 0x7f6f880acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.162+0000 7f6f90565700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f88159500 0x7f6f88154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.162+0000 7f6f90565700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f8815dbf0 0x7f6f880caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f90565700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880ad260 con 0x7f6f8815dbf0 2023-11-28T13:01:54.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f90565700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880b2b10 con 0x7f6f88159500 2023-11-28T13:01:54.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f90565700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f88154860 con 0x7f6f880b1630 2023-11-28T13:01:54.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8eb02700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f8815dbf0 0x7f6f880caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8eb02700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f8815dbf0 0x7f6f880caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60676/0 (socket says 172.21.15.6:60676) 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8eb02700 1 -- 172.21.15.6:0/2762176088 learned_addr learned my addr 172.21.15.6:0/2762176088 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8db00700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f88159500 0x7f6f88154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8e301700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f880b1630 0x7f6f880acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8eb02700 1 -- 172.21.15.6:0/2762176088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f880b1630 msgr2=0x7f6f880acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8eb02700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f880b1630 0x7f6f880acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8eb02700 1 -- 172.21.15.6:0/2762176088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f88159500 msgr2=0x7f6f88154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8eb02700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f88159500 0x7f6f88154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8eb02700 1 -- 172.21.15.6:0/2762176088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f780094a0 con 0x7f6f8815dbf0 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f8eb02700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f8815dbf0 0x7f6f880caf70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000cb60 tx=0x7f6f8000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f7effd700 1 -- 172.21.15.6:0/2762176088 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8000ab50 con 0x7f6f8815dbf0 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f7effd700 1 -- 172.21.15.6:0/2762176088 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8000acb0 con 0x7f6f8815dbf0 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f90565700 1 -- 172.21.15.6:0/2762176088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8815d3f0 con 0x7f6f8815dbf0 2023-11-28T13:01:54.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f7effd700 1 -- 172.21.15.6:0/2762176088 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f80007310 con 0x7f6f8815dbf0 2023-11-28T13:01:54.168 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.163+0000 7f6f90565700 1 -- 172.21.15.6:0/2762176088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8815d110 con 0x7f6f8815dbf0 2023-11-28T13:01:54.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.164+0000 7f6f90565700 1 -- 172.21.15.6:0/2762176088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8815d6d0 con 0x7f6f8815dbf0 2023-11-28T13:01:54.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.165+0000 7f6f7effd700 1 -- 172.21.15.6:0/2762176088 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6f8001a030 con 0x7f6f8815dbf0 2023-11-28T13:01:54.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.165+0000 7f6f7effd700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f6c0682f0 0x7f6f6c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.165+0000 7f6f7effd700 1 -- 172.21.15.6:0/2762176088 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6f80085ab0 con 0x7f6f8815dbf0 2023-11-28T13:01:54.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.165+0000 7f6f8e301700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f6c0682f0 0x7f6f6c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.166+0000 7f6f8e301700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f6c0682f0 0x7f6f6c06a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6f780090b0 tx=0x7f6f7801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.170+0000 7f6f7effd700 1 -- 172.21.15.6:0/2762176088 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6f800521d0 con 0x7f6f8815dbf0 2023-11-28T13:01:54.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.361+0000 7f6f90565700 1 -- 172.21.15.6:0/2762176088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f880b2c80 con 0x7f6f8815dbf0 2023-11-28T13:01:54.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.361+0000 7f6f7effd700 1 -- 172.21.15.6:0/2762176088 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f800180a0 con 0x7f6f8815dbf0 2023-11-28T13:01:54.363 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.363+0000 7f6f90565700 1 -- 172.21.15.6:0/2762176088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f6c0682f0 msgr2=0x7f6f6c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.363+0000 7f6f90565700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f6c0682f0 0x7f6f6c06a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6f780090b0 tx=0x7f6f7801c040 comp rx=0 tx=0).stop 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.363+0000 7f6f90565700 1 -- 172.21.15.6:0/2762176088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f8815dbf0 msgr2=0x7f6f880caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.363+0000 7f6f90565700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f8815dbf0 0x7f6f880caf70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000cb60 tx=0x7f6f8000cf20 comp rx=0 tx=0).stop 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.363+0000 7f6f90565700 1 -- 172.21.15.6:0/2762176088 shutdown_connections 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.363+0000 7f6f90565700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f8815dbf0 0x7f6f880caf70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.363+0000 7f6f90565700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6f6c0682f0 0x7f6f6c06a7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.363+0000 7f6f90565700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6f880b1630 0x7f6f880acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.363+0000 7f6f90565700 1 --2- 172.21.15.6:0/2762176088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6f88159500 0x7f6f88154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.363+0000 7f6f90565700 1 -- 172.21.15.6:0/2762176088 >> 172.21.15.6:0/2762176088 conn(0x7f6f880cc630 msgr2=0x7f6f8806be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:54.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.364+0000 7f6f90565700 1 -- 172.21.15.6:0/2762176088 shutdown_connections 2023-11-28T13:01:54.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.364+0000 7f6f90565700 1 -- 172.21.15.6:0/2762176088 wait complete. 2023-11-28T13:01:54.377 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:54.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.501+0000 7f8431e2d700 1 Processor -- start 2023-11-28T13:01:54.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.501+0000 7f8431e2d700 1 -- start start 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.502+0000 7f8431e2d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f842c158ad0 0x7f842c0711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.502+0000 7f8431e2d700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c0cc860 con 0x7f842c15cfc0 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.502+0000 7f8431e2d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c0ccb40 con 0x7f842c0cd620 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.502+0000 7f8431e2d700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c15d3f0 con 0x7f842c158ad0 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.502+0000 7f842b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f842c158ad0 0x7f842c0711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.502+0000 7f842b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f842c158ad0 0x7f842c0711f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60680/0 (socket says 172.21.15.6:60680) 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.502+0000 7f842b7fe700 1 -- 172.21.15.6:0/4048925731 learned_addr learned my addr 172.21.15.6:0/4048925731 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.502+0000 7f842bfff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f842c15cfc0 0x7f842c0c9480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48004/0 (socket says 172.21.15.6:48004) 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.503+0000 7f842b7fe700 1 -- 172.21.15.6:0/4048925731 >> v1:172.21.15.61:6789/0 conn(0x7f842c0cd620 legacy=0x7f842c15c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:54.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.503+0000 7f842b7fe700 1 -- 172.21.15.6:0/4048925731 >> v1:172.21.15.6:6789/0 conn(0x7f842c15cfc0 legacy=0x7f842c0c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.503+0000 7f842b7fe700 1 -- 172.21.15.6:0/4048925731 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c0cce20 con 0x7f842c158ad0 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.503+0000 7f842a7fc700 1 -- 172.21.15.6:0/4048925731 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490379314 0 0) 0x7f842c0cc860 con 0x7f842c15cfc0 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.503+0000 7f842b7fe700 1 --2- 172.21.15.6:0/4048925731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c158ad0 0x7f842c0711f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f841c009920 tx=0x7f841c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e6e0efadda4d25 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.503+0000 7f842a7fc700 1 -- 172.21.15.6:0/4048925731 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c004230 con 0x7f842c158ad0 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.503+0000 7f842a7fc700 1 -- 172.21.15.6:0/4048925731 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c004390 con 0x7f842c158ad0 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.503+0000 7f842a7fc700 1 -- 172.21.15.6:0/4048925731 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c009e70 con 0x7f842c158ad0 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.503+0000 7f8431e2d700 1 -- 172.21.15.6:0/4048925731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c158ad0 msgr2=0x7f842c0711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.503+0000 7f8431e2d700 1 --2- 172.21.15.6:0/4048925731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c158ad0 0x7f842c0711f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f841c009920 tx=0x7f841c009c30 comp rx=0 tx=0).stop 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.504+0000 7f8431e2d700 1 -- 172.21.15.6:0/4048925731 shutdown_connections 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.504+0000 7f8431e2d700 1 --2- 172.21.15.6:0/4048925731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c158ad0 0x7f842c0711f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.504+0000 7f8431e2d700 1 -- 172.21.15.6:0/4048925731 >> 172.21.15.6:0/4048925731 conn(0x7f842c0ce9e0 msgr2=0x7f842c0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:54.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.504+0000 7f8431e2d700 1 -- 172.21.15.6:0/4048925731 shutdown_connections 2023-11-28T13:01:54.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.504+0000 7f8431e2d700 1 -- 172.21.15.6:0/4048925731 wait complete. 2023-11-28T13:01:54.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.505+0000 7f8431e2d700 1 Processor -- start 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.505+0000 7f8431e2d700 1 -- start start 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8431e2d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c0cd620 0x7f842c0c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8431e2d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f842c158ad0 0x7f842c144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8431e2d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f842c15cfc0 0x7f842c0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8431e2d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c15b6c0 con 0x7f842c0cd620 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8431e2d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c15a7d0 con 0x7f842c158ad0 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8431e2d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c15aab0 con 0x7f842c15cfc0 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f842b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c0cd620 0x7f842c0c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f842b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c0cd620 0x7f842c0c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60686/0 (socket says 172.21.15.6:60686) 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f842b7fe700 1 -- 172.21.15.6:0/2278155586 learned_addr learned my addr 172.21.15.6:0/2278155586 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f842b7fe700 1 -- 172.21.15.6:0/2278155586 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f842c15cfc0 msgr2=0x7f842c0b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f842b7fe700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f842c15cfc0 0x7f842c0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f842b7fe700 1 -- 172.21.15.6:0/2278155586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f842c158ad0 msgr2=0x7f842c144c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f842b7fe700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f842c158ad0 0x7f842c144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f842b7fe700 1 -- 172.21.15.6:0/2278155586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c009550 con 0x7f842c0cd620 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f842b7fe700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c0cd620 0x7f842c0c4e10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8423fff700 1 -- 172.21.15.6:0/2278155586 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c003b70 con 0x7f842c0cd620 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8423fff700 1 -- 172.21.15.6:0/2278155586 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841c003cd0 con 0x7f842c0cd620 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8431e2d700 1 -- 172.21.15.6:0/2278155586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c15d610 con 0x7f842c0cd620 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8423fff700 1 -- 172.21.15.6:0/2278155586 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841c020650 con 0x7f842c0cd620 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.506+0000 7f8431e2d700 1 -- 172.21.15.6:0/2278155586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c0cd100 con 0x7f842c0cd620 2023-11-28T13:01:54.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.507+0000 7f8431e2d700 1 -- 172.21.15.6:0/2278155586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c158ea0 con 0x7f842c0cd620 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.508+0000 7f8423fff700 1 -- 172.21.15.6:0/2278155586 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f841c00e040 con 0x7f842c0cd620 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.508+0000 7f8423fff700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f840c0682f0 0x7f840c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.508+0000 7f8423fff700 1 -- 172.21.15.6:0/2278155586 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f841c089950 con 0x7f842c0cd620 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.508+0000 7f842affd700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f840c0682f0 0x7f840c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.509+0000 7f842affd700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f840c0682f0 0x7f840c06a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f842c1503c0 tx=0x7f8414007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.513+0000 7f8423fff700 1 -- 172.21.15.6:0/2278155586 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f841c056120 con 0x7f842c0cd620 2023-11-28T13:01:54.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.723+0000 7f8431e2d700 1 -- 172.21.15.6:0/2278155586 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f842c068570 con 0x7f842c0cd620 2023-11-28T13:01:54.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.723+0000 7f8423fff700 1 -- 172.21.15.6:0/2278155586 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f841c059740 con 0x7f842c0cd620 2023-11-28T13:01:54.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 -- 172.21.15.6:0/2278155586 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f840c0682f0 msgr2=0x7f840c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.726 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f840c0682f0 0x7f840c06a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f842c1503c0 tx=0x7f8414007a40 comp rx=0 tx=0).stop 2023-11-28T13:01:54.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 -- 172.21.15.6:0/2278155586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c0cd620 msgr2=0x7f842c0c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c0cd620 0x7f842c0c4e10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f841c000c00 tx=0x7f841c004750 comp rx=0 tx=0).stop 2023-11-28T13:01:54.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 -- 172.21.15.6:0/2278155586 shutdown_connections 2023-11-28T13:01:54.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f840c0682f0 0x7f840c06a7a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f842c15cfc0 0x7f842c0b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f842c0cd620 0x7f842c0c4e10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 --2- 172.21.15.6:0/2278155586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f842c158ad0 0x7f842c144c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 -- 172.21.15.6:0/2278155586 >> 172.21.15.6:0/2278155586 conn(0x7f842c0ce9e0 msgr2=0x7f842c153300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:54.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 -- 172.21.15.6:0/2278155586 shutdown_connections 2023-11-28T13:01:54.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.725+0000 7f8431e2d700 1 -- 172.21.15.6:0/2278155586 wait complete. 2023-11-28T13:01:54.728 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:01:54.738 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:01:54.738 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:01:54.756 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:01:54.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.919+0000 7f03439f7700 1 Processor -- start 2023-11-28T13:01:54.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.919+0000 7f03439f7700 1 -- start start 2023-11-28T13:01:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f03439f7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f033c15a6c0 0x7f033c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f03439f7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f033c0d0020 0x7f033c0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f03439f7700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033c0cfb90 con 0x7f033c0b2660 2023-11-28T13:01:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f03439f7700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c0cb930 con 0x7f033c0d0020 2023-11-28T13:01:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f03439f7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0cf260 con 0x7f033c15a6c0 2023-11-28T13:01:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f0340f92700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f033c0d0020 0x7f033c0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f0341f94700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f033c0b2660 0x7f033c0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48018/0 (socket says 172.21.15.6:48018) 2023-11-28T13:01:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f0341f94700 1 -- 172.21.15.6:0/1419028237 learned_addr learned my addr 172.21.15.6:0/1419028237 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f0340f92700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f033c0d0020 0x7f033c0ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60696/0 (socket says 172.21.15.6:60696) 2023-11-28T13:01:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f0341793700 1 --2- 172.21.15.6:0/1419028237 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f033c15a6c0 0x7f033c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f0340f92700 1 -- 172.21.15.6:0/1419028237 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f033c15a6c0 msgr2=0x7f033c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f0340f92700 1 --2- 172.21.15.6:0/1419028237 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f033c15a6c0 0x7f033c0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f0340f92700 1 -- 172.21.15.6:0/1419028237 >> v1:172.21.15.6:6789/0 conn(0x7f033c0b2660 legacy=0x7f033c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f0333fff700 1 -- 172.21.15.6:0/1419028237 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133930354 0 0) 0x7f033c0cfb90 con 0x7f033c0b2660 2023-11-28T13:01:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.920+0000 7f0340f92700 1 -- 172.21.15.6:0/1419028237 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c0cf540 con 0x7f033c0d0020 2023-11-28T13:01:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f0340f92700 1 --2- 172.21.15.6:0/1419028237 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 0x7f033c0ccea0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0338009840 tx=0x7f0338009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7caa5827380a26 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f0333fff700 1 -- 172.21.15.6:0/1419028237 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338011830 con 0x7f033c0d0020 2023-11-28T13:01:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f0333fff700 1 -- 172.21.15.6:0/1419028237 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0338011990 con 0x7f033c0d0020 2023-11-28T13:01:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f0333fff700 1 -- 172.21.15.6:0/1419028237 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338011d20 con 0x7f033c0d0020 2023-11-28T13:01:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f03439f7700 1 -- 172.21.15.6:0/1419028237 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 msgr2=0x7f033c0ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f03439f7700 1 --2- 172.21.15.6:0/1419028237 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 0x7f033c0ccea0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0338009840 tx=0x7f0338009b50 comp rx=0 tx=0).stop 2023-11-28T13:01:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f03439f7700 1 -- 172.21.15.6:0/1419028237 shutdown_connections 2023-11-28T13:01:54.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f03439f7700 1 --2- 172.21.15.6:0/1419028237 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f033c15a6c0 0x7f033c0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f03439f7700 1 --2- 172.21.15.6:0/1419028237 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 0x7f033c0ccea0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f03439f7700 1 -- 172.21.15.6:0/1419028237 >> 172.21.15.6:0/1419028237 conn(0x7f033c0cd660 msgr2=0x7f033c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:54.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f03439f7700 1 -- 172.21.15.6:0/1419028237 shutdown_connections 2023-11-28T13:01:54.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.921+0000 7f03439f7700 1 -- 172.21.15.6:0/1419028237 wait complete. 2023-11-28T13:01:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.923+0000 7f03439f7700 1 Processor -- start 2023-11-28T13:01:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.923+0000 7f03439f7700 1 -- start start 2023-11-28T13:01:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.923+0000 7f03439f7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f033c0b2660 0x7f033c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.923+0000 7f03439f7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 0x7f033c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.923+0000 7f03439f7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f033c15a6c0 0x7f033c0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.923+0000 7f03439f7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0c8b10 con 0x7f033c0d0020 2023-11-28T13:01:54.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.923+0000 7f03439f7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c06bb80 con 0x7f033c0b2660 2023-11-28T13:01:54.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.923+0000 7f03439f7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c15b570 con 0x7f033c15a6c0 2023-11-28T13:01:54.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.923+0000 7f0340f92700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 0x7f033c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.923+0000 7f0340f92700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 0x7f033c0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60700/0 (socket says 172.21.15.6:60700) 2023-11-28T13:01:54.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0340f92700 1 -- 172.21.15.6:0/4054782293 learned_addr learned my addr 172.21.15.6:0/4054782293 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:54.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0341f94700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f033c15a6c0 0x7f033c0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0341793700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f033c0b2660 0x7f033c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0340f92700 1 -- 172.21.15.6:0/4054782293 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f033c15a6c0 msgr2=0x7f033c0c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0340f92700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f033c15a6c0 0x7f033c0c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0340f92700 1 -- 172.21.15.6:0/4054782293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f033c0b2660 msgr2=0x7f033c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0340f92700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f033c0b2660 0x7f033c0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0340f92700 1 -- 172.21.15.6:0/4054782293 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03380094a0 con 0x7f033c0d0020 2023-11-28T13:01:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0340f92700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 0x7f033c0c7cb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f03380090b0 tx=0x7f0338004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0331ffb700 1 -- 172.21.15.6:0/4054782293 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380038a0 con 0x7f033c0d0020 2023-11-28T13:01:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0331ffb700 1 -- 172.21.15.6:0/4054782293 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033801f070 con 0x7f033c0d0020 2023-11-28T13:01:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f03439f7700 1 -- 172.21.15.6:0/4054782293 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c0cd3e0 con 0x7f033c0d0020 2023-11-28T13:01:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f0331ffb700 1 -- 172.21.15.6:0/4054782293 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338003c20 con 0x7f033c0d0020 2023-11-28T13:01:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.924+0000 7f03439f7700 1 -- 172.21.15.6:0/4054782293 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c0cf820 con 0x7f033c0d0020 2023-11-28T13:01:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.925+0000 7f03439f7700 1 -- 172.21.15.6:0/4054782293 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c0d0470 con 0x7f033c0d0020 2023-11-28T13:01:54.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.926+0000 7f0331ffb700 1 -- 172.21.15.6:0/4054782293 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f033800f050 con 0x7f033c0d0020 2023-11-28T13:01:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.926+0000 7f0331ffb700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03200682f0 0x7f032006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.926+0000 7f0331ffb700 1 -- 172.21.15.6:0/4054782293 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0338021070 con 0x7f033c0d0020 2023-11-28T13:01:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.926+0000 7f0341793700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03200682f0 0x7f032006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.927+0000 7f0341793700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03200682f0 0x7f032006a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b9b0 tx=0x7f032c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:54.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:54.931+0000 7f0331ffb700 1 -- 172.21.15.6:0/4054782293 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0338053f40 con 0x7f033c0d0020 2023-11-28T13:01:55.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.127+0000 7f03439f7700 1 -- 172.21.15.6:0/4054782293 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f033c0675a0 con 0x7f033c0d0020 2023-11-28T13:01:55.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.129+0000 7f0331ffb700 1 -- 172.21.15.6:0/4054782293 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17974 (secure 0 0 0) 0x7f0338057560 con 0x7f033c0d0020 2023-11-28T13:01:55.130 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:01:55.130 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":40,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:01:43.596467+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:41.144348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:41.554568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:01:55.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 -- 172.21.15.6:0/4054782293 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03200682f0 msgr2=0x7f032006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03200682f0 0x7f032006a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b9b0 tx=0x7f032c008040 comp rx=0 tx=0).stop 2023-11-28T13:01:55.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 -- 172.21.15.6:0/4054782293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 msgr2=0x7f033c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 0x7f033c0c7cb0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f03380090b0 tx=0x7f0338004750 comp rx=0 tx=0).stop 2023-11-28T13:01:55.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 -- 172.21.15.6:0/4054782293 shutdown_connections 2023-11-28T13:01:55.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03200682f0 0x7f032006a7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f033c15a6c0 0x7f033c0c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f033c0b2660 0x7f033c0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 --2- 172.21.15.6:0/4054782293 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f033c0d0020 0x7f033c0c7cb0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 -- 172.21.15.6:0/4054782293 >> 172.21.15.6:0/4054782293 conn(0x7f033c0cd660 msgr2=0x7f033c0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:55.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 -- 172.21.15.6:0/4054782293 shutdown_connections 2023-11-28T13:01:55.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.131+0000 7f03439f7700 1 -- 172.21.15.6:0/4054782293 wait complete. 2023-11-28T13:01:55.144 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:01:41.144348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:01:41.554568+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:01:55.145 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:01:55.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.271+0000 7f94b0e82700 1 Processor -- start 2023-11-28T13:01:55.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.271+0000 7f94b0e82700 1 -- start start 2023-11-28T13:01:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94b0e82700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f94ac15a410 0x7f94ac0adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94b0e82700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f94ac15df40 0x7f94ac070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94b0e82700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ac0712e0 con 0x7f94ac070ea0 2023-11-28T13:01:55.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94b0e82700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ac15a7e0 con 0x7f94ac15df40 2023-11-28T13:01:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94b0e82700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ac15e3c0 con 0x7f94ac15a410 2023-11-28T13:01:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a9d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f94ac15df40 0x7f94ac070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a9d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f94ac15df40 0x7f94ac070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60710/0 (socket says 172.21.15.6:60710) 2023-11-28T13:01:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a9d9b700 1 -- 172.21.15.6:0/262051559 learned_addr learned my addr 172.21.15.6:0/262051559 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94aa59c700 1 --2- 172.21.15.6:0/262051559 >> v2:172.21.15.61:3300/0 conn(0x7f94ac15a410 0x7f94ac0adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a9d9b700 1 -- 172.21.15.6:0/262051559 >> v2:172.21.15.61:3300/0 conn(0x7f94ac15a410 msgr2=0x7f94ac0adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a9d9b700 1 --2- 172.21.15.6:0/262051559 >> v2:172.21.15.61:3300/0 conn(0x7f94ac15a410 0x7f94ac0adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a9d9b700 1 -- 172.21.15.6:0/262051559 >> v1:172.21.15.61:6789/0 conn(0x7f94ac070ea0 legacy=0x7f94ac06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a9d9b700 1 -- 172.21.15.6:0/262051559 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac15e6a0 con 0x7f94ac15df40 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a9d9b700 1 --2- 172.21.15.6:0/262051559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15df40 0x7f94ac070960 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f94ac071cb0 tx=0x7f949c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c35a35a1a1bb87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a959a700 1 -- 172.21.15.6:0/262051559 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c004230 con 0x7f94ac15df40 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a959a700 1 -- 172.21.15.6:0/262051559 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c004390 con 0x7f94ac15df40 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.272+0000 7f94a959a700 1 -- 172.21.15.6:0/262051559 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c0036a0 con 0x7f94ac15df40 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.273+0000 7f94b0e82700 1 -- 172.21.15.6:0/262051559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15df40 msgr2=0x7f94ac070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.273+0000 7f94b0e82700 1 --2- 172.21.15.6:0/262051559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15df40 0x7f94ac070960 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f94ac071cb0 tx=0x7f949c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.273+0000 7f94b0e82700 1 -- 172.21.15.6:0/262051559 shutdown_connections 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.273+0000 7f94b0e82700 1 --2- 172.21.15.6:0/262051559 >> v2:172.21.15.61:3300/0 conn(0x7f94ac15a410 0x7f94ac0adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.273+0000 7f94b0e82700 1 --2- 172.21.15.6:0/262051559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15df40 0x7f94ac070960 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.273+0000 7f94b0e82700 1 -- 172.21.15.6:0/262051559 >> 172.21.15.6:0/262051559 conn(0x7f94ac0ce9e0 msgr2=0x7f94ac0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:55.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.273+0000 7f94b0e82700 1 -- 172.21.15.6:0/262051559 shutdown_connections 2023-11-28T13:01:55.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.273+0000 7f94b0e82700 1 -- 172.21.15.6:0/262051559 wait complete. 2023-11-28T13:01:55.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.274+0000 7f94b0e82700 1 Processor -- start 2023-11-28T13:01:55.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.274+0000 7f94b0e82700 1 -- start start 2023-11-28T13:01:55.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.275+0000 7f94b0e82700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f94ac070ea0 0x7f94ac154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.275+0000 7f94b0e82700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15a410 0x7f94ac0c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.275+0000 7f94b0e82700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f94ac0c8c90 0x7f94ac154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.275+0000 7f94b0e82700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0c9a60 con 0x7f94ac15a410 2023-11-28T13:01:55.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.275+0000 7f94b0e82700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac154660 con 0x7f94ac070ea0 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.275+0000 7f94b0e82700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0c91c0 con 0x7f94ac0c8c90 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.275+0000 7f94a9d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15a410 0x7f94ac0c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.275+0000 7f94a9d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15a410 0x7f94ac0c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60716/0 (socket says 172.21.15.6:60716) 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.275+0000 7f94a9d9b700 1 -- 172.21.15.6:0/244496471 learned_addr learned my addr 172.21.15.6:0/244496471 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f94a9d9b700 1 -- 172.21.15.6:0/244496471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f94ac0c8c90 msgr2=0x7f94ac154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f94aa59c700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f94ac070ea0 0x7f94ac154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f94aad9d700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f94ac0c8c90 0x7f94ac154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f94a9d9b700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f94ac0c8c90 0x7f94ac154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f94a9d9b700 1 -- 172.21.15.6:0/244496471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f94ac070ea0 msgr2=0x7f94ac154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f94a9d9b700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f94ac070ea0 0x7f94ac154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f94a9d9b700 1 -- 172.21.15.6:0/244496471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c009550 con 0x7f94ac15a410 2023-11-28T13:01:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f94a9d9b700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15a410 0x7f94ac0c8750 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f949c009130 tx=0x7f949c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f949affd700 1 -- 172.21.15.6:0/244496471 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c011630 con 0x7f94ac15a410 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f949affd700 1 -- 172.21.15.6:0/244496471 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c011790 con 0x7f94ac15a410 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f949affd700 1 -- 172.21.15.6:0/244496471 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c011b20 con 0x7f94ac15a410 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f94b0e82700 1 -- 172.21.15.6:0/244496471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac15ec60 con 0x7f94ac15a410 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.276+0000 7f94b0e82700 1 -- 172.21.15.6:0/244496471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac15e980 con 0x7f94ac15a410 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.277+0000 7f94b0e82700 1 -- 172.21.15.6:0/244496471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac0adbd0 con 0x7f94ac15a410 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.277+0000 7f949affd700 1 -- 172.21.15.6:0/244496471 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f949c02a070 con 0x7f94ac15a410 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.278+0000 7f949affd700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f948c0682f0 0x7f948c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.278+0000 7f949affd700 1 -- 172.21.15.6:0/244496471 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f949c0872d0 con 0x7f94ac15a410 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.278+0000 7f94aa59c700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f948c0682f0 0x7f948c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.278+0000 7f94aa59c700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f948c0682f0 0x7f948c06a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f94ac155cf0 tx=0x7f94a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.282+0000 7f949affd700 1 -- 172.21.15.6:0/244496471 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f949c053970 con 0x7f94ac15a410 2023-11-28T13:01:55.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.475+0000 7f94b0e82700 1 -- 172.21.15.6:0/244496471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f94ac0739f0 con 0x7f94ac15a410 2023-11-28T13:01:55.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.476+0000 7f949affd700 1 -- 172.21.15.6:0/244496471 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f949c056f90 con 0x7f94ac15a410 2023-11-28T13:01:55.477 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:01:55.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 -- 172.21.15.6:0/244496471 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f948c0682f0 msgr2=0x7f948c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f948c0682f0 0x7f948c06a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f94ac155cf0 tx=0x7f94a0008040 comp rx=0 tx=0).stop 2023-11-28T13:01:55.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 -- 172.21.15.6:0/244496471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15a410 msgr2=0x7f94ac0c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15a410 0x7f94ac0c8750 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f949c009130 tx=0x7f949c004750 comp rx=0 tx=0).stop 2023-11-28T13:01:55.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 -- 172.21.15.6:0/244496471 shutdown_connections 2023-11-28T13:01:55.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f948c0682f0 0x7f948c06a7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f94ac0c8c90 0x7f94ac154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f94ac070ea0 0x7f94ac154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 --2- 172.21.15.6:0/244496471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94ac15a410 0x7f94ac0c8750 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 -- 172.21.15.6:0/244496471 >> 172.21.15.6:0/244496471 conn(0x7f94ac0ce9e0 msgr2=0x7f94ac0ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:55.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 -- 172.21.15.6:0/244496471 shutdown_connections 2023-11-28T13:01:55.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.478+0000 7f94b0e82700 1 -- 172.21.15.6:0/244496471 wait complete. 2023-11-28T13:01:55.491 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:01:55.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.618+0000 7f6a5e948700 1 Processor -- start 2023-11-28T13:01:55.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.618+0000 7f6a5e948700 1 -- start start 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.618+0000 7f6a5e948700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6a580cf5f0 0x7f6a580caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.618+0000 7f6a5e948700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a580ce550 con 0x7f6a580cb3e0 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a5e948700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a580ce830 con 0x7f6a580b1630 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a5e948700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a580ceb10 con 0x7f6a580cf5f0 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a5cee5700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6a580cb3e0 0x7f6a5806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48046/0 (socket says 172.21.15.6:48046) 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a5cee5700 1 -- 172.21.15.6:0/1717294968 learned_addr learned my addr 172.21.15.6:0/1717294968 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438547144 0 0) 0x7f6a580ce550 con 0x7f6a580cb3e0 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a440035b0 con 0x7f6a580cb3e0 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a577fe700 1 --2- 172.21.15.6:0/1717294968 >> v2:172.21.15.61:3300/0 conn(0x7f6a580cf5f0 0x7f6a580caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1603552601 0 0) 0x7f6a440035b0 con 0x7f6a580cb3e0 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 >> v2:172.21.15.61:3300/0 conn(0x7f6a580cf5f0 msgr2=0x7f6a580caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 --2- 172.21.15.6:0/1717294968 >> v2:172.21.15.61:3300/0 conn(0x7f6a580cf5f0 0x7f6a580caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 >> v1:172.21.15.61:6789/0 conn(0x7f6a580b1630 legacy=0x7f6a580b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580cedf0 con 0x7f6a580cb3e0 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f6a500019f0 con 0x7f6a580cb3e0 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a577fe700 1 --2- 172.21.15.6:0/1717294968 >> v2:172.21.15.61:3300/0 conn(0x7f6a580cf5f0 0x7f6a580caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 >> v1:172.21.15.6:6789/0 conn(0x7f6a580cb3e0 legacy=0x7f6a5806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a44003a30 0x7f6a44003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a44004850 0x7f6a44004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a44005250 0x7f6a4400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580ceb10 con 0x7f6a44004850 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4400c160 con 0x7f6a44003a30 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4400c320 con 0x7f6a44005250 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.619+0000 7f6a577fe700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a44004850 0x7f6a44004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a57fff700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a44003a30 0x7f6a44003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a5cee5700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a44005250 0x7f6a4400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a577fe700 1 -- 172.21.15.6:0/1717294968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a44005250 msgr2=0x7f6a4400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a577fe700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a44005250 0x7f6a4400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a577fe700 1 -- 172.21.15.6:0/1717294968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a44003a30 msgr2=0x7f6a44003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a577fe700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a44003a30 0x7f6a44003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a577fe700 1 -- 172.21.15.6:0/1717294968 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440035b0 con 0x7f6a44004850 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a5cee5700 1 -- 172.21.15.6:0/1717294968 reap_dead start 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a577fe700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a44004850 0x7f6a44004cc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009950 tx=0x7f6a44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01899a67dbf5f73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800c070 con 0x7f6a44004850 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a48013610 con 0x7f6a44004850 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a56ffd700 1 -- 172.21.15.6:0/1717294968 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a480139a0 con 0x7f6a44004850 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a5e948700 1 -- 172.21.15.6:0/1717294968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a44004850 msgr2=0x7f6a44004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.620+0000 7f6a5e948700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a44004850 0x7f6a44004cc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009950 tx=0x7f6a44005880 comp rx=0 tx=0).stop 2023-11-28T13:01:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.621+0000 7f6a5e948700 1 -- 172.21.15.6:0/1717294968 shutdown_connections 2023-11-28T13:01:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.621+0000 7f6a5e948700 1 --2- 172.21.15.6:0/1717294968 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a44004850 0x7f6a44004cc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.621+0000 7f6a5e948700 1 -- 172.21.15.6:0/1717294968 >> 172.21.15.6:0/1717294968 conn(0x7f6a580cc630 msgr2=0x7f6a580cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.621+0000 7f6a5e948700 1 -- 172.21.15.6:0/1717294968 shutdown_connections 2023-11-28T13:01:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.621+0000 7f6a5e948700 1 -- 172.21.15.6:0/1717294968 wait complete. 2023-11-28T13:01:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.622+0000 7f6a5e948700 1 Processor -- start 2023-11-28T13:01:55.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.622+0000 7f6a5e948700 1 -- start start 2023-11-28T13:01:55.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5e948700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a44004850 0x7f6a580b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5e948700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5815e740 0x7f6a5815c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5e948700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a580ad7a0 0x7f6a580adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5e948700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a581f2730 con 0x7f6a580ad7a0 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5e948700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580ae150 con 0x7f6a5815e740 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5e948700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580ae990 con 0x7f6a44004850 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5cee5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a580ad7a0 0x7f6a580adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5cee5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a580ad7a0 0x7f6a580adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60738/0 (socket says 172.21.15.6:60738) 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5cee5700 1 -- 172.21.15.6:0/2717574948 learned_addr learned my addr 172.21.15.6:0/2717574948 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a57fff700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a44004850 0x7f6a580b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a577fe700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5815e740 0x7f6a5815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5cee5700 1 -- 172.21.15.6:0/2717574948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a44004850 msgr2=0x7f6a580b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5cee5700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a44004850 0x7f6a580b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5cee5700 1 -- 172.21.15.6:0/2717574948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5815e740 msgr2=0x7f6a5815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5cee5700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5815e740 0x7f6a5815c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5cee5700 1 -- 172.21.15.6:0/2717574948 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580ce140 con 0x7f6a580ad7a0 2023-11-28T13:01:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a57fff700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a44004850 0x7f6a580b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a577fe700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5815e740 0x7f6a5815c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5cee5700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a580ad7a0 0x7f6a580adc10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6a580d0c50 tx=0x7f6a50007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a54ff9700 1 -- 172.21.15.6:0/2717574948 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50017070 con 0x7f6a580ad7a0 2023-11-28T13:01:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a54ff9700 1 -- 172.21.15.6:0/2717574948 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5000ec30 con 0x7f6a580ad7a0 2023-11-28T13:01:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a54ff9700 1 -- 172.21.15.6:0/2717574948 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000ede0 con 0x7f6a580ad7a0 2023-11-28T13:01:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.623+0000 7f6a5e948700 1 -- 172.21.15.6:0/2717574948 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a580cf0d0 con 0x7f6a580ad7a0 2023-11-28T13:01:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.624+0000 7f6a5e948700 1 -- 172.21.15.6:0/2717574948 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a580ce2d0 con 0x7f6a580ad7a0 2023-11-28T13:01:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.625+0000 7f6a5e948700 1 -- 172.21.15.6:0/2717574948 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a580739f0 con 0x7f6a580ad7a0 2023-11-28T13:01:55.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.625+0000 7f6a54ff9700 1 -- 172.21.15.6:0/2717574948 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a50013420 con 0x7f6a580ad7a0 2023-11-28T13:01:55.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.625+0000 7f6a54ff9700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a380682f0 0x7f6a3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.626+0000 7f6a54ff9700 1 -- 172.21.15.6:0/2717574948 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6a50085cc0 con 0x7f6a580ad7a0 2023-11-28T13:01:55.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.626+0000 7f6a57fff700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a380682f0 0x7f6a3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.626+0000 7f6a57fff700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a380682f0 0x7f6a3806a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6a40000c00 tx=0x7f6a40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.630+0000 7f6a54ff9700 1 -- 172.21.15.6:0/2717574948 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a50052380 con 0x7f6a580ad7a0 2023-11-28T13:01:55.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.823+0000 7f6a5e948700 1 -- 172.21.15.6:0/2717574948 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a580b2230 con 0x7f6a580ad7a0 2023-11-28T13:01:55.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.823+0000 7f6a54ff9700 1 -- 172.21.15.6:0/2717574948 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a500559a0 con 0x7f6a580ad7a0 2023-11-28T13:01:55.825 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:01:55.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.825+0000 7f6a5e948700 1 -- 172.21.15.6:0/2717574948 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a380682f0 msgr2=0x7f6a3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.825+0000 7f6a5e948700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a380682f0 0x7f6a3806a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6a40000c00 tx=0x7f6a40009040 comp rx=0 tx=0).stop 2023-11-28T13:01:55.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.825+0000 7f6a5e948700 1 -- 172.21.15.6:0/2717574948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a580ad7a0 msgr2=0x7f6a580adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.825+0000 7f6a5e948700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a580ad7a0 0x7f6a580adc10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6a580d0c50 tx=0x7f6a50007c90 comp rx=0 tx=0).stop 2023-11-28T13:01:55.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.825+0000 7f6a5e948700 1 -- 172.21.15.6:0/2717574948 shutdown_connections 2023-11-28T13:01:55.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.825+0000 7f6a5e948700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6a580ad7a0 0x7f6a580adc10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.825+0000 7f6a5e948700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6a380682f0 0x7f6a3806a7a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.825+0000 7f6a5e948700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6a44004850 0x7f6a580b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.825+0000 7f6a5e948700 1 --2- 172.21.15.6:0/2717574948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a5815e740 0x7f6a5815c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.825+0000 7f6a5e948700 1 -- 172.21.15.6:0/2717574948 >> 172.21.15.6:0/2717574948 conn(0x7f6a580cc630 msgr2=0x7f6a5806cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:55.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.826+0000 7f6a5e948700 1 -- 172.21.15.6:0/2717574948 shutdown_connections 2023-11-28T13:01:55.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.826+0000 7f6a5e948700 1 -- 172.21.15.6:0/2717574948 wait complete. 2023-11-28T13:01:55.839 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:01:55.966 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.965+0000 7faa81ba9700 1 Processor -- start 2023-11-28T13:01:55.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.965+0000 7faa81ba9700 1 -- start start 2023-11-28T13:01:55.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa81ba9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c15a640 0x7faa7c0c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa81ba9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7faa7c06b900 0x7faa7c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa81ba9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c0ae730 con 0x7faa7c0cd140 2023-11-28T13:01:55.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa81ba9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7c0cb210 con 0x7faa7c06b900 2023-11-28T13:01:55.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa81ba9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c06e9e0 con 0x7faa7c15a640 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa7b7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c15a640 0x7faa7c0c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa7bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7faa7c06b900 0x7faa7c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa7b7fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c15a640 0x7faa7c0c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:50446/0 (socket says 172.21.15.6:50446) 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa7affd700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7faa7c0cd140 0x7faa7c06b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:38092/0 (socket says 172.21.15.6:38092) 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa7b7fe700 1 -- 172.21.15.6:0/341831064 learned_addr learned my addr 172.21.15.6:0/341831064 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.966+0000 7faa7bfff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7faa7c06b900 0x7faa7c0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:36232/0 (socket says 172.21.15.6:36232) 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.967+0000 7faa7b7fe700 1 -- 172.21.15.6:0/341831064 >> v2:172.21.15.61:3300/0 conn(0x7faa7c06b900 msgr2=0x7faa7c0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.967+0000 7faa7b7fe700 1 --2- 172.21.15.6:0/341831064 >> v2:172.21.15.61:3300/0 conn(0x7faa7c06b900 0x7faa7c0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.967+0000 7faa7b7fe700 1 -- 172.21.15.6:0/341831064 >> v1:172.21.15.61:6789/0 conn(0x7faa7c0cd140 legacy=0x7faa7c06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.967+0000 7faa7b7fe700 1 -- 172.21.15.6:0/341831064 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c0ae980 con 0x7faa7c15a640 2023-11-28T13:01:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.967+0000 7faa7bfff700 1 --2- 172.21.15.6:0/341831064 >> v2:172.21.15.61:3300/0 conn(0x7faa7c06b900 0x7faa7c0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.967+0000 7faa7b7fe700 1 --2- 172.21.15.6:0/341831064 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c15a640 0x7faa7c0c3700 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa640098f0 tx=0x7faa64009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee6c32cc4963c6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.967+0000 7faa7a7fc700 1 -- 172.21.15.6:0/341831064 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64011830 con 0x7faa7c15a640 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.967+0000 7faa7a7fc700 1 -- 172.21.15.6:0/341831064 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa64011990 con 0x7faa7c15a640 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.967+0000 7faa7a7fc700 1 -- 172.21.15.6:0/341831064 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64011d20 con 0x7faa7c15a640 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.967+0000 7faa81ba9700 1 -- 172.21.15.6:0/341831064 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c15a640 msgr2=0x7faa7c0c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.968+0000 7faa81ba9700 1 --2- 172.21.15.6:0/341831064 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c15a640 0x7faa7c0c3700 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa640098f0 tx=0x7faa64009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.968+0000 7faa81ba9700 1 -- 172.21.15.6:0/341831064 shutdown_connections 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.968+0000 7faa81ba9700 1 --2- 172.21.15.6:0/341831064 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c15a640 0x7faa7c0c3700 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.968+0000 7faa81ba9700 1 --2- 172.21.15.6:0/341831064 >> v2:172.21.15.61:3300/0 conn(0x7faa7c06b900 0x7faa7c0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.968+0000 7faa81ba9700 1 -- 172.21.15.6:0/341831064 >> 172.21.15.6:0/341831064 conn(0x7faa7c0cc630 msgr2=0x7faa7c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.968+0000 7faa81ba9700 1 -- 172.21.15.6:0/341831064 shutdown_connections 2023-11-28T13:01:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.968+0000 7faa81ba9700 1 -- 172.21.15.6:0/341831064 wait complete. 2023-11-28T13:01:55.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.969+0000 7faa81ba9700 1 Processor -- start 2023-11-28T13:01:55.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.969+0000 7faa81ba9700 1 -- start start 2023-11-28T13:01:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.970+0000 7faa81ba9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c06b900 0x7faa7c14d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.970+0000 7faa81ba9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd140 0x7faa7c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.970+0000 7faa81ba9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c15a640 0x7faa7c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.970+0000 7faa81ba9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c15c080 con 0x7faa7c15a640 2023-11-28T13:01:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.970+0000 7faa81ba9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c14d5d0 con 0x7faa7c06b900 2023-11-28T13:01:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.970+0000 7faa81ba9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c15c290 con 0x7faa7c0cd140 2023-11-28T13:01:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c15a640 0x7faa7c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c15a640 0x7faa7c159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60752/0 (socket says 172.21.15.6:60752) 2023-11-28T13:01:55.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7bfff700 1 -- 172.21.15.6:0/3064720819 learned_addr learned my addr 172.21.15.6:0/3064720819 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7affd700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd140 0x7faa7c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7b7fe700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c06b900 0x7faa7c14d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7bfff700 1 -- 172.21.15.6:0/3064720819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd140 msgr2=0x7faa7c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7bfff700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd140 0x7faa7c0b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7bfff700 1 -- 172.21.15.6:0/3064720819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c06b900 msgr2=0x7faa7c14d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7bfff700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c06b900 0x7faa7c14d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7bfff700 1 -- 172.21.15.6:0/3064720819 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa74009a70 con 0x7faa7c15a640 2023-11-28T13:01:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7b7fe700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c06b900 0x7faa7c14d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa7bfff700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c15a640 0x7faa7c159390 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7faa7400efb0 tx=0x7faa7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa73fff700 1 -- 172.21.15.6:0/3064720819 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7400cd40 con 0x7faa7c15a640 2023-11-28T13:01:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa73fff700 1 -- 172.21.15.6:0/3064720819 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7400f040 con 0x7faa7c15a640 2023-11-28T13:01:55.977 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa81ba9700 1 -- 172.21.15.6:0/3064720819 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa64009550 con 0x7faa7c15a640 2023-11-28T13:01:55.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa73fff700 1 -- 172.21.15.6:0/3064720819 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7400cea0 con 0x7faa7c15a640 2023-11-28T13:01:55.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.971+0000 7faa81ba9700 1 -- 172.21.15.6:0/3064720819 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c0aef40 con 0x7faa7c15a640 2023-11-28T13:01:55.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.972+0000 7faa81ba9700 1 -- 172.21.15.6:0/3064720819 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c0aec60 con 0x7faa7c15a640 2023-11-28T13:01:55.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.973+0000 7faa73fff700 1 -- 172.21.15.6:0/3064720819 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faa740146a0 con 0x7faa7c15a640 2023-11-28T13:01:55.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.973+0000 7faa73fff700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 0x7faa5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:55.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.973+0000 7faa73fff700 1 -- 172.21.15.6:0/3064720819 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7faa74086b40 con 0x7faa7c15a640 2023-11-28T13:01:55.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.973+0000 7faa7b7fe700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 0x7faa5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:55.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.974+0000 7faa7b7fe700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 0x7faa5c06a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faa64009c40 tx=0x7faa64008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:55.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:55.977+0000 7faa73fff700 1 -- 172.21.15.6:0/3064720819 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa74053200 con 0x7faa7c15a640 2023-11-28T13:01:56.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.171+0000 7faa81ba9700 1 -- 172.21.15.6:0/3064720819 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa7c0af220 con 0x7faa7c15a640 2023-11-28T13:01:56.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.172+0000 7faa73fff700 1 -- 172.21.15.6:0/3064720819 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa74018020 con 0x7faa7c15a640 2023-11-28T13:01:56.173 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:01:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 -- 172.21.15.6:0/3064720819 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 msgr2=0x7faa5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 0x7faa5c06a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faa64009c40 tx=0x7faa64008030 comp rx=0 tx=0).stop 2023-11-28T13:01:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 -- 172.21.15.6:0/3064720819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c15a640 msgr2=0x7faa7c159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c15a640 0x7faa7c159390 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7faa7400efb0 tx=0x7faa7400c3a0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 -- 172.21.15.6:0/3064720819 shutdown_connections 2023-11-28T13:01:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faa7c15a640 0x7faa7c159390 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faa7c06b900 0x7faa7c14d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faa5c0682f0 0x7faa5c06a7a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 --2- 172.21.15.6:0/3064720819 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faa7c0cd140 0x7faa7c0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 -- 172.21.15.6:0/3064720819 >> 172.21.15.6:0/3064720819 conn(0x7faa7c0cc630 msgr2=0x7faa7c1552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 -- 172.21.15.6:0/3064720819 shutdown_connections 2023-11-28T13:01:56.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.174+0000 7faa81ba9700 1 -- 172.21.15.6:0/3064720819 wait complete. 2023-11-28T13:01:56.187 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:01:56.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.313+0000 7fc6efbca700 1 Processor -- start 2023-11-28T13:01:56.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.313+0000 7fc6efbca700 1 -- start start 2023-11-28T13:01:56.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.314+0000 7fc6efbca700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc6e80b62d0 0x7fc6e80cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.314+0000 7fc6efbca700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 0x7fc6e80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.314+0000 7fc6efbca700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e80b3020 con 0x7fc6e80b2bc0 2023-11-28T13:01:56.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.314+0000 7fc6efbca700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e80cd2a0 con 0x7fc6e80b62d0 2023-11-28T13:01:56.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.314+0000 7fc6efbca700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80d0400 con 0x7fc6e806f8b0 2023-11-28T13:01:56.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed966700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc6e80b62d0 0x7fc6e80cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed165700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 0x7fc6e80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed966700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc6e80b62d0 0x7fc6e80cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:36242/0 (socket says 172.21.15.6:36242) 2023-11-28T13:01:56.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ee167700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7fc6e80b2bc0 0x7fc6e80ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:38110/0 (socket says 172.21.15.6:38110) 2023-11-28T13:01:56.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed966700 1 -- 172.21.15.6:0/711068887 learned_addr learned my addr 172.21.15.6:0/711068887 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:56.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed165700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 0x7fc6e80b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:50460/0 (socket says 172.21.15.6:50460) 2023-11-28T13:01:56.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed966700 1 -- 172.21.15.6:0/711068887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 msgr2=0x7fc6e80b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:56.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed966700 1 --2- 172.21.15.6:0/711068887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 0x7fc6e80b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed966700 1 -- 172.21.15.6:0/711068887 >> v1:172.21.15.61:6789/0 conn(0x7fc6e80b2bc0 legacy=0x7fc6e80ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ec964700 1 -- 172.21.15.6:0/711068887 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386758071 0 0) 0x7fc6e80b3020 con 0x7fc6e80b2bc0 2023-11-28T13:01:56.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed966700 1 -- 172.21.15.6:0/711068887 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80d06e0 con 0x7fc6e80b62d0 2023-11-28T13:01:56.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed165700 1 --2- 172.21.15.6:0/711068887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 0x7fc6e80b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:56.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ed966700 1 --2- 172.21.15.6:0/711068887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6e80b62d0 0x7fc6e80cb630 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80098f0 tx=0x7fc6d8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1f83d8779650be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:56.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ec964700 1 -- 172.21.15.6:0/711068887 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8011830 con 0x7fc6e80b62d0 2023-11-28T13:01:56.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ec964700 1 -- 172.21.15.6:0/711068887 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8011990 con 0x7fc6e80b62d0 2023-11-28T13:01:56.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.315+0000 7fc6ec964700 1 -- 172.21.15.6:0/711068887 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8011d20 con 0x7fc6e80b62d0 2023-11-28T13:01:56.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.316+0000 7fc6efbca700 1 -- 172.21.15.6:0/711068887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6e80b62d0 msgr2=0x7fc6e80cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.316+0000 7fc6efbca700 1 --2- 172.21.15.6:0/711068887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6e80b62d0 0x7fc6e80cb630 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80098f0 tx=0x7fc6d8009c00 comp rx=0 tx=0).stop 2023-11-28T13:01:56.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.316+0000 7fc6efbca700 1 -- 172.21.15.6:0/711068887 shutdown_connections 2023-11-28T13:01:56.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.316+0000 7fc6efbca700 1 --2- 172.21.15.6:0/711068887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6e80b62d0 0x7fc6e80cb630 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.316+0000 7fc6efbca700 1 --2- 172.21.15.6:0/711068887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 0x7fc6e80b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.316+0000 7fc6efbca700 1 -- 172.21.15.6:0/711068887 >> 172.21.15.6:0/711068887 conn(0x7fc6e80cd680 msgr2=0x7fc6e80cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:56.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.316+0000 7fc6efbca700 1 -- 172.21.15.6:0/711068887 shutdown_connections 2023-11-28T13:01:56.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.316+0000 7fc6efbca700 1 -- 172.21.15.6:0/711068887 wait complete. 2023-11-28T13:01:56.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.317+0000 7fc6efbca700 1 Processor -- start 2023-11-28T13:01:56.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.317+0000 7fc6efbca700 1 -- start start 2023-11-28T13:01:56.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6efbca700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 0x7fc6e8148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6efbca700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6e80b2bc0 0x7fc6e81e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6efbca700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80b62d0 0x7fc6e81591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6efbca700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8155870 con 0x7fc6e80b62d0 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6efbca700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81df920 con 0x7fc6e80b2bc0 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6efbca700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81dfc00 con 0x7fc6e806f8b0 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6ee167700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80b62d0 0x7fc6e81591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6ee167700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80b62d0 0x7fc6e81591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60766/0 (socket says 172.21.15.6:60766) 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6ee167700 1 -- 172.21.15.6:0/3573521841 learned_addr learned my addr 172.21.15.6:0/3573521841 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6ed165700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6e80b2bc0 0x7fc6e81e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6ed966700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 0x7fc6e8148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6ee167700 1 -- 172.21.15.6:0/3573521841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 msgr2=0x7fc6e8148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6ee167700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 0x7fc6e8148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6ee167700 1 -- 172.21.15.6:0/3573521841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6e80b2bc0 msgr2=0x7fc6e81e43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6ee167700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6e80b2bc0 0x7fc6e81e43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.318+0000 7fc6ee167700 1 -- 172.21.15.6:0/3573521841 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e40094a0 con 0x7fc6e80b62d0 2023-11-28T13:01:56.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.319+0000 7fc6ee167700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80b62d0 0x7fc6e81591d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0002230 tx=0x7fc6e000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:56.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.319+0000 7fc6de7fc700 1 -- 172.21.15.6:0/3573521841 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000ac20 con 0x7fc6e80b62d0 2023-11-28T13:01:56.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.319+0000 7fc6de7fc700 1 -- 172.21.15.6:0/3573521841 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e000ada0 con 0x7fc6e80b62d0 2023-11-28T13:01:56.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.319+0000 7fc6efbca700 1 -- 172.21.15.6:0/3573521841 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d8009550 con 0x7fc6e80b62d0 2023-11-28T13:01:56.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.319+0000 7fc6de7fc700 1 -- 172.21.15.6:0/3573521841 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0007310 con 0x7fc6e80b62d0 2023-11-28T13:01:56.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.319+0000 7fc6efbca700 1 -- 172.21.15.6:0/3573521841 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e80b66a0 con 0x7fc6e80b62d0 2023-11-28T13:01:56.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.320+0000 7fc6efbca700 1 -- 172.21.15.6:0/3573521841 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e80cffb0 con 0x7fc6e80b62d0 2023-11-28T13:01:56.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.320+0000 7fc6de7fc700 1 -- 172.21.15.6:0/3573521841 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc6e0010070 con 0x7fc6e80b62d0 2023-11-28T13:01:56.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.321+0000 7fc6de7fc700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc6cc068320 0x7fc6cc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.321+0000 7fc6ed966700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc6cc068320 0x7fc6cc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.321+0000 7fc6de7fc700 1 -- 172.21.15.6:0/3573521841 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc6e00857a0 con 0x7fc6e80b62d0 2023-11-28T13:01:56.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.321+0000 7fc6ed966700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc6cc068320 0x7fc6cc06a7d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8014040 tx=0x7fc6d8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:56.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.325+0000 7fc6de7fc700 1 -- 172.21.15.6:0/3573521841 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc6e0051ec0 con 0x7fc6e80b62d0 2023-11-28T13:01:56.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.517+0000 7fc6efbca700 1 -- 172.21.15.6:0/3573521841 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6e806fd00 con 0x7fc6e80b62d0 2023-11-28T13:01:56.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.518+0000 7fc6de7fc700 1 -- 172.21.15.6:0/3573521841 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6e0018020 con 0x7fc6e80b62d0 2023-11-28T13:01:56.519 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:01:56.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.519+0000 7fc6efbca700 1 -- 172.21.15.6:0/3573521841 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc6cc068320 msgr2=0x7fc6cc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.519+0000 7fc6efbca700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc6cc068320 0x7fc6cc06a7d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8014040 tx=0x7fc6d8008030 comp rx=0 tx=0).stop 2023-11-28T13:01:56.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.519+0000 7fc6efbca700 1 -- 172.21.15.6:0/3573521841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80b62d0 msgr2=0x7fc6e81591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.519+0000 7fc6efbca700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80b62d0 0x7fc6e81591d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0002230 tx=0x7fc6e000a060 comp rx=0 tx=0).stop 2023-11-28T13:01:56.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.519+0000 7fc6efbca700 1 -- 172.21.15.6:0/3573521841 shutdown_connections 2023-11-28T13:01:56.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.519+0000 7fc6efbca700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc6e80b62d0 0x7fc6e81591d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.519+0000 7fc6efbca700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc6cc068320 0x7fc6cc06a7d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.519+0000 7fc6efbca700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc6e806f8b0 0x7fc6e8148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.519+0000 7fc6efbca700 1 --2- 172.21.15.6:0/3573521841 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6e80b2bc0 0x7fc6e81e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.520+0000 7fc6efbca700 1 -- 172.21.15.6:0/3573521841 >> 172.21.15.6:0/3573521841 conn(0x7fc6e80cd680 msgr2=0x7fc6e80b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.520+0000 7fc6efbca700 1 -- 172.21.15.6:0/3573521841 shutdown_connections 2023-11-28T13:01:56.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.520+0000 7fc6efbca700 1 -- 172.21.15.6:0/3573521841 wait complete. 2023-11-28T13:01:56.533 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:01:56.658 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.657+0000 7fb16e70a700 1 Processor -- start 2023-11-28T13:01:56.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.657+0000 7fb16e70a700 1 -- start start 2023-11-28T13:01:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.658+0000 7fb16e70a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb16815aa20 0x7fb1680b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.658+0000 7fb16e70a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680ca990 con 0x7fb1680b5500 2023-11-28T13:01:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.658+0000 7fb16e70a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680caf50 con 0x7fb16815a650 2023-11-28T13:01:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.658+0000 7fb16e70a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1680aef10 con 0x7fb16815aa20 2023-11-28T13:01:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.658+0000 7fb16cca7700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb1680b5500 0x7fb1680cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48090/0 (socket says 172.21.15.6:48090) 2023-11-28T13:01:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.658+0000 7fb16cca7700 1 -- 172.21.15.6:0/3392843789 learned_addr learned my addr 172.21.15.6:0/3392843789 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.658+0000 7fb1677fe700 1 --2- 172.21.15.6:0/3392843789 >> v2:172.21.15.61:3300/0 conn(0x7fb16815aa20 0x7fb1680b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.658+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288715848 0 0) 0x7fb1680ca990 con 0x7fb1680b5500 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.658+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1540035b0 con 0x7fb1680b5500 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631887642 0 0) 0x7fb1680caf50 con 0x7fb16815a650 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1680ca990 con 0x7fb16815a650 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2663415408 0 0) 0x7fb1540035b0 con 0x7fb1680b5500 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 >> v2:172.21.15.61:3300/0 conn(0x7fb16815aa20 msgr2=0x7fb1680b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 --2- 172.21.15.6:0/3392843789 >> v2:172.21.15.61:3300/0 conn(0x7fb16815aa20 0x7fb1680b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 >> v1:172.21.15.61:6789/0 conn(0x7fb16815a650 legacy=0x7fb1680ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680af1f0 con 0x7fb1680b5500 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fb1600019f0 con 0x7fb1680b5500 2023-11-28T13:01:56.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb1677fe700 1 --2- 172.21.15.6:0/3392843789 >> v2:172.21.15.61:3300/0 conn(0x7fb16815aa20 0x7fb1680b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 >> v1:172.21.15.6:6789/0 conn(0x7fb1680b5500 legacy=0x7fb1680cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb154003be0 0x7fb154003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb154004850 0x7fb154004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb154005200 0x7fb15400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680aef10 con 0x7fb154003be0 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15400c0e0 con 0x7fb154005200 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15400c270 con 0x7fb154004850 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb167fff700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb154003be0 0x7fb154003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb1677fe700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb154004850 0x7fb154004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb16cca7700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb154005200 0x7fb15400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb167fff700 1 -- 172.21.15.6:0/3392843789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb154004850 msgr2=0x7fb154004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:56.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb167fff700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb154004850 0x7fb154004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb167fff700 1 -- 172.21.15.6:0/3392843789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb154005200 msgr2=0x7fb15400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb167fff700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb154005200 0x7fb15400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb16cca7700 1 -- 172.21.15.6:0/3392843789 reap_dead start 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb167fff700 1 -- 172.21.15.6:0/3392843789 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1540035b0 con 0x7fb154003be0 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.659+0000 7fb167fff700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb154003be0 0x7fb154003fd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb150007fd0 tx=0x7fb154005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcf7b1a49fb992c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.660+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150001430 con 0x7fb154003be0 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.660+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1500052f0 con 0x7fb154003be0 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.660+0000 7fb166ffd700 1 -- 172.21.15.6:0/3392843789 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150001d40 con 0x7fb154003be0 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.660+0000 7fb16e70a700 1 -- 172.21.15.6:0/3392843789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb154003be0 msgr2=0x7fb154003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.660+0000 7fb16e70a700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb154003be0 0x7fb154003fd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb150007fd0 tx=0x7fb154005880 comp rx=0 tx=0).stop 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.660+0000 7fb16e70a700 1 -- 172.21.15.6:0/3392843789 shutdown_connections 2023-11-28T13:01:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.660+0000 7fb16e70a700 1 --2- 172.21.15.6:0/3392843789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb154003be0 0x7fb154003fd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.660+0000 7fb16e70a700 1 -- 172.21.15.6:0/3392843789 >> 172.21.15.6:0/3392843789 conn(0x7fb1680b1650 msgr2=0x7fb1680b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.660+0000 7fb16e70a700 1 -- 172.21.15.6:0/3392843789 shutdown_connections 2023-11-28T13:01:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.660+0000 7fb16e70a700 1 -- 172.21.15.6:0/3392843789 wait complete. 2023-11-28T13:01:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.662+0000 7fb16e70a700 1 Processor -- start 2023-11-28T13:01:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.662+0000 7fb16e70a700 1 -- start start 2023-11-28T13:01:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb16e70a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb154003be0 0x7fb1681efd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb16e70a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1680b3450 0x7fb1680b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb16e70a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1680b38c0 0x7fb1681f3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb16e70a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680b3e20 con 0x7fb1680b3450 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb16e70a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681f4670 con 0x7fb154003be0 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb16e70a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16814d050 con 0x7fb1680b38c0 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb1677fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1680b3450 0x7fb1680b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb1677fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1680b3450 0x7fb1680b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60780/0 (socket says 172.21.15.6:60780) 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb1677fe700 1 -- 172.21.15.6:0/3937522268 learned_addr learned my addr 172.21.15.6:0/3937522268 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb167fff700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb154003be0 0x7fb1681efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb16cca7700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1680b38c0 0x7fb1681f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb1677fe700 1 -- 172.21.15.6:0/3937522268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1680b38c0 msgr2=0x7fb1681f3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb1677fe700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1680b38c0 0x7fb1681f3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb1677fe700 1 -- 172.21.15.6:0/3937522268 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb154003be0 msgr2=0x7fb1681efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb1677fe700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb154003be0 0x7fb1681efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb1677fe700 1 -- 172.21.15.6:0/3937522268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680b2950 con 0x7fb1680b3450 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb167fff700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb154003be0 0x7fb1681efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.663+0000 7fb1677fe700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1680b3450 0x7fb1680b4870 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb158011d00 tx=0x7fb15800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.664+0000 7fb164ff9700 1 -- 172.21.15.6:0/3937522268 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15800fb30 con 0x7fb1680b3450 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.664+0000 7fb164ff9700 1 -- 172.21.15.6:0/3937522268 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15800fc90 con 0x7fb1680b3450 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.664+0000 7fb16e70a700 1 -- 172.21.15.6:0/3937522268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1680cabe0 con 0x7fb1680b3450 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.664+0000 7fb164ff9700 1 -- 172.21.15.6:0/3937522268 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158004030 con 0x7fb1680b3450 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.664+0000 7fb16e70a700 1 -- 172.21.15.6:0/3937522268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1680b2ae0 con 0x7fb1680b3450 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.665+0000 7fb16e70a700 1 -- 172.21.15.6:0/3937522268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1680b46c0 con 0x7fb1680b3450 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.665+0000 7fb164ff9700 1 -- 172.21.15.6:0/3937522268 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb158024070 con 0x7fb1680b3450 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.665+0000 7fb164ff9700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb1480682f0 0x7fb14806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:01:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.665+0000 7fb164ff9700 1 -- 172.21.15.6:0/3937522268 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb1580863c0 con 0x7fb1680b3450 2023-11-28T13:01:56.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.666+0000 7fb167fff700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb1480682f0 0x7fb14806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:01:56.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.666+0000 7fb167fff700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb1480682f0 0x7fb14806a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb150015040 tx=0x7fb15000a000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:01:56.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.670+0000 7fb164ff9700 1 -- 172.21.15.6:0/3937522268 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb158052ae0 con 0x7fb1680b3450 2023-11-28T13:01:56.708 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:01:56.711 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:01:56.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.880+0000 7fb16e70a700 1 -- 172.21.15.6:0/3937522268 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb1680675a0 con 0x7fb1680b3450 2023-11-28T13:01:56.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.881+0000 7fb164ff9700 1 -- 172.21.15.6:0/3937522268 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb158022080 con 0x7fb1680b3450 2023-11-28T13:01:56.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 -- 172.21.15.6:0/3937522268 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb1480682f0 msgr2=0x7fb14806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb1480682f0 0x7fb14806a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb150015040 tx=0x7fb15000a000 comp rx=0 tx=0).stop 2023-11-28T13:01:56.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 -- 172.21.15.6:0/3937522268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1680b3450 msgr2=0x7fb1680b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:01:56.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1680b3450 0x7fb1680b4870 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb158011d00 tx=0x7fb15800f520 comp rx=0 tx=0).stop 2023-11-28T13:01:56.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 -- 172.21.15.6:0/3937522268 shutdown_connections 2023-11-28T13:01:56.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb1480682f0 0x7fb14806a7a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb1680b38c0 0x7fb1681f3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb154003be0 0x7fb1681efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 --2- 172.21.15.6:0/3937522268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1680b3450 0x7fb1680b4870 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:01:56.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 -- 172.21.15.6:0/3937522268 >> 172.21.15.6:0/3937522268 conn(0x7fb1680b1650 msgr2=0x7fb1681572b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:01:56.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 -- 172.21.15.6:0/3937522268 shutdown_connections 2023-11-28T13:01:56.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:01:56.883+0000 7fb16e70a700 1 -- 172.21.15.6:0/3937522268 wait complete. 2023-11-28T13:01:56.886 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:01:56.896 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2023-11-28T13:01:56.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:01:56.896 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T13:01:56.935 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[122161]: fuse finished with error 0 and tester_r 0 2023-11-28T13:01:56.936 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:02:03.038 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:02:03.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:03.039 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:02:03.060 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:03.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.236+0000 7f7246197700 1 Processor -- start 2023-11-28T13:02:03.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.236+0000 7f7246197700 1 -- start start 2023-11-28T13:02:03.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.236+0000 7f7246197700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f724015a6c0 0x7f72400cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.236+0000 7f7246197700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 0x7f72400ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.236+0000 7f7246197700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72400cfb90 con 0x7f72400b2660 2023-11-28T13:02:03.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.236+0000 7f7246197700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72400cb930 con 0x7f724015a6c0 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.236+0000 7f7246197700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72400cf260 con 0x7f72400d0020 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723f7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f724015a6c0 0x7f72400cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723f7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f724015a6c0 0x7f72400cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60788/0 (socket says 172.21.15.6:60788) 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723f7fe700 1 -- 172.21.15.6:0/1871811194 learned_addr learned my addr 172.21.15.6:0/1871811194 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723effd700 1 --2- 172.21.15.6:0/1871811194 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 0x7f72400ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723f7fe700 1 -- 172.21.15.6:0/1871811194 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 msgr2=0x7f72400ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723f7fe700 1 --2- 172.21.15.6:0/1871811194 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 0x7f72400ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723f7fe700 1 -- 172.21.15.6:0/1871811194 >> v1:172.21.15.61:6789/0 conn(0x7f72400b2660 legacy=0x7f72400b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723f7fe700 1 -- 172.21.15.6:0/1871811194 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72400cf540 con 0x7f724015a6c0 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723f7fe700 1 --2- 172.21.15.6:0/1871811194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 0x7f72400cb230 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f72280098f0 tx=0x7f7228009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1994d97491ab1b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723e7fc700 1 -- 172.21.15.6:0/1871811194 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7228004230 con 0x7f724015a6c0 2023-11-28T13:02:03.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723e7fc700 1 -- 172.21.15.6:0/1871811194 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7228004390 con 0x7f724015a6c0 2023-11-28T13:02:03.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.237+0000 7f723e7fc700 1 -- 172.21.15.6:0/1871811194 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72280036a0 con 0x7f724015a6c0 2023-11-28T13:02:03.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.238+0000 7f7246197700 1 -- 172.21.15.6:0/1871811194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 msgr2=0x7f72400cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.238+0000 7f7246197700 1 --2- 172.21.15.6:0/1871811194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 0x7f72400cb230 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f72280098f0 tx=0x7f7228009c00 comp rx=0 tx=0).stop 2023-11-28T13:02:03.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.238+0000 7f7246197700 1 -- 172.21.15.6:0/1871811194 shutdown_connections 2023-11-28T13:02:03.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.238+0000 7f7246197700 1 --2- 172.21.15.6:0/1871811194 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 0x7f72400cb230 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.238+0000 7f7246197700 1 --2- 172.21.15.6:0/1871811194 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 0x7f72400ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.238+0000 7f7246197700 1 -- 172.21.15.6:0/1871811194 >> 172.21.15.6:0/1871811194 conn(0x7f72400cd660 msgr2=0x7f72400cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:03.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.238+0000 7f7246197700 1 -- 172.21.15.6:0/1871811194 shutdown_connections 2023-11-28T13:02:03.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.238+0000 7f7246197700 1 -- 172.21.15.6:0/1871811194 wait complete. 2023-11-28T13:02:03.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.239+0000 7f7246197700 1 Processor -- start 2023-11-28T13:02:03.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.239+0000 7f7246197700 1 -- start start 2023-11-28T13:02:03.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f7246197700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72400b2660 0x7f72400adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f7246197700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 0x7f72400ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f7246197700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 0x7f72401f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f7246197700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72400af220 con 0x7f724015a6c0 2023-11-28T13:02:03.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f7246197700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72400ae5a0 con 0x7f72400b2660 2023-11-28T13:02:03.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f7246197700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72400ae880 con 0x7f72400d0020 2023-11-28T13:02:03.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f723ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 0x7f72401f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f723ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 0x7f72401f1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60798/0 (socket says 172.21.15.6:60798) 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f723ffff700 1 -- 172.21.15.6:0/3762746415 learned_addr learned my addr 172.21.15.6:0/3762746415 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f723effd700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 0x7f72400ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f723f7fe700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72400b2660 0x7f72400adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f723ffff700 1 -- 172.21.15.6:0/3762746415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 msgr2=0x7f72400ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f723ffff700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 0x7f72400ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f723ffff700 1 -- 172.21.15.6:0/3762746415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72400b2660 msgr2=0x7f72400adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f723ffff700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72400b2660 0x7f72400adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.240+0000 7f723ffff700 1 -- 172.21.15.6:0/3762746415 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7228009550 con 0x7f724015a6c0 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.241+0000 7f723effd700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 0x7f72400ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.241+0000 7f723ffff700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 0x7f72401f1d80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f723800cc10 tx=0x7f723800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.241+0000 7f7237fff700 1 -- 172.21.15.6:0/3762746415 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723800ab10 con 0x7f724015a6c0 2023-11-28T13:02:03.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.241+0000 7f7237fff700 1 -- 172.21.15.6:0/3762746415 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723800ac70 con 0x7f724015a6c0 2023-11-28T13:02:03.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.241+0000 7f7246197700 1 -- 172.21.15.6:0/3762746415 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72400cd3e0 con 0x7f724015a6c0 2023-11-28T13:02:03.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.241+0000 7f7237fff700 1 -- 172.21.15.6:0/3762746415 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238012420 con 0x7f724015a6c0 2023-11-28T13:02:03.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.241+0000 7f7246197700 1 -- 172.21.15.6:0/3762746415 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72400cf820 con 0x7f724015a6c0 2023-11-28T13:02:03.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.242+0000 7f7246197700 1 -- 172.21.15.6:0/3762746415 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72400d0470 con 0x7f724015a6c0 2023-11-28T13:02:03.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.242+0000 7f7237fff700 1 -- 172.21.15.6:0/3762746415 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7238010050 con 0x7f724015a6c0 2023-11-28T13:02:03.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.243+0000 7f7237fff700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72200682f0 0x7f722006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.243+0000 7f7237fff700 1 -- 172.21.15.6:0/3762746415 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7238085f50 con 0x7f724015a6c0 2023-11-28T13:02:03.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.243+0000 7f723f7fe700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72200682f0 0x7f722006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.243+0000 7f723f7fe700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72200682f0 0x7f722006a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7228009130 tx=0x7f7228008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:03.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.247+0000 7f7237fff700 1 -- 172.21.15.6:0/3762746415 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7238052610 con 0x7f724015a6c0 2023-11-28T13:02:03.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.483+0000 7f7246197700 1 -- 172.21.15.6:0/3762746415 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72400af360 con 0x7f724015a6c0 2023-11-28T13:02:03.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.483+0000 7f7237fff700 1 -- 172.21.15.6:0/3762746415 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f72400af360 con 0x7f724015a6c0 2023-11-28T13:02:03.485 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:03.485 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4026920958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4026920958},{"type":"v1","addr":"172.21.15.61:6833","nonce":4026920958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2212733196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2212733196},{"type":"v1","addr":"172.21.15.61:6835","nonce":2212733196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1674709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1674709721},{"type":"v1","addr":"172.21.15.6:6836","nonce":1674709721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:42.595217+0000","modified":"2023-11-28T13:01:45.602537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/2789212407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":2789212407},{"type":"v1","addr":"172.21.15.6:6837","nonce":2789212407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:02:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 -- 172.21.15.6:0/3762746415 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72200682f0 msgr2=0x7f722006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72200682f0 0x7f722006a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7228009130 tx=0x7f7228008030 comp rx=0 tx=0).stop 2023-11-28T13:02:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 -- 172.21.15.6:0/3762746415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 msgr2=0x7f72401f1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 0x7f72401f1d80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f723800cc10 tx=0x7f723800cf20 comp rx=0 tx=0).stop 2023-11-28T13:02:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 -- 172.21.15.6:0/3762746415 shutdown_connections 2023-11-28T13:02:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724015a6c0 0x7f72401f1d80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f72400b2660 0x7f72400adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f72200682f0 0x7f722006a7a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 --2- 172.21.15.6:0/3762746415 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f72400d0020 0x7f72400ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 -- 172.21.15.6:0/3762746415 >> 172.21.15.6:0/3762746415 conn(0x7f72400cd660 msgr2=0x7f724006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 -- 172.21.15.6:0/3762746415 shutdown_connections 2023-11-28T13:02:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.485+0000 7f7246197700 1 -- 172.21.15.6:0/3762746415 wait complete. 2023-11-28T13:02:03.488 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2023-11-28T13:02:03.500 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-28T13:02:03.659 INFO:teuthology.orchestra.run.smithi006.stdout:{"message":"","return_code":0} 2023-11-28T13:02:03.660 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:02:03.660 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:03.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.783+0000 7f37a7972700 1 Processor -- start 2023-11-28T13:02:03.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.783+0000 7f37a7972700 1 -- start start 2023-11-28T13:02:03.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.784+0000 7f37a7972700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37a00b2660 0x7f37a00b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.784+0000 7f37a7972700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a00cfb90 con 0x7f37a00cb230 2023-11-28T13:02:03.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.784+0000 7f37a7972700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a00cb680 con 0x7f37a015a6c0 2023-11-28T13:02:03.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.784+0000 7f37a7972700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00b3020 con 0x7f37a00b2660 2023-11-28T13:02:03.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.784+0000 7f37a4f0d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f37a00cb230 0x7f37a00ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48124/0 (socket says 172.21.15.6:48124) 2023-11-28T13:02:03.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.784+0000 7f37a4f0d700 1 -- 172.21.15.6:0/1828476419 learned_addr learned my addr 172.21.15.6:0/1828476419 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:03.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154175593 0 0) 0x7f37a00cfb90 con 0x7f37a00cb230 2023-11-28T13:02:03.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f378c0035b0 con 0x7f37a00cb230 2023-11-28T13:02:03.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f37a5f0f700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37a00b2660 0x7f37a00b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1695613670 0 0) 0x7f378c0035b0 con 0x7f37a00cb230 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37a00b2660 msgr2=0x7f37a00b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37a00b2660 0x7f37a00b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 >> v1:172.21.15.61:6789/0 conn(0x7f37a015a6c0 legacy=0x7f37a00d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a00cb930 con 0x7f37a00cb230 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f379c001670 con 0x7f37a00cb230 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 >> v1:172.21.15.6:6789/0 conn(0x7f37a00cb230 legacy=0x7f37a00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37a00b2660 0x7f378c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f378c003fb0 0x7f378c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f378c004d90 0x7f378c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00b3020 con 0x7f378c003fb0 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c00bb80 con 0x7f378c004d90 2023-11-28T13:02:03.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c00bd40 con 0x7f37a00b2660 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f37a4f0d700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f378c003fb0 0x7f378c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f37a570e700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37a00b2660 0x7f378c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f37a5f0f700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f378c004d90 0x7f378c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f37a4f0d700 1 -- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37a00b2660 msgr2=0x7f378c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f37a4f0d700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f37a00b2660 0x7f378c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f37a4f0d700 1 -- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f378c004d90 msgr2=0x7f378c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.785+0000 7f37a4f0d700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f378c004d90 0x7f378c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.786+0000 7f37a4f0d700 1 -- 172.21.15.6:0/1828476419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c0035b0 con 0x7f378c003fb0 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.786+0000 7f37a4f0d700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f378c003fb0 0x7f378c004850 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c210 tx=0x7f378c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2ceaf36b37eb0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.786+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c007c10 con 0x7f378c003fb0 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.786+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00cdb0 con 0x7f378c003fb0 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.786+0000 7f3797fff700 1 -- 172.21.15.6:0/1828476419 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c005ba0 con 0x7f378c003fb0 2023-11-28T13:02:03.800 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.786+0000 7f37a7972700 1 -- 172.21.15.6:0/1828476419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f378c003fb0 msgr2=0x7f378c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.786+0000 7f37a7972700 1 --2- 172.21.15.6:0/1828476419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f378c003fb0 0x7f378c004850 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c210 tx=0x7f378c005880 comp rx=0 tx=0).stop 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.786+0000 7f37a5f0f700 1 -- 172.21.15.6:0/1828476419 reap_dead start 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.786+0000 7f37a7972700 1 -- 172.21.15.6:0/1828476419 shutdown_connections 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.786+0000 7f37a7972700 1 -- 172.21.15.6:0/1828476419 >> 172.21.15.6:0/1828476419 conn(0x7f37a00cd660 msgr2=0x7f37a00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.787+0000 7f37a7972700 1 -- 172.21.15.6:0/1828476419 shutdown_connections 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.787+0000 7f37a7972700 1 -- 172.21.15.6:0/1828476419 wait complete. 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.788+0000 7f37a7972700 1 Processor -- start 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.788+0000 7f37a7972700 1 -- start start 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.788+0000 7f37a7972700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f378c003fb0 0x7f37a0158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.788+0000 7f37a7972700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37a01f2970 0x7f37a00c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.788+0000 7f37a7972700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37a00c6a60 0x7f37a00c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.788+0000 7f37a7972700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a0154ae0 con 0x7f37a00c6a60 2023-11-28T13:02:03.801 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.788+0000 7f37a7972700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a0157760 con 0x7f37a01f2970 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.788+0000 7f37a7972700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a0157a40 con 0x7f378c003fb0 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a5f0f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37a00c6a60 0x7f37a00c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a5f0f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37a00c6a60 0x7f37a00c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60818/0 (socket says 172.21.15.6:60818) 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a5f0f700 1 -- 172.21.15.6:0/366969835 learned_addr learned my addr 172.21.15.6:0/366969835 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a4f0d700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37a01f2970 0x7f37a00c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a570e700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f378c003fb0 0x7f37a0158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a5f0f700 1 -- 172.21.15.6:0/366969835 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f378c003fb0 msgr2=0x7f37a0158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a5f0f700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f378c003fb0 0x7f37a0158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a5f0f700 1 -- 172.21.15.6:0/366969835 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37a01f2970 msgr2=0x7f37a00c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a5f0f700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37a01f2970 0x7f37a00c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a5f0f700 1 -- 172.21.15.6:0/366969835 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a00cf540 con 0x7f37a00c6a60 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a570e700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f378c003fb0 0x7f37a0158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:03.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a4f0d700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37a01f2970 0x7f37a00c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a5f0f700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37a00c6a60 0x7f37a00c6ed0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f37a0072970 tx=0x7f3798011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f3795ffb700 1 -- 172.21.15.6:0/366969835 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379800e040 con 0x7f37a00c6a60 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f3795ffb700 1 -- 172.21.15.6:0/366969835 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379800f040 con 0x7f37a00c6a60 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f37a7972700 1 -- 172.21.15.6:0/366969835 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a00cf260 con 0x7f37a00c6a60 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.789+0000 7f3795ffb700 1 -- 172.21.15.6:0/366969835 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379800fdb0 con 0x7f37a00c6a60 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.790+0000 7f37a7972700 1 -- 172.21.15.6:0/366969835 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a00cf820 con 0x7f37a00c6a60 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.791+0000 7f37a7972700 1 -- 172.21.15.6:0/366969835 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a00675a0 con 0x7f37a00c6a60 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.791+0000 7f3795ffb700 1 -- 172.21.15.6:0/366969835 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3798022460 con 0x7f37a00c6a60 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.791+0000 7f3795ffb700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3784068320 0x7f378406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.791+0000 7f3795ffb700 1 -- 172.21.15.6:0/366969835 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f37980864a0 con 0x7f37a00c6a60 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.791+0000 7f37a570e700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3784068320 0x7f378406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:03.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.792+0000 7f37a570e700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3784068320 0x7f378406a7d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3790007fd0 tx=0x7f3790009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:03.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:03.796+0000 7f3795ffb700 1 -- 172.21.15.6:0/366969835 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3798052b60 con 0x7f37a00c6a60 2023-11-28T13:02:04.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.036+0000 7f37a7972700 1 -- 172.21.15.6:0/366969835 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37a00d0410 con 0x7f37a00c6a60 2023-11-28T13:02:04.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.036+0000 7f3795ffb700 1 -- 172.21.15.6:0/366969835 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f3798056180 con 0x7f37a00c6a60 2023-11-28T13:02:04.038 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:04.038 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4026920958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4026920958},{"type":"v1","addr":"172.21.15.61:6833","nonce":4026920958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2212733196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2212733196},{"type":"v1","addr":"172.21.15.61:6835","nonce":2212733196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1674709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1674709721},{"type":"v1","addr":"172.21.15.6:6836","nonce":1674709721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:42.595217+0000","modified":"2023-11-28T13:01:45.602537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/2789212407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":2789212407},{"type":"v1","addr":"172.21.15.6:6837","nonce":2789212407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:02:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 -- 172.21.15.6:0/366969835 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3784068320 msgr2=0x7f378406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3784068320 0x7f378406a7d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3790007fd0 tx=0x7f3790009040 comp rx=0 tx=0).stop 2023-11-28T13:02:04.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 -- 172.21.15.6:0/366969835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37a00c6a60 msgr2=0x7f37a00c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:04.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37a00c6a60 0x7f37a00c6ed0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f37a0072970 tx=0x7f3798011f80 comp rx=0 tx=0).stop 2023-11-28T13:02:04.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 -- 172.21.15.6:0/366969835 shutdown_connections 2023-11-28T13:02:04.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37a00c6a60 0x7f37a00c6ed0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3784068320 0x7f378406a7d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f378c003fb0 0x7f37a0158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 --2- 172.21.15.6:0/366969835 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37a01f2970 0x7f37a00c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:04.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 -- 172.21.15.6:0/366969835 >> 172.21.15.6:0/366969835 conn(0x7f37a00cd660 msgr2=0x7f37a00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:04.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 -- 172.21.15.6:0/366969835 shutdown_connections 2023-11-28T13:02:04.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:04.039+0000 7f37a7972700 1 -- 172.21.15.6:0/366969835 wait complete. 2023-11-28T13:02:04.042 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2023-11-28T13:02:04.054 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2023-11-28T13:02:04.200 INFO:teuthology.orchestra.run.smithi006.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-28T13:02:04.200 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-28T13:02:14.201 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-28T13:02:14.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.251+0000 7f7cbaf8a080 1 Processor -- start 2023-11-28T13:02:14.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.251+0000 7f7cbaf8a080 1 -- start start 2023-11-28T13:02:14.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cbaf8a080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560566699d40 0x560566727e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cbaf8a080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605667299b0 0x56056672a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cbaf8a080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560566729d80 0x56056669a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cbaf8a080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x56056658ffe0 con 0x5605667299b0 2023-11-28T13:02:14.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cbaf8a080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560566681670 con 0x560566729d80 2023-11-28T13:02:14.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cbaf8a080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605666992e0 con 0x560566699d40 2023-11-28T13:02:14.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb088c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605667299b0 0x56056672a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb088c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605667299b0 0x56056672a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60830/0 (socket says 172.21.15.6:60830) 2023-11-28T13:02:14.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb088c700 1 -- 172.21.15.6:0/3708686047 learned_addr learned my addr 172.21.15.6:0/3708686047 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:14.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb188e700 1 --2- 172.21.15.6:0/3708686047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560566729d80 0x56056669a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb108d700 1 --2- 172.21.15.6:0/3708686047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560566699d40 0x560566727e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb088c700 1 -- 172.21.15.6:0/3708686047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560566699d40 msgr2=0x560566727e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:14.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb088c700 1 --2- 172.21.15.6:0/3708686047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560566699d40 0x560566727e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb088c700 1 -- 172.21.15.6:0/3708686047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560566729d80 msgr2=0x56056669a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb088c700 1 --2- 172.21.15.6:0/3708686047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560566729d80 0x56056669a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb088c700 1 -- 172.21.15.6:0/3708686047 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c005d20 con 0x5605667299b0 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb188e700 1 --2- 172.21.15.6:0/3708686047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560566729d80 0x56056669a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.252+0000 7f7cb088c700 1 --2- 172.21.15.6:0/3708686047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605667299b0 0x56056672a1d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400a8f0 tx=0x7f7ca400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7289770373a4ef3e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cab7fe700 1 -- 172.21.15.6:0/3708686047 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4008b60 con 0x5605667299b0 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cab7fe700 1 -- 172.21.15.6:0/3708686047 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca4008ce0 con 0x5605667299b0 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cab7fe700 1 -- 172.21.15.6:0/3708686047 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca401c650 con 0x5605667299b0 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3708686047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605667299b0 msgr2=0x56056672a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cbaf8a080 1 --2- 172.21.15.6:0/3708686047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605667299b0 0x56056672a1d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400a8f0 tx=0x7f7ca400d000 comp rx=0 tx=0).stop 2023-11-28T13:02:14.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3708686047 shutdown_connections 2023-11-28T13:02:14.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cbaf8a080 1 --2- 172.21.15.6:0/3708686047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560566729d80 0x56056669a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cbaf8a080 1 --2- 172.21.15.6:0/3708686047 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x560566699d40 0x560566727e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cbaf8a080 1 --2- 172.21.15.6:0/3708686047 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x5605667299b0 0x56056672a1d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3708686047 >> 172.21.15.6:0/3708686047 conn(0x56056669c7e0 msgr2=0x5605666560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:14.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3708686047 shutdown_connections 2023-11-28T13:02:14.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.253+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3708686047 wait complete. 2023-11-28T13:02:14.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.254+0000 7f7cbaf8a080 1 Processor -- start 2023-11-28T13:02:14.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.254+0000 7f7cbaf8a080 1 -- start start 2023-11-28T13:02:14.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cbaf8a080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560566699d40 0x56056671ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cbaf8a080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x5605667299b0 0x56056672af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cbaf8a080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560566729d80 0x56056672b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cbaf8a080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x560566722780 con 0x560566729d80 2023-11-28T13:02:14.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cbaf8a080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56056671ef60 con 0x560566699d40 2023-11-28T13:02:14.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cbaf8a080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x56056669a6b0 con 0x5605667299b0 2023-11-28T13:02:14.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb188e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560566729d80 0x56056672b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb188e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560566729d80 0x56056672b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60838/0 (socket says 172.21.15.6:60838) 2023-11-28T13:02:14.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb188e700 1 -- 172.21.15.6:0/3046885374 learned_addr learned my addr 172.21.15.6:0/3046885374 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:14.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb088c700 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x5605667299b0 0x56056672af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb108d700 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560566699d40 0x56056671ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb188e700 1 -- 172.21.15.6:0/3046885374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x5605667299b0 msgr2=0x56056672af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb188e700 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x5605667299b0 0x56056672af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb188e700 1 -- 172.21.15.6:0/3046885374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560566699d40 msgr2=0x56056671ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb188e700 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560566699d40 0x56056671ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb188e700 1 -- 172.21.15.6:0/3046885374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605665c24f0 con 0x560566729d80 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.255+0000 7f7cb188e700 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560566729d80 0x56056672b460 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7cac004df0 tx=0x7f7cac001780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.256+0000 7f7caa7fc700 1 -- 172.21.15.6:0/3046885374 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac002060 con 0x560566729d80 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.256+0000 7f7caa7fc700 1 -- 172.21.15.6:0/3046885374 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac013480 con 0x560566729d80 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.256+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3046885374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56056659b830 con 0x560566729d80 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.256+0000 7f7caa7fc700 1 -- 172.21.15.6:0/3046885374 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac013830 con 0x560566729d80 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.256+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3046885374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca4005de0 con 0x560566729d80 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.257+0000 7f7caa7fc700 1 -- 172.21.15.6:0/3046885374 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7cac01a030 con 0x560566729d80 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.258+0000 7f7caa7fc700 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7c94066c10 0x7f7c940690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.258+0000 7f7caa7fc700 1 -- 172.21.15.6:0/3046885374 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7cac0842e0 con 0x560566729d80 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.258+0000 7f7cb108d700 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7c94066c10 0x7f7c940690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.258+0000 7f7cbaf8a080 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x560566726d80 0x560566720fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.258+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3046885374 --> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] -- osd_op(unknown.0.0:1 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x56056672aac0 con 0x560566726d80 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.258+0000 7f7cb088c700 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x560566726d80 0x560566720fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.258+0000 7f7cb108d700 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7c94066c10 0x7f7c940690c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5605666568d0 tx=0x7f7c9c00c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.258+0000 7f7cb088c700 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x560566726d80 0x560566720fd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cb088c700 1 -- 172.21.15.6:0/3046885374 <== osd.0 v2:172.21.15.6:6809/103502 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f7ca4008960 con 0x560566726d80 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x560566726d80 msgr2=0x560566720fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cbaf8a080 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x560566726d80 0x560566720fd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7c94066c10 msgr2=0x7f7c940690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cbaf8a080 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7c94066c10 0x7f7c940690c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5605666568d0 tx=0x7f7c9c00c040 comp rx=0 tx=0).stop 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560566729d80 msgr2=0x56056672b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cbaf8a080 1 --2- 172.21.15.6:0/3046885374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x560566729d80 0x56056672b460 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7cac004df0 tx=0x7f7cac001780 comp rx=0 tx=0).stop 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cb188e700 1 -- 172.21.15.6:0/3046885374 reap_dead start 2023-11-28T13:02:14.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3046885374 shutdown_connections 2023-11-28T13:02:14.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3046885374 >> 172.21.15.6:0/3046885374 conn(0x56056669c7e0 msgr2=0x560566655dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:14.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3046885374 shutdown_connections 2023-11-28T13:02:14.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.259+0000 7f7cbaf8a080 1 -- 172.21.15.6:0/3046885374 wait complete. 2023-11-28T13:02:14.266 INFO:teuthology.orchestra.run.smithi006.stdout:mytag 2023-11-28T13:02:14.267 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-28T13:02:14.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bdc025080 1 Processor -- start 2023-11-28T13:02:14.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bdc025080 1 -- start start 2023-11-28T13:02:14.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bdc025080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55bf5fc57330 0x55bf5fc2bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bdc025080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc591d0 0x55bf5fc56000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bdc025080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55bf5fc5a230 0x55bf5fc56540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bdc025080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5fbd1000 con 0x55bf5fc591d0 2023-11-28T13:02:14.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bdc025080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5fb63fe0 con 0x55bf5fc5a230 2023-11-28T13:02:14.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bdc025080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf5fc292b0 con 0x55bf5fc57330 2023-11-28T13:02:14.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bd1927700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc591d0 0x55bf5fc56000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bd1927700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc591d0 0x55bf5fc56000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60852/0 (socket says 172.21.15.6:60852) 2023-11-28T13:02:14.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.313+0000 7f6bd1927700 1 -- 172.21.15.6:0/1606931335 learned_addr learned my addr 172.21.15.6:0/1606931335 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd2929700 1 --2- 172.21.15.6:0/1606931335 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55bf5fc5a230 0x55bf5fc56540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd2128700 1 --2- 172.21.15.6:0/1606931335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55bf5fc57330 0x55bf5fc2bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd1927700 1 -- 172.21.15.6:0/1606931335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55bf5fc57330 msgr2=0x55bf5fc2bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd1927700 1 --2- 172.21.15.6:0/1606931335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55bf5fc57330 0x55bf5fc2bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd1927700 1 -- 172.21.15.6:0/1606931335 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55bf5fc5a230 msgr2=0x55bf5fc56540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd1927700 1 --2- 172.21.15.6:0/1606931335 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55bf5fc5a230 0x55bf5fc56540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd1927700 1 -- 172.21.15.6:0/1606931335 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc4005d20 con 0x55bf5fc591d0 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd1927700 1 --2- 172.21.15.6:0/1606931335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc591d0 0x55bf5fc56000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55bf5fc15f00 tx=0x7f6bc400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297dd832e403459a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd0925700 1 -- 172.21.15.6:0/1606931335 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4008960 con 0x55bf5fc591d0 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd0925700 1 -- 172.21.15.6:0/1606931335 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc4008b00 con 0x55bf5fc591d0 2023-11-28T13:02:14.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.314+0000 7f6bd0925700 1 -- 172.21.15.6:0/1606931335 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc401d430 con 0x55bf5fc591d0 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 -- 172.21.15.6:0/1606931335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc591d0 msgr2=0x55bf5fc56000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 --2- 172.21.15.6:0/1606931335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc591d0 0x55bf5fc56000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55bf5fc15f00 tx=0x7f6bc400d000 comp rx=0 tx=0).stop 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 -- 172.21.15.6:0/1606931335 shutdown_connections 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 --2- 172.21.15.6:0/1606931335 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55bf5fc5a230 0x55bf5fc56540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 --2- 172.21.15.6:0/1606931335 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55bf5fc57330 0x55bf5fc2bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 --2- 172.21.15.6:0/1606931335 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc591d0 0x55bf5fc56000 secure :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55bf5fc15f00 tx=0x7f6bc400d000 comp rx=0 tx=0).stop 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 -- 172.21.15.6:0/1606931335 >> 172.21.15.6:0/1606931335 conn(0x55bf5fc72130 msgr2=0x55bf5fc42bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 -- 172.21.15.6:0/1606931335 shutdown_connections 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 -- 172.21.15.6:0/1606931335 wait complete. 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 Processor -- start 2023-11-28T13:02:14.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.315+0000 7f6bdc025080 1 -- start start 2023-11-28T13:02:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.316+0000 7f6bdc025080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55bf5fc57330 0x55bf5fc73240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.316+0000 7f6bdc025080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55bf5fc5a230 0x55bf5fcf3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.316+0000 7f6bdc025080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc6bab0 0x55bf5fc52ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.316+0000 7f6bdc025080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5fc56b60 con 0x55bf5fc6bab0 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.316+0000 7f6bdc025080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5fc55f50 con 0x55bf5fc57330 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.316+0000 7f6bdc025080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf5fc56230 con 0x55bf5fc5a230 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.316+0000 7f6bd2929700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc6bab0 0x55bf5fc52ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.316+0000 7f6bd2929700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc6bab0 0x55bf5fc52ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60860/0 (socket says 172.21.15.6:60860) 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.316+0000 7f6bd2929700 1 -- 172.21.15.6:0/1324480369 learned_addr learned my addr 172.21.15.6:0/1324480369 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bd1927700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55bf5fc5a230 0x55bf5fcf3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bd2128700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55bf5fc57330 0x55bf5fc73240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bd2929700 1 -- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55bf5fc5a230 msgr2=0x55bf5fcf3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bd2929700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55bf5fc5a230 0x55bf5fcf3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bd2929700 1 -- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55bf5fc57330 msgr2=0x55bf5fc73240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bd2929700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55bf5fc57330 0x55bf5fc73240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bd2929700 1 -- 172.21.15.6:0/1324480369 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf5fb7c120 con 0x55bf5fc6bab0 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bd1927700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55bf5fc5a230 0x55bf5fcf3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bd2929700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc6bab0 0x55bf5fc52ae0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc004e70 tx=0x7f6bcc001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bc37fe700 1 -- 172.21.15.6:0/1324480369 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc002080 con 0x55bf5fc6bab0 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bc37fe700 1 -- 172.21.15.6:0/1324480369 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc011380 con 0x55bf5fc6bab0 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bdc025080 1 -- 172.21.15.6:0/1324480369 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf5fb6f830 con 0x55bf5fc6bab0 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bc37fe700 1 -- 172.21.15.6:0/1324480369 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc011730 con 0x55bf5fc6bab0 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.317+0000 7f6bdc025080 1 -- 172.21.15.6:0/1324480369 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf5fb7c360 con 0x55bf5fc6bab0 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.318+0000 7f6bc37fe700 1 -- 172.21.15.6:0/1324480369 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6bcc011890 con 0x55bf5fc6bab0 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.319+0000 7f6bc37fe700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6bb4066c10 0x7f6bb40690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.319+0000 7f6bc37fe700 1 -- 172.21.15.6:0/1324480369 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6bcc083bc0 con 0x55bf5fc6bab0 2023-11-28T13:02:14.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.319+0000 7f6bd2128700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6bb4066c10 0x7f6bb40690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.319+0000 7f6bdc025080 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55bf5fc754a0 0x55bf5fc75870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.319+0000 7f6bdc025080 1 -- 172.21.15.6:0/1324480369 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- osd_op(unknown.0.0:1 6.7 6:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55bf5fc75db0 con 0x55bf5fc754a0 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.319+0000 7f6bd1927700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55bf5fc754a0 0x55bf5fc75870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.319+0000 7f6bd2128700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6bb4066c10 0x7f6bb40690c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55bf5fd00bf0 tx=0x7f6bc8005640 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bd1927700 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55bf5fc754a0 0x55bf5fc75870 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bd1927700 1 -- 172.21.15.6:0/1324480369 <== osd.5 v2:172.21.15.61:6808/103780 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f6bc4008d70 con 0x55bf5fc754a0 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bdc025080 1 -- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55bf5fc754a0 msgr2=0x55bf5fc75870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bdc025080 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55bf5fc754a0 0x55bf5fc75870 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bdc025080 1 -- 172.21.15.6:0/1324480369 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6bb4066c10 msgr2=0x7f6bb40690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bdc025080 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6bb4066c10 0x7f6bb40690c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55bf5fd00bf0 tx=0x7f6bc8005640 comp rx=0 tx=0).stop 2023-11-28T13:02:14.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bdc025080 1 -- 172.21.15.6:0/1324480369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc6bab0 msgr2=0x55bf5fc52ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bdc025080 1 --2- 172.21.15.6:0/1324480369 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55bf5fc6bab0 0x55bf5fc52ae0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc004e70 tx=0x7f6bcc001910 comp rx=0 tx=0).stop 2023-11-28T13:02:14.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bd2929700 1 -- 172.21.15.6:0/1324480369 reap_dead start 2023-11-28T13:02:14.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bdc025080 1 -- 172.21.15.6:0/1324480369 shutdown_connections 2023-11-28T13:02:14.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bdc025080 1 -- 172.21.15.6:0/1324480369 >> 172.21.15.6:0/1324480369 conn(0x55bf5fc72130 msgr2=0x55bf5fc42bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:14.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bdc025080 1 -- 172.21.15.6:0/1324480369 shutdown_connections 2023-11-28T13:02:14.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.320+0000 7f6bdc025080 1 -- 172.21.15.6:0/1324480369 wait complete. 2023-11-28T13:02:14.328 INFO:teuthology.orchestra.run.smithi006.stdout:mytag 2023-11-28T13:02:14.328 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-28T13:02:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.372+0000 7fa2d01ad080 1 Processor -- start 2023-11-28T13:02:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.372+0000 7fa2d01ad080 1 -- start start 2023-11-28T13:02:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2d01ad080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001d51c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2d01ad080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e001ddb720 0x55e001d31440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2d01ad080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e001dde410 0x55e001d31980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2d01ad080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e001d4eca0 con 0x55e001d06bc0 2023-11-28T13:02:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2d01ad080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e001d50b10 con 0x55e001dde410 2023-11-28T13:02:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2d01ad080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e001d4f320 con 0x55e001ddb720 2023-11-28T13:02:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2c62b0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001d51c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2c62b0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001d51c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60872/0 (socket says 172.21.15.6:60872) 2023-11-28T13:02:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2c62b0700 1 -- 172.21.15.6:0/2098950613 learned_addr learned my addr 172.21.15.6:0/2098950613 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2c5aaf700 1 --2- 172.21.15.6:0/2098950613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e001ddb720 0x55e001d31440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2c6ab1700 1 --2- 172.21.15.6:0/2098950613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e001dde410 0x55e001d31980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2c62b0700 1 -- 172.21.15.6:0/2098950613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e001ddb720 msgr2=0x55e001d31440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2c62b0700 1 --2- 172.21.15.6:0/2098950613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e001ddb720 0x55e001d31440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2c62b0700 1 -- 172.21.15.6:0/2098950613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e001dde410 msgr2=0x55e001d31980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2c62b0700 1 --2- 172.21.15.6:0/2098950613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e001dde410 0x55e001d31980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.373+0000 7fa2c62b0700 1 -- 172.21.15.6:0/2098950613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc005de0 con 0x55e001d06bc0 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2c62b0700 1 --2- 172.21.15.6:0/2098950613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001d51c40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000c00 tx=0x7fa2bc00d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf2230505eb76da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2c4aad700 1 -- 172.21.15.6:0/2098950613 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc008670 con 0x55e001d06bc0 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2c4aad700 1 -- 172.21.15.6:0/2098950613 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc008b40 con 0x55e001d06bc0 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2c4aad700 1 -- 172.21.15.6:0/2098950613 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc01e450 con 0x55e001d06bc0 2023-11-28T13:02:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2d01ad080 1 -- 172.21.15.6:0/2098950613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 msgr2=0x55e001d51c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2d01ad080 1 --2- 172.21.15.6:0/2098950613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001d51c40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000c00 tx=0x7fa2bc00d000 comp rx=0 tx=0).stop 2023-11-28T13:02:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2d01ad080 1 -- 172.21.15.6:0/2098950613 shutdown_connections 2023-11-28T13:02:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2d01ad080 1 --2- 172.21.15.6:0/2098950613 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e001dde410 0x55e001d31980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2d01ad080 1 --2- 172.21.15.6:0/2098950613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001d51c40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2d01ad080 1 --2- 172.21.15.6:0/2098950613 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e001ddb720 0x55e001d31440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2d01ad080 1 -- 172.21.15.6:0/2098950613 >> 172.21.15.6:0/2098950613 conn(0x55e001d33300 msgr2=0x55e001d04370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2d01ad080 1 -- 172.21.15.6:0/2098950613 shutdown_connections 2023-11-28T13:02:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.374+0000 7fa2d01ad080 1 -- 172.21.15.6:0/2098950613 wait complete. 2023-11-28T13:02:14.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.375+0000 7fa2d01ad080 1 Processor -- start 2023-11-28T13:02:14.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.375+0000 7fa2d01ad080 1 -- start start 2023-11-28T13:02:14.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2d01ad080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001e6dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2d01ad080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e001ddb720 0x55e001dd1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2d01ad080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e001dde410 0x55e001dd1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2d01ad080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e001ddcd00 con 0x55e001d06bc0 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2d01ad080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e001ddd230 con 0x55e001ddb720 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2d01ad080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e001dd1790 con 0x55e001dde410 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2c62b0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001e6dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2c62b0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001e6dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60878/0 (socket says 172.21.15.6:60878) 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2c62b0700 1 -- 172.21.15.6:0/1494899565 learned_addr learned my addr 172.21.15.6:0/1494899565 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2c6ab1700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e001dde410 0x55e001dd1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.376+0000 7fa2c5aaf700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e001ddb720 0x55e001dd1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2c62b0700 1 -- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e001dde410 msgr2=0x55e001dd1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2c62b0700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e001dde410 0x55e001dd1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2c62b0700 1 -- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e001ddb720 msgr2=0x55e001dd1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2c62b0700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e001ddb720 0x55e001dd1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2c62b0700 1 -- 172.21.15.6:0/1494899565 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e001c4c4a0 con 0x55e001d06bc0 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2c6ab1700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e001dde410 0x55e001dd1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2c62b0700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001e6dc10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc01e040 tx=0x7fa2bc003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2b77fe700 1 -- 172.21.15.6:0/1494899565 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc008670 con 0x55e001d06bc0 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2b77fe700 1 -- 172.21.15.6:0/1494899565 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc01e9e0 con 0x55e001d06bc0 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2d01ad080 1 -- 172.21.15.6:0/1494899565 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e001c734f0 con 0x55e001d06bc0 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2b77fe700 1 -- 172.21.15.6:0/1494899565 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc01ed90 con 0x55e001d06bc0 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.377+0000 7fa2d01ad080 1 -- 172.21.15.6:0/1494899565 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e001c59120 con 0x55e001d06bc0 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.378+0000 7fa2b77fe700 1 -- 172.21.15.6:0/1494899565 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa2bc027410 con 0x55e001d06bc0 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.379+0000 7fa2b77fe700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa2a8066c10 0x7fa2a80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.379+0000 7fa2b77fe700 1 -- 172.21.15.6:0/1494899565 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa2bc087390 con 0x55e001d06bc0 2023-11-28T13:02:14.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.379+0000 7fa2c5aaf700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa2a8066c10 0x7fa2a80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.379+0000 7fa2d01ad080 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55e001ddd400 0x55e001d4b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.379+0000 7fa2d01ad080 1 -- 172.21.15.6:0/1494899565 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- osd_op(unknown.0.0:1 7.36 7:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55e001d31e80 con 0x55e001ddd400 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.379+0000 7fa2c5aaf700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa2a8066c10 0x7fa2a80690c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55e001d51db0 tx=0x7fa2b80058c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.379+0000 7fa2c6ab1700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55e001ddd400 0x55e001d4b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2c6ab1700 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55e001ddd400 0x55e001d4b100 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2c6ab1700 1 -- 172.21.15.6:0/1494899565 <== osd.5 v2:172.21.15.61:6808/103780 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fa2c000e7f0 con 0x55e001ddd400 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2d01ad080 1 -- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55e001ddd400 msgr2=0x55e001d4b100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2d01ad080 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55e001ddd400 0x55e001d4b100 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2d01ad080 1 -- 172.21.15.6:0/1494899565 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa2a8066c10 msgr2=0x7fa2a80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2d01ad080 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa2a8066c10 0x7fa2a80690c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55e001d51db0 tx=0x7fa2b80058c0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2d01ad080 1 -- 172.21.15.6:0/1494899565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 msgr2=0x55e001e6dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2d01ad080 1 --2- 172.21.15.6:0/1494899565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e001d06bc0 0x55e001e6dc10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc01e040 tx=0x7fa2bc003020 comp rx=0 tx=0).stop 2023-11-28T13:02:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2c6ab1700 1 -- 172.21.15.6:0/1494899565 reap_dead start 2023-11-28T13:02:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2d01ad080 1 -- 172.21.15.6:0/1494899565 shutdown_connections 2023-11-28T13:02:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2d01ad080 1 -- 172.21.15.6:0/1494899565 >> 172.21.15.6:0/1494899565 conn(0x55e001d33300 msgr2=0x55e001d31440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2d01ad080 1 -- 172.21.15.6:0/1494899565 shutdown_connections 2023-11-28T13:02:14.386 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.380+0000 7fa2d01ad080 1 -- 172.21.15.6:0/1494899565 wait complete. 2023-11-28T13:02:14.387 INFO:teuthology.orchestra.run.smithi006.stdout:mytag 2023-11-28T13:02:14.387 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-28T13:02:14.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81eb8ed080 1 Processor -- start 2023-11-28T13:02:14.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81eb8ed080 1 -- start start 2023-11-28T13:02:14.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81eb8ed080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f4005ca870 0x55f4005f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81eb8ed080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4005f6d70 0x55f4005c6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81eb8ed080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55f4006a06e0 0x55f400698910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81eb8ed080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400502fe0 con 0x55f4005f6d70 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81eb8ed080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f40069d650 con 0x55f4005ca870 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81eb8ed080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f40069d370 con 0x55f4006a06e0 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81e11ef700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4005f6d70 0x55f4005c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81e11ef700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4005f6d70 0x55f4005c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60896/0 (socket says 172.21.15.6:60896) 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.430+0000 7f81e11ef700 1 -- 172.21.15.6:0/949206286 learned_addr learned my addr 172.21.15.6:0/949206286 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81e19f0700 1 --2- 172.21.15.6:0/949206286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f4005ca870 0x55f4005f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81e21f1700 1 --2- 172.21.15.6:0/949206286 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55f4006a06e0 0x55f400698910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81e11ef700 1 -- 172.21.15.6:0/949206286 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55f4006a06e0 msgr2=0x55f400698910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81e11ef700 1 --2- 172.21.15.6:0/949206286 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55f4006a06e0 0x55f400698910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81e11ef700 1 -- 172.21.15.6:0/949206286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f4005ca870 msgr2=0x55f4005f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81e11ef700 1 --2- 172.21.15.6:0/949206286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f4005ca870 0x55f4005f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81e11ef700 1 -- 172.21.15.6:0/949206286 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d4005de0 con 0x55f4005f6d70 2023-11-28T13:02:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81e19f0700 1 --2- 172.21.15.6:0/949206286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f4005ca870 0x55f4005f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81e11ef700 1 --2- 172.21.15.6:0/949206286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4005f6d70 0x55f4005c6970 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f81d800a7a0 tx=0x7f81d800aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe322ff2290eca7a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81d3fff700 1 -- 172.21.15.6:0/949206286 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d800f070 con 0x55f4005f6d70 2023-11-28T13:02:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81d3fff700 1 -- 172.21.15.6:0/949206286 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d80084c0 con 0x55f4005f6d70 2023-11-28T13:02:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81d3fff700 1 -- 172.21.15.6:0/949206286 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8008830 con 0x55f4005f6d70 2023-11-28T13:02:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81eb8ed080 1 -- 172.21.15.6:0/949206286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4005f6d70 msgr2=0x55f4005c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81eb8ed080 1 --2- 172.21.15.6:0/949206286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4005f6d70 0x55f4005c6970 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f81d800a7a0 tx=0x7f81d800aab0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81eb8ed080 1 -- 172.21.15.6:0/949206286 shutdown_connections 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81eb8ed080 1 --2- 172.21.15.6:0/949206286 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55f4006a06e0 0x55f400698910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81eb8ed080 1 --2- 172.21.15.6:0/949206286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f4005ca870 0x55f4005f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.431+0000 7f81eb8ed080 1 --2- 172.21.15.6:0/949206286 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4005f6d70 0x55f4005c6970 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.432+0000 7f81eb8ed080 1 -- 172.21.15.6:0/949206286 >> 172.21.15.6:0/949206286 conn(0x55f4005f5300 msgr2=0x55f4005c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.432+0000 7f81eb8ed080 1 -- 172.21.15.6:0/949206286 shutdown_connections 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.432+0000 7f81eb8ed080 1 -- 172.21.15.6:0/949206286 wait complete. 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.432+0000 7f81eb8ed080 1 Processor -- start 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.432+0000 7f81eb8ed080 1 -- start start 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81eb8ed080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f4005ca870 0x55f4005f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81eb8ed080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55f4005f6d70 0x55f400692000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81eb8ed080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4006a06e0 0x55f400692540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81eb8ed080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4006951c0 con 0x55f4006a06e0 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81eb8ed080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4005c9c10 con 0x55f4005ca870 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81eb8ed080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f400694f40 con 0x55f4005f6d70 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81e21f1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4006a06e0 0x55f400692540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81e21f1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4006a06e0 0x55f400692540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60904/0 (socket says 172.21.15.6:60904) 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81e21f1700 1 -- 172.21.15.6:0/587000925 learned_addr learned my addr 172.21.15.6:0/587000925 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81e19f0700 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f4005ca870 0x55f4005f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81e11ef700 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55f4005f6d70 0x55f400692000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81e21f1700 1 -- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55f4005f6d70 msgr2=0x55f400692000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81e21f1700 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55f4005f6d70 0x55f400692000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81e21f1700 1 -- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f4005ca870 msgr2=0x55f4005f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81e21f1700 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f4005ca870 0x55f4005f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.433+0000 7f81e21f1700 1 -- 172.21.15.6:0/587000925 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d800a420 con 0x55f4006a06e0 2023-11-28T13:02:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.434+0000 7f81e21f1700 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4006a06e0 0x55f400692540 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f81dc004250 tx=0x7f81dc006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.434+0000 7f81d2ffd700 1 -- 172.21.15.6:0/587000925 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00ddc0 con 0x55f4006a06e0 2023-11-28T13:02:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.434+0000 7f81d2ffd700 1 -- 172.21.15.6:0/587000925 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc0030d0 con 0x55f4006a06e0 2023-11-28T13:02:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.434+0000 7f81eb8ed080 1 -- 172.21.15.6:0/587000925 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f40051b120 con 0x55f4006a06e0 2023-11-28T13:02:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.434+0000 7f81d2ffd700 1 -- 172.21.15.6:0/587000925 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc001540 con 0x55f4006a06e0 2023-11-28T13:02:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.434+0000 7f81eb8ed080 1 -- 172.21.15.6:0/587000925 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f40050e830 con 0x55f4006a06e0 2023-11-28T13:02:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.435+0000 7f81d2ffd700 1 -- 172.21.15.6:0/587000925 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f81dc0016a0 con 0x55f4006a06e0 2023-11-28T13:02:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.435+0000 7f81d2ffd700 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f81c4066c10 0x7f81c40690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.436+0000 7f81d2ffd700 1 -- 172.21.15.6:0/587000925 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f81dc083f60 con 0x55f4006a06e0 2023-11-28T13:02:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.436+0000 7f81e19f0700 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f81c4066c10 0x7f81c40690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.436+0000 7f81eb8ed080 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x55f40068e7b0 0x55f40068ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.436+0000 7f81eb8ed080 1 -- 172.21.15.6:0/587000925 --> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] -- osd_op(unknown.0.0:1 7.19 7:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f40068f130 con 0x55f40068e7b0 2023-11-28T13:02:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.436+0000 7f81e11ef700 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x55f40068e7b0 0x55f40068ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.436+0000 7f81e19f0700 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f81c4066c10 0x7f81c40690c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f81d4015040 tx=0x7f81d400f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.436+0000 7f81e11ef700 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x55f40068e7b0 0x55f40068ebf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81e11ef700 1 -- 172.21.15.6:0/587000925 <== osd.6 v2:172.21.15.61:6800/103779 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f81d80089a0 con 0x55f40068e7b0 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81eb8ed080 1 -- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x55f40068e7b0 msgr2=0x55f40068ebf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81eb8ed080 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x55f40068e7b0 0x55f40068ebf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81eb8ed080 1 -- 172.21.15.6:0/587000925 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f81c4066c10 msgr2=0x7f81c40690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81eb8ed080 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f81c4066c10 0x7f81c40690c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f81d4015040 tx=0x7f81d400f000 comp rx=0 tx=0).stop 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81eb8ed080 1 -- 172.21.15.6:0/587000925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4006a06e0 msgr2=0x55f400692540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81eb8ed080 1 --2- 172.21.15.6:0/587000925 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55f4006a06e0 0x55f400692540 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f81dc004250 tx=0x7f81dc006f90 comp rx=0 tx=0).stop 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81e21f1700 1 -- 172.21.15.6:0/587000925 reap_dead start 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81eb8ed080 1 -- 172.21.15.6:0/587000925 shutdown_connections 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81eb8ed080 1 -- 172.21.15.6:0/587000925 >> 172.21.15.6:0/587000925 conn(0x55f4005f5300 msgr2=0x55f400612910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81eb8ed080 1 -- 172.21.15.6:0/587000925 shutdown_connections 2023-11-28T13:02:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.437+0000 7f81eb8ed080 1 -- 172.21.15.6:0/587000925 wait complete. 2023-11-28T13:02:14.444 INFO:teuthology.orchestra.run.smithi006.stdout:mytag 2023-11-28T13:02:14.444 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-28T13:02:14.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd07ab4a080 1 Processor -- start 2023-11-28T13:02:14.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd07ab4a080 1 -- start start 2023-11-28T13:02:14.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd07ab4a080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d143846330 0x55d1438443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd07ab4a080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d1438413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd07ab4a080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d143849230 0x55d143841920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd07ab4a080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d143752fe0 con 0x55d1438481d0 2023-11-28T13:02:14.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd07ab4a080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d14397c9f0 con 0x55d143846330 2023-11-28T13:02:14.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd07ab4a080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d143843ae0 con 0x55d143849230 2023-11-28T13:02:14.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d1438413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d1438413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60916/0 (socket says 172.21.15.6:60916) 2023-11-28T13:02:14.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.488+0000 7fd06bfff700 1 -- 172.21.15.6:0/779666555 learned_addr learned my addr 172.21.15.6:0/779666555 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:14.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd07144e700 1 --2- 172.21.15.6:0/779666555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d143849230 0x55d143841920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd070c4d700 1 --2- 172.21.15.6:0/779666555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d143846330 0x55d1438443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd06bfff700 1 -- 172.21.15.6:0/779666555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d143849230 msgr2=0x55d143841920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd06bfff700 1 --2- 172.21.15.6:0/779666555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d143849230 0x55d143841920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd06bfff700 1 -- 172.21.15.6:0/779666555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d143846330 msgr2=0x55d1438443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd06bfff700 1 --2- 172.21.15.6:0/779666555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d143846330 0x55d1438443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd06bfff700 1 -- 172.21.15.6:0/779666555 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064005de0 con 0x55d1438481d0 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd070c4d700 1 --2- 172.21.15.6:0/779666555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d143846330 0x55d1438443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd06bfff700 1 --2- 172.21.15.6:0/779666555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d1438413e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd06400a4f0 tx=0x7fd06400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e74aa67e0f7d17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd06affd700 1 -- 172.21.15.6:0/779666555 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064008ce0 con 0x55d1438481d0 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd06affd700 1 -- 172.21.15.6:0/779666555 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd064008e40 con 0x55d1438481d0 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd06affd700 1 -- 172.21.15.6:0/779666555 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06401c670 con 0x55d1438481d0 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd07ab4a080 1 -- 172.21.15.6:0/779666555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 msgr2=0x55d1438413e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd07ab4a080 1 --2- 172.21.15.6:0/779666555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d1438413e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd06400a4f0 tx=0x7fd06400d000 comp rx=0 tx=0).stop 2023-11-28T13:02:14.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd07ab4a080 1 -- 172.21.15.6:0/779666555 shutdown_connections 2023-11-28T13:02:14.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd07ab4a080 1 --2- 172.21.15.6:0/779666555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d143849230 0x55d143841920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd07ab4a080 1 --2- 172.21.15.6:0/779666555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d143846330 0x55d1438443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd07ab4a080 1 --2- 172.21.15.6:0/779666555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d1438413e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd07ab4a080 1 -- 172.21.15.6:0/779666555 >> 172.21.15.6:0/779666555 conn(0x55d143861130 msgr2=0x55d143831bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:14.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd07ab4a080 1 -- 172.21.15.6:0/779666555 shutdown_connections 2023-11-28T13:02:14.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.489+0000 7fd07ab4a080 1 -- 172.21.15.6:0/779666555 wait complete. 2023-11-28T13:02:14.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.490+0000 7fd07ab4a080 1 Processor -- start 2023-11-28T13:02:14.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.490+0000 7fd07ab4a080 1 -- start start 2023-11-28T13:02:14.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd07ab4a080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d143846330 0x55d1438ddb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd07ab4a080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d14397c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd07ab4a080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d143849230 0x55d1438ee850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd07ab4a080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1438ee240 con 0x55d1438481d0 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd07ab4a080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1438f1c10 con 0x55d143849230 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd07ab4a080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1438efa40 con 0x55d143846330 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d14397c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd06bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d14397c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60922/0 (socket says 172.21.15.6:60922) 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd06bfff700 1 -- 172.21.15.6:0/324853657 learned_addr learned my addr 172.21.15.6:0/324853657 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd070c4d700 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d143846330 0x55d1438ddb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd07144e700 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d143849230 0x55d1438ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd06bfff700 1 -- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d143846330 msgr2=0x55d1438ddb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd06bfff700 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d143846330 0x55d1438ddb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd06bfff700 1 -- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d143849230 msgr2=0x55d1438ee850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd06bfff700 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d143849230 0x55d1438ee850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd06bfff700 1 -- 172.21.15.6:0/324853657 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1437854f0 con 0x55d1438481d0 2023-11-28T13:02:14.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd06bfff700 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d14397c110 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x55d143804f00 tx=0x7fd064003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.491+0000 7fd069ffb700 1 -- 172.21.15.6:0/324853657 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06401ca60 con 0x55d1438481d0 2023-11-28T13:02:14.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.492+0000 7fd069ffb700 1 -- 172.21.15.6:0/324853657 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06402a070 con 0x55d1438481d0 2023-11-28T13:02:14.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.492+0000 7fd07ab4a080 1 -- 172.21.15.6:0/324853657 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d14376b120 con 0x55d1438481d0 2023-11-28T13:02:14.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.492+0000 7fd069ffb700 1 -- 172.21.15.6:0/324853657 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06401ce20 con 0x55d1438481d0 2023-11-28T13:02:14.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.492+0000 7fd07ab4a080 1 -- 172.21.15.6:0/324853657 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d14375e830 con 0x55d1438481d0 2023-11-28T13:02:14.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:02:14.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.493+0000 7fd069ffb700 1 -- 172.21.15.6:0/324853657 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd064036070 con 0x55d1438481d0 2023-11-28T13:02:14.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.493+0000 7fd069ffb700 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd054066c10 0x7fd0540690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.493+0000 7fd069ffb700 1 -- 172.21.15.6:0/324853657 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fd0640873f0 con 0x55d1438481d0 2023-11-28T13:02:14.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.494+0000 7fd070c4d700 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd054066c10 0x7fd0540690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.494+0000 7fd07ab4a080 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x55d1438f2590 0x55d1438f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:14.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.494+0000 7fd07ab4a080 1 -- 172.21.15.6:0/324853657 --> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] -- osd_op(unknown.0.0:1 7.12 7:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55d1438f2ed0 con 0x55d1438f2590 2023-11-28T13:02:14.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.494+0000 7fd070c4d700 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd054066c10 0x7fd0540690c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c6d0 tx=0x7fd05c005950 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.494+0000 7fd07144e700 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x55d1438f2590 0x55d1438f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.494+0000 7fd07144e700 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x55d1438f2590 0x55d1438f2990 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07144e700 1 -- 172.21.15.6:0/324853657 <== osd.6 v2:172.21.15.61:6800/103779 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd06c00e900 con 0x55d1438f2590 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07ab4a080 1 -- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x55d1438f2590 msgr2=0x55d1438f2990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07ab4a080 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x55d1438f2590 0x55d1438f2990 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07ab4a080 1 -- 172.21.15.6:0/324853657 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd054066c10 msgr2=0x7fd0540690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07ab4a080 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd054066c10 0x7fd0540690c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c6d0 tx=0x7fd05c005950 comp rx=0 tx=0).stop 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07ab4a080 1 -- 172.21.15.6:0/324853657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 msgr2=0x55d14397c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07ab4a080 1 --2- 172.21.15.6:0/324853657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d1438481d0 0x55d14397c110 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x55d143804f00 tx=0x7fd064003020 comp rx=0 tx=0).stop 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07144e700 1 -- 172.21.15.6:0/324853657 reap_dead start 2023-11-28T13:02:14.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07ab4a080 1 -- 172.21.15.6:0/324853657 shutdown_connections 2023-11-28T13:02:14.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07ab4a080 1 -- 172.21.15.6:0/324853657 >> 172.21.15.6:0/324853657 conn(0x55d143861130 msgr2=0x55d14381c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:14.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07ab4a080 1 -- 172.21.15.6:0/324853657 shutdown_connections 2023-11-28T13:02:14.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:14.495+0000 7fd07ab4a080 1 -- 172.21.15.6:0/324853657 wait complete. 2023-11-28T13:02:14.503 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:02:14.503 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:02:14.549 DEBUG:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:02:14.600 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:02:14.601 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:02:14.601 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:14.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3446 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:02:14.619 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:02:14.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:14.619 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:02:14.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:02:14.676 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-28T13:02:14.677 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:02:14.677 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:14.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:02:14.697 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:02:14.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:14.698 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T13:02:14.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:02:14.758 INFO:teuthology.orchestra.run.smithi061.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T13:02:14.759 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:02:14.759 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:14.776 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3446 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:02:14.777 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:02:14.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:14.777 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T13:02:14.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:02:14.834 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T13:02:14.834 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:02:14.834 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:14.855 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:02:14.855 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:02:14.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:14.855 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T13:02:14.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:02:14.915 INFO:teuthology.orchestra.run.smithi061.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T13:02:14.916 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:15.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.040+0000 7f787f8d2700 1 Processor -- start 2023-11-28T13:02:15.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.040+0000 7f787f8d2700 1 -- start start 2023-11-28T13:02:15.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787f8d2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f787815c890 0x7f78780b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787f8d2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f78780ceec0 0x7f787815d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787f8d2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787815d2d0 con 0x7f7878160d30 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787f8d2700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78780b2ba0 con 0x7f78780ceec0 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787f8d2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787815dca0 con 0x7f787815c890 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787de6f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f78780ceec0 0x7f787815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787de6f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f78780ceec0 0x7f787815d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60934/0 (socket says 172.21.15.6:60934) 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787de6f700 1 -- 172.21.15.6:0/268947719 learned_addr learned my addr 172.21.15.6:0/268947719 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787de6f700 1 -- 172.21.15.6:0/268947719 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f787815c890 msgr2=0x7f78780b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787d66e700 1 --2- 172.21.15.6:0/268947719 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f787815c890 0x7f78780b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787de6f700 1 --2- 172.21.15.6:0/268947719 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f787815c890 0x7f78780b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787de6f700 1 -- 172.21.15.6:0/268947719 >> v1:172.21.15.61:6789/0 conn(0x7f7878160d30 legacy=0x7f78780ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787de6f700 1 -- 172.21.15.6:0/268947719 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78781611a0 con 0x7f78780ceec0 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787d66e700 1 --2- 172.21.15.6:0/268947719 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f787815c890 0x7f78780b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f787de6f700 1 --2- 172.21.15.6:0/268947719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 0x7f787815d760 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7870009e50 tx=0x7f787000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224b1ea65a66f7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.041+0000 7f786ffff700 1 -- 172.21.15.6:0/268947719 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000e070 con 0x7f78780ceec0 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.042+0000 7f786ffff700 1 -- 172.21.15.6:0/268947719 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870003a20 con 0x7f78780ceec0 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.042+0000 7f786ffff700 1 -- 172.21.15.6:0/268947719 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870003dd0 con 0x7f78780ceec0 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.042+0000 7f787f8d2700 1 -- 172.21.15.6:0/268947719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 msgr2=0x7f787815d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.042+0000 7f787f8d2700 1 --2- 172.21.15.6:0/268947719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 0x7f787815d760 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7870009e50 tx=0x7f787000b330 comp rx=0 tx=0).stop 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.042+0000 7f787f8d2700 1 -- 172.21.15.6:0/268947719 shutdown_connections 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.042+0000 7f787f8d2700 1 --2- 172.21.15.6:0/268947719 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 0x7f787815d760 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.042+0000 7f787f8d2700 1 --2- 172.21.15.6:0/268947719 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f787815c890 0x7f78780b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.042+0000 7f787f8d2700 1 -- 172.21.15.6:0/268947719 >> 172.21.15.6:0/268947719 conn(0x7f787806e9b0 msgr2=0x7f78780cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:15.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.042+0000 7f787f8d2700 1 -- 172.21.15.6:0/268947719 shutdown_connections 2023-11-28T13:02:15.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.042+0000 7f787f8d2700 1 -- 172.21.15.6:0/268947719 wait complete. 2023-11-28T13:02:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.044+0000 7f787f8d2700 1 Processor -- start 2023-11-28T13:02:15.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.044+0000 7f787f8d2700 1 -- start start 2023-11-28T13:02:15.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787f8d2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 0x7f787815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787f8d2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f787815c890 0x7f78780b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787f8d2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7878160d30 0x7f78780b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787f8d2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780b4380 con 0x7f78780ceec0 2023-11-28T13:02:15.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787f8d2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780b4660 con 0x7f787815c890 2023-11-28T13:02:15.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787f8d2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78780b4940 con 0x7f7878160d30 2023-11-28T13:02:15.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787d66e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 0x7f787815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787d66e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 0x7f787815ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60936/0 (socket says 172.21.15.6:60936) 2023-11-28T13:02:15.051 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787d66e700 1 -- 172.21.15.6:0/719369073 learned_addr learned my addr 172.21.15.6:0/719369073 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:15.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787ce6d700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f787815c890 0x7f78780b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787de6f700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7878160d30 0x7f78780b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787d66e700 1 -- 172.21.15.6:0/719369073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7878160d30 msgr2=0x7f78780b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787d66e700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7878160d30 0x7f78780b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787d66e700 1 -- 172.21.15.6:0/719369073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f787815c890 msgr2=0x7f78780b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787d66e700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f787815c890 0x7f78780b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787d66e700 1 -- 172.21.15.6:0/719369073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7870009b30 con 0x7f78780ceec0 2023-11-28T13:02:15.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787de6f700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7878160d30 0x7f78780b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:15.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787ce6d700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f787815c890 0x7f78780b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:15.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f787d66e700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 0x7f787815ba20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f787400c750 tx=0x7f787400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:15.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f786dffb700 1 -- 172.21.15.6:0/719369073 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874011800 con 0x7f78780ceec0 2023-11-28T13:02:15.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.045+0000 7f786dffb700 1 -- 172.21.15.6:0/719369073 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7874011960 con 0x7f78780ceec0 2023-11-28T13:02:15.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.046+0000 7f787f8d2700 1 -- 172.21.15.6:0/719369073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78780b3070 con 0x7f78780ceec0 2023-11-28T13:02:15.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.046+0000 7f786dffb700 1 -- 172.21.15.6:0/719369073 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874011cd0 con 0x7f78780ceec0 2023-11-28T13:02:15.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.046+0000 7f787f8d2700 1 -- 172.21.15.6:0/719369073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78780b2d90 con 0x7f78780ceec0 2023-11-28T13:02:15.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.047+0000 7f787f8d2700 1 -- 172.21.15.6:0/719369073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787815cf60 con 0x7f78780ceec0 2023-11-28T13:02:15.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.047+0000 7f786dffb700 1 -- 172.21.15.6:0/719369073 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7874020070 con 0x7f78780ceec0 2023-11-28T13:02:15.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.047+0000 7f786dffb700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f785c0682f0 0x7f785c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.047+0000 7f786dffb700 1 -- 172.21.15.6:0/719369073 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7874085c90 con 0x7f78780ceec0 2023-11-28T13:02:15.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.048+0000 7f787ce6d700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f785c0682f0 0x7f785c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.048+0000 7f787ce6d700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f785c0682f0 0x7f785c06a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78781565a0 tx=0x7f7868008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:15.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.052+0000 7f786dffb700 1 -- 172.21.15.6:0/719369073 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7874052370 con 0x7f78780ceec0 2023-11-28T13:02:15.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.291+0000 7f787f8d2700 1 -- 172.21.15.6:0/719369073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7878068570 con 0x7f78780ceec0 2023-11-28T13:02:15.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.291+0000 7f786dffb700 1 -- 172.21.15.6:0/719369073 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f7874055990 con 0x7f78780ceec0 2023-11-28T13:02:15.293 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:15.293 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4026920958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4026920958},{"type":"v1","addr":"172.21.15.61:6833","nonce":4026920958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2212733196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2212733196},{"type":"v1","addr":"172.21.15.61:6835","nonce":2212733196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1674709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1674709721},{"type":"v1","addr":"172.21.15.6:6836","nonce":1674709721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:42.595217+0000","modified":"2023-11-28T13:01:45.602537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/2789212407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":2789212407},{"type":"v1","addr":"172.21.15.6:6837","nonce":2789212407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:02:15.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 -- 172.21.15.6:0/719369073 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f785c0682f0 msgr2=0x7f785c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f785c0682f0 0x7f785c06a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78781565a0 tx=0x7f7868008040 comp rx=0 tx=0).stop 2023-11-28T13:02:15.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 -- 172.21.15.6:0/719369073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 msgr2=0x7f787815ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 0x7f787815ba20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f787400c750 tx=0x7f787400cb10 comp rx=0 tx=0).stop 2023-11-28T13:02:15.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 -- 172.21.15.6:0/719369073 shutdown_connections 2023-11-28T13:02:15.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f785c0682f0 0x7f785c06a7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7878160d30 0x7f78780b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78780ceec0 0x7f787815ba20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 --2- 172.21.15.6:0/719369073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f787815c890 0x7f78780b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 -- 172.21.15.6:0/719369073 >> 172.21.15.6:0/719369073 conn(0x7f787806e9b0 msgr2=0x7f787806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:15.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 -- 172.21.15.6:0/719369073 shutdown_connections 2023-11-28T13:02:15.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.294+0000 7f787f8d2700 1 -- 172.21.15.6:0/719369073 wait complete. 2023-11-28T13:02:15.296 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2023-11-28T13:02:15.307 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:15.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.430+0000 7f95402f7700 1 Processor -- start 2023-11-28T13:02:15.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.430+0000 7f95402f7700 1 -- start start 2023-11-28T13:02:15.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.431+0000 7f95402f7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f953806e400 0x7f953815db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.431+0000 7f95402f7700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953815e0b0 con 0x7f95380ce890 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.431+0000 7f95402f7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953815d120 con 0x7f9538159590 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.431+0000 7f95402f7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f953815d6e0 con 0x7f953806e400 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.431+0000 7f953e894700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f95380ce890 0x7f95380caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48268/0 (socket says 172.21.15.6:48268) 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.431+0000 7f953e894700 1 -- 172.21.15.6:0/4050062432 learned_addr learned my addr 172.21.15.6:0/4050062432 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.431+0000 7f953d892700 1 --2- 172.21.15.6:0/4050062432 >> v2:172.21.15.61:3300/0 conn(0x7f953806e400 0x7f953815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.432+0000 7f953d091700 1 -- 172.21.15.6:0/4050062432 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519686224 0 0) 0x7f9530001490 con 0x7f95380ce890 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.432+0000 7f953d091700 1 -- 172.21.15.6:0/4050062432 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95240035b0 con 0x7f95380ce890 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.432+0000 7f953d892700 1 -- 172.21.15.6:0/4050062432 >> v1:172.21.15.61:6789/0 conn(0x7f9538159590 legacy=0x7f95380ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.432+0000 7f953d091700 1 -- 172.21.15.6:0/4050062432 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517926305 0 0) 0x7f953815d120 con 0x7f9538159590 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.432+0000 7f953d892700 1 -- 172.21.15.6:0/4050062432 >> v1:172.21.15.6:6789/0 conn(0x7f95380ce890 legacy=0x7f95380caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.432+0000 7f953d892700 1 -- 172.21.15.6:0/4050062432 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95380aef60 con 0x7f953806e400 2023-11-28T13:02:15.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.432+0000 7f953d892700 1 --2- 172.21.15.6:0/4050062432 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f953806e400 0x7f953815db70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9534009840 tx=0x7f9534009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=544eae3921e6e48d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:15.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.432+0000 7f953d091700 1 -- 172.21.15.6:0/4050062432 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534011830 con 0x7f953806e400 2023-11-28T13:02:15.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.432+0000 7f953d091700 1 -- 172.21.15.6:0/4050062432 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9534011990 con 0x7f953806e400 2023-11-28T13:02:15.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.432+0000 7f953d091700 1 -- 172.21.15.6:0/4050062432 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534011d20 con 0x7f953806e400 2023-11-28T13:02:15.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.433+0000 7f95402f7700 1 -- 172.21.15.6:0/4050062432 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f953806e400 msgr2=0x7f953815db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.433+0000 7f95402f7700 1 --2- 172.21.15.6:0/4050062432 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f953806e400 0x7f953815db70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9534009840 tx=0x7f9534009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.433+0000 7f95402f7700 1 -- 172.21.15.6:0/4050062432 shutdown_connections 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.433+0000 7f95402f7700 1 --2- 172.21.15.6:0/4050062432 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f953806e400 0x7f953815db70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.433+0000 7f95402f7700 1 -- 172.21.15.6:0/4050062432 >> 172.21.15.6:0/4050062432 conn(0x7f95380b1650 msgr2=0x7f95380b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.433+0000 7f95402f7700 1 -- 172.21.15.6:0/4050062432 shutdown_connections 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.433+0000 7f95402f7700 1 -- 172.21.15.6:0/4050062432 wait complete. 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.434+0000 7f95402f7700 1 Processor -- start 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.434+0000 7f95402f7700 1 -- start start 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f95402f7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f953806e400 0x7f95380c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f95402f7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95380ce890 0x7f95380b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f95402f7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9538159590 0x7f95380b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f95402f7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95380c3220 con 0x7f9538159590 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f95402f7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95380b5ac0 con 0x7f95380ce890 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f95402f7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9538153770 con 0x7f953806e400 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f953e894700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9538159590 0x7f95380b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f953e894700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9538159590 0x7f95380b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60954/0 (socket says 172.21.15.6:60954) 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f953e894700 1 -- 172.21.15.6:0/3857991472 learned_addr learned my addr 172.21.15.6:0/3857991472 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f953e093700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f953806e400 0x7f95380c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f953d892700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95380ce890 0x7f95380b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f953e894700 1 -- 172.21.15.6:0/3857991472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f953806e400 msgr2=0x7f95380c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f953e894700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f953806e400 0x7f95380c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f953e894700 1 -- 172.21.15.6:0/3857991472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95380ce890 msgr2=0x7f95380b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f953e894700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95380ce890 0x7f95380b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.435+0000 7f953e894700 1 -- 172.21.15.6:0/3857991472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95340094a0 con 0x7f9538159590 2023-11-28T13:02:15.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.436+0000 7f953e894700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9538159590 0x7f95380b4130 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f953000ee10 tx=0x7f953000c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:15.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.436+0000 7f952effd700 1 -- 172.21.15.6:0/3857991472 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530011470 con 0x7f9538159590 2023-11-28T13:02:15.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.436+0000 7f952effd700 1 -- 172.21.15.6:0/3857991472 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95300115d0 con 0x7f9538159590 2023-11-28T13:02:15.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.436+0000 7f95402f7700 1 -- 172.21.15.6:0/3857991472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953815d370 con 0x7f9538159590 2023-11-28T13:02:15.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.436+0000 7f952effd700 1 -- 172.21.15.6:0/3857991472 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530011960 con 0x7f9538159590 2023-11-28T13:02:15.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.436+0000 7f95402f7700 1 -- 172.21.15.6:0/3857991472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95380af240 con 0x7f9538159590 2023-11-28T13:02:15.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.437+0000 7f95402f7700 1 -- 172.21.15.6:0/3857991472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9538159960 con 0x7f9538159590 2023-11-28T13:02:15.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.437+0000 7f952effd700 1 -- 172.21.15.6:0/3857991472 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9530021070 con 0x7f9538159590 2023-11-28T13:02:15.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.438+0000 7f952effd700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f951c0682f0 0x7f951c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.438+0000 7f952effd700 1 -- 172.21.15.6:0/3857991472 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f95300886d0 con 0x7f9538159590 2023-11-28T13:02:15.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.438+0000 7f953e093700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f951c0682f0 0x7f951c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.438+0000 7f953e093700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f951c0682f0 0x7f951c06a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9528000f60 tx=0x7f9528009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:15.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.442+0000 7f952effd700 1 -- 172.21.15.6:0/3857991472 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9530054db0 con 0x7f9538159590 2023-11-28T13:02:15.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.636+0000 7f95402f7700 1 -- 172.21.15.6:0/3857991472 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95380c3360 con 0x7f9538159590 2023-11-28T13:02:15.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.637+0000 7f952effd700 1 -- 172.21.15.6:0/3857991472 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18066 (secure 0 0 0) 0x7f95380c3360 con 0x7f9538159590 2023-11-28T13:02:15.639 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:15.639 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":42,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:11.026507+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:41.144348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:41.554568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.18","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:15.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 -- 172.21.15.6:0/3857991472 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f951c0682f0 msgr2=0x7f951c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f951c0682f0 0x7f951c06a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9528000f60 tx=0x7f9528009040 comp rx=0 tx=0).stop 2023-11-28T13:02:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 -- 172.21.15.6:0/3857991472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9538159590 msgr2=0x7f95380b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9538159590 0x7f95380b4130 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f953000ee10 tx=0x7f953000c550 comp rx=0 tx=0).stop 2023-11-28T13:02:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 -- 172.21.15.6:0/3857991472 shutdown_connections 2023-11-28T13:02:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9538159590 0x7f95380b4130 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f951c0682f0 0x7f951c06a7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f953806e400 0x7f95380c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 --2- 172.21.15.6:0/3857991472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95380ce890 0x7f95380b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 -- 172.21.15.6:0/3857991472 >> 172.21.15.6:0/3857991472 conn(0x7f95380b1650 msgr2=0x7f95380cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:15.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 -- 172.21.15.6:0/3857991472 shutdown_connections 2023-11-28T13:02:15.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.639+0000 7f95402f7700 1 -- 172.21.15.6:0/3857991472 wait complete. 2023-11-28T13:02:15.653 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:01:41.144348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:01:41.554568+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:02:15.653 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:02:15.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.779+0000 7f37085f0700 1 Processor -- start 2023-11-28T13:02:15.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.779+0000 7f37085f0700 1 -- start start 2023-11-28T13:02:15.791 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f37085f0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f37000cd140 0x7f370006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f37085f0700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37000ae730 con 0x7f370006d140 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f37085f0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37000c3e10 con 0x7f370015a640 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f37085f0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37000cb210 con 0x7f37000cd140 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f3706b8d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f370006d140 0x7f37000cb630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48282/0 (socket says 172.21.15.6:48282) 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f3706b8d700 1 -- 172.21.15.6:0/484633492 learned_addr learned my addr 172.21.15.6:0/484633492 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f3705b8b700 1 --2- 172.21.15.6:0/484633492 >> v2:172.21.15.61:3300/0 conn(0x7f37000cd140 0x7f370006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f370538a700 1 -- 172.21.15.6:0/484633492 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133416983 0 0) 0x7f37000ae730 con 0x7f370006d140 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f370538a700 1 -- 172.21.15.6:0/484633492 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36ec0035b0 con 0x7f370006d140 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f3705b8b700 1 -- 172.21.15.6:0/484633492 >> v1:172.21.15.61:6789/0 conn(0x7f370015a640 legacy=0x7f37000c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f3705b8b700 1 -- 172.21.15.6:0/484633492 >> v1:172.21.15.6:6789/0 conn(0x7f370006d140 legacy=0x7f37000cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f3705b8b700 1 -- 172.21.15.6:0/484633492 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370006e9e0 con 0x7f37000cd140 2023-11-28T13:02:15.792 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.780+0000 7f3705b8b700 1 --2- 172.21.15.6:0/484633492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37000cd140 0x7f370006b1f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009840 tx=0x7f36f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea7d2bce559cd635 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.781+0000 7f370538a700 1 -- 172.21.15.6:0/484633492 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0011830 con 0x7f37000cd140 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.781+0000 7f370538a700 1 -- 172.21.15.6:0/484633492 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f0011990 con 0x7f37000cd140 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.781+0000 7f370538a700 1 -- 172.21.15.6:0/484633492 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0011d20 con 0x7f37000cd140 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.781+0000 7f37085f0700 1 -- 172.21.15.6:0/484633492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37000cd140 msgr2=0x7f370006b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.781+0000 7f37085f0700 1 --2- 172.21.15.6:0/484633492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37000cd140 0x7f370006b1f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009840 tx=0x7f36f0009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.781+0000 7f37085f0700 1 -- 172.21.15.6:0/484633492 shutdown_connections 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.781+0000 7f37085f0700 1 --2- 172.21.15.6:0/484633492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37000cd140 0x7f370006b1f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.781+0000 7f37085f0700 1 -- 172.21.15.6:0/484633492 >> 172.21.15.6:0/484633492 conn(0x7f37000cc630 msgr2=0x7f37000cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.781+0000 7f37085f0700 1 -- 172.21.15.6:0/484633492 shutdown_connections 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.781+0000 7f37085f0700 1 -- 172.21.15.6:0/484633492 wait complete. 2023-11-28T13:02:15.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.782+0000 7f37085f0700 1 Processor -- start 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.782+0000 7f37085f0700 1 -- start start 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f37085f0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f370006d140 0x7f370014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f37085f0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37000cd140 0x7f37000b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f37085f0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f370015a640 0x7f370015c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f37085f0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370014d620 con 0x7f37000cd140 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f37085f0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370014cb80 con 0x7f370006d140 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f37085f0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37000b0bc0 con 0x7f370015a640 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f3705b8b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37000cd140 0x7f37000b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f3705b8b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37000cd140 0x7f37000b54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60966/0 (socket says 172.21.15.6:60966) 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f3705b8b700 1 -- 172.21.15.6:0/2916182759 learned_addr learned my addr 172.21.15.6:0/2916182759 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f370638c700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f370006d140 0x7f370014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f3706b8d700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f370015a640 0x7f370015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f3705b8b700 1 -- 172.21.15.6:0/2916182759 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f370015a640 msgr2=0x7f370015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f3705b8b700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f370015a640 0x7f370015c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f3705b8b700 1 -- 172.21.15.6:0/2916182759 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f370006d140 msgr2=0x7f370014cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.783+0000 7f3705b8b700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f370006d140 0x7f370014cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.784+0000 7f3705b8b700 1 -- 172.21.15.6:0/2916182759 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f00094a0 con 0x7f37000cd140 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.784+0000 7f370638c700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f370006d140 0x7f370014cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.784+0000 7f3705b8b700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37000cd140 0x7f37000b54b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009fd0 tx=0x7f36f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.784+0000 7f36f6ffd700 1 -- 172.21.15.6:0/2916182759 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f00038a0 con 0x7f37000cd140 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.784+0000 7f37085f0700 1 -- 172.21.15.6:0/2916182759 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37000aec60 con 0x7f37000cd140 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.784+0000 7f36f6ffd700 1 -- 172.21.15.6:0/2916182759 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f001f070 con 0x7f37000cd140 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.784+0000 7f37085f0700 1 -- 172.21.15.6:0/2916182759 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37000ae980 con 0x7f37000cd140 2023-11-28T13:02:15.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.784+0000 7f36f6ffd700 1 -- 172.21.15.6:0/2916182759 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0003c00 con 0x7f37000cd140 2023-11-28T13:02:15.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.785+0000 7f37085f0700 1 -- 172.21.15.6:0/2916182759 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e80052f0 con 0x7f37000cd140 2023-11-28T13:02:15.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.785+0000 7f36f6ffd700 1 -- 172.21.15.6:0/2916182759 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f36f000f050 con 0x7f37000cd140 2023-11-28T13:02:15.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.786+0000 7f36f6ffd700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36e40682f0 0x7f36e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:15.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.786+0000 7f36f6ffd700 1 -- 172.21.15.6:0/2916182759 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f36f0021070 con 0x7f37000cd140 2023-11-28T13:02:15.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.786+0000 7f370638c700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36e40682f0 0x7f36e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:15.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.786+0000 7f370638c700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36e40682f0 0x7f36e406a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3700158bb0 tx=0x7f36fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:15.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.790+0000 7f36f6ffd700 1 -- 172.21.15.6:0/2916182759 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f36f0053dd0 con 0x7f37000cd140 2023-11-28T13:02:15.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.981+0000 7f37085f0700 1 -- 172.21.15.6:0/2916182759 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f36e8005b20 con 0x7f37000cd140 2023-11-28T13:02:15.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.981+0000 7f36f6ffd700 1 -- 172.21.15.6:0/2916182759 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7f36f0015020 con 0x7f37000cd140 2023-11-28T13:02:15.983 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:02:15.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.983+0000 7f37085f0700 1 -- 172.21.15.6:0/2916182759 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36e40682f0 msgr2=0x7f36e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.983+0000 7f37085f0700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36e40682f0 0x7f36e406a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3700158bb0 tx=0x7f36fc009040 comp rx=0 tx=0).stop 2023-11-28T13:02:15.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.983+0000 7f37085f0700 1 -- 172.21.15.6:0/2916182759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37000cd140 msgr2=0x7f37000b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:15.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.983+0000 7f37085f0700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37000cd140 0x7f37000b54b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009fd0 tx=0x7f36f0004750 comp rx=0 tx=0).stop 2023-11-28T13:02:15.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.983+0000 7f37085f0700 1 -- 172.21.15.6:0/2916182759 shutdown_connections 2023-11-28T13:02:15.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.983+0000 7f37085f0700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f36e40682f0 0x7f36e406a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.983+0000 7f37085f0700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f370015a640 0x7f370015c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.983+0000 7f37085f0700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f370006d140 0x7f370014cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.983+0000 7f37085f0700 1 --2- 172.21.15.6:0/2916182759 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f37000cd140 0x7f37000b54b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:15.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.983+0000 7f37085f0700 1 -- 172.21.15.6:0/2916182759 >> 172.21.15.6:0/2916182759 conn(0x7f37000cc630 msgr2=0x7f37001593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:15.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.984+0000 7f37085f0700 1 -- 172.21.15.6:0/2916182759 shutdown_connections 2023-11-28T13:02:15.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:15.984+0000 7f37085f0700 1 -- 172.21.15.6:0/2916182759 wait complete. 2023-11-28T13:02:15.997 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:02:16.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.124+0000 7f1acf198700 1 Processor -- start 2023-11-28T13:02:16.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.124+0000 7f1acf198700 1 -- start start 2023-11-28T13:02:16.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.124+0000 7f1acf198700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1ac815c4b0 0x7f1ac815d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.124+0000 7f1acf198700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1ac815c880 0x7f1ac806a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.124+0000 7f1acf198700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac80cae80 con 0x7f1ac80c9950 2023-11-28T13:02:16.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.124+0000 7f1acf198700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac815dba0 con 0x7f1ac815c4b0 2023-11-28T13:02:16.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.124+0000 7f1acf198700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac80c9de0 con 0x7f1ac815c880 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1accf34700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1ac815c4b0 0x7f1ac815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1accf34700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1ac815c4b0 0x7f1ac815d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60972/0 (socket says 172.21.15.6:60972) 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1accf34700 1 -- 172.21.15.6:0/380644655 learned_addr learned my addr 172.21.15.6:0/380644655 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1ac7fff700 1 --2- 172.21.15.6:0/380644655 >> v2:172.21.15.61:3300/0 conn(0x7f1ac815c880 0x7f1ac806a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1accf34700 1 -- 172.21.15.6:0/380644655 >> v2:172.21.15.61:3300/0 conn(0x7f1ac815c880 msgr2=0x7f1ac806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1accf34700 1 --2- 172.21.15.6:0/380644655 >> v2:172.21.15.61:3300/0 conn(0x7f1ac815c880 0x7f1ac806a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1accf34700 1 -- 172.21.15.6:0/380644655 >> v1:172.21.15.61:6789/0 conn(0x7f1ac80c9950 legacy=0x7f1ac81601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1accf34700 1 -- 172.21.15.6:0/380644655 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac80ca0c0 con 0x7f1ac815c4b0 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1accf34700 1 --2- 172.21.15.6:0/380644655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac815c4b0 0x7f1ac815d660 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8070d00 tx=0x7f1abc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847af2fe35e48adb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1ac77fe700 1 -- 172.21.15.6:0/380644655 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc011830 con 0x7f1ac815c4b0 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1ac77fe700 1 -- 172.21.15.6:0/380644655 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc011990 con 0x7f1ac815c4b0 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.125+0000 7f1ac77fe700 1 -- 172.21.15.6:0/380644655 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc011d20 con 0x7f1ac815c4b0 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.126+0000 7f1acf198700 1 -- 172.21.15.6:0/380644655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac815c4b0 msgr2=0x7f1ac815d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.126+0000 7f1acf198700 1 --2- 172.21.15.6:0/380644655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac815c4b0 0x7f1ac815d660 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8070d00 tx=0x7f1abc009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.126+0000 7f1acf198700 1 -- 172.21.15.6:0/380644655 shutdown_connections 2023-11-28T13:02:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.126+0000 7f1acf198700 1 --2- 172.21.15.6:0/380644655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac815c4b0 0x7f1ac815d660 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.126+0000 7f1acf198700 1 --2- 172.21.15.6:0/380644655 >> v2:172.21.15.61:3300/0 conn(0x7f1ac815c880 0x7f1ac806a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.126+0000 7f1acf198700 1 -- 172.21.15.6:0/380644655 >> 172.21.15.6:0/380644655 conn(0x7f1ac80b0df0 msgr2=0x7f1ac80b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.126+0000 7f1acf198700 1 -- 172.21.15.6:0/380644655 shutdown_connections 2023-11-28T13:02:16.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.126+0000 7f1acf198700 1 -- 172.21.15.6:0/380644655 wait complete. 2023-11-28T13:02:16.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.128+0000 7f1acf198700 1 Processor -- start 2023-11-28T13:02:16.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.128+0000 7f1acf198700 1 -- start start 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1acf198700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac80c9950 0x7f1ac80cded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1acf198700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ac815c4b0 0x7f1ac80c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1acf198700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac815c880 0x7f1ac80c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1acf198700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80ad1a0 con 0x7f1ac80c9950 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1acf198700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80b2130 con 0x7f1ac815c880 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1acf198700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80ac520 con 0x7f1ac815c4b0 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1accf34700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac80c9950 0x7f1ac80cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1accf34700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac80c9950 0x7f1ac80cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60978/0 (socket says 172.21.15.6:60978) 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1accf34700 1 -- 172.21.15.6:0/3590338445 learned_addr learned my addr 172.21.15.6:0/3590338445 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1ac7fff700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ac815c4b0 0x7f1ac80c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1acd735700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac815c880 0x7f1ac80c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1accf34700 1 -- 172.21.15.6:0/3590338445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ac815c4b0 msgr2=0x7f1ac80c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1accf34700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ac815c4b0 0x7f1ac80c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1accf34700 1 -- 172.21.15.6:0/3590338445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac815c880 msgr2=0x7f1ac80c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1accf34700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac815c880 0x7f1ac80c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1accf34700 1 -- 172.21.15.6:0/3590338445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc009550 con 0x7f1ac80c9950 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1ac7fff700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ac815c4b0 0x7f1ac80c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1acd735700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac815c880 0x7f1ac80c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1accf34700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac80c9950 0x7f1ac80cded0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1abc000c00 tx=0x7f1abc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1ac57fa700 1 -- 172.21.15.6:0/3590338445 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc004570 con 0x7f1ac80c9950 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1ac57fa700 1 -- 172.21.15.6:0/3590338445 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc01f070 con 0x7f1ac80c9950 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1acf198700 1 -- 172.21.15.6:0/3590338445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac80ca680 con 0x7f1ac80c9950 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.129+0000 7f1ac57fa700 1 -- 172.21.15.6:0/3590338445 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0038c0 con 0x7f1ac80c9950 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.130+0000 7f1acf198700 1 -- 172.21.15.6:0/3590338445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac80ca3a0 con 0x7f1ac80c9950 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.130+0000 7f1acf198700 1 -- 172.21.15.6:0/3590338445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac80ca960 con 0x7f1ac80c9950 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.131+0000 7f1ac57fa700 1 -- 172.21.15.6:0/3590338445 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1abc02a020 con 0x7f1ac80c9950 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.131+0000 7f1ac57fa700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1aac0682f0 0x7f1aac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.131+0000 7f1ac57fa700 1 -- 172.21.15.6:0/3590338445 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f1abc087530 con 0x7f1ac80c9950 2023-11-28T13:02:16.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.131+0000 7f1ac7fff700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1aac0682f0 0x7f1aac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.132+0000 7f1ac7fff700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1aac0682f0 0x7f1aac06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80b4e60 tx=0x7f1ab4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:16.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.136+0000 7f1ac57fa700 1 -- 172.21.15.6:0/3590338445 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1abc053b20 con 0x7f1ac80c9950 2023-11-28T13:02:16.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.329+0000 7f1acf198700 1 -- 172.21.15.6:0/3590338445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ac80739f0 con 0x7f1ac80c9950 2023-11-28T13:02:16.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.329+0000 7f1ac57fa700 1 -- 172.21.15.6:0/3590338445 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1abc015020 con 0x7f1ac80c9950 2023-11-28T13:02:16.330 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:02:16.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 -- 172.21.15.6:0/3590338445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1aac0682f0 msgr2=0x7f1aac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1aac0682f0 0x7f1aac06a7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80b4e60 tx=0x7f1ab4008040 comp rx=0 tx=0).stop 2023-11-28T13:02:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 -- 172.21.15.6:0/3590338445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac80c9950 msgr2=0x7f1ac80cded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac80c9950 0x7f1ac80cded0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1abc000c00 tx=0x7f1abc004750 comp rx=0 tx=0).stop 2023-11-28T13:02:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 -- 172.21.15.6:0/3590338445 shutdown_connections 2023-11-28T13:02:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac815c880 0x7f1ac80c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ac80c9950 0x7f1ac80cded0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1aac0682f0 0x7f1aac06a7a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 --2- 172.21.15.6:0/3590338445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ac815c4b0 0x7f1ac80c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 -- 172.21.15.6:0/3590338445 >> 172.21.15.6:0/3590338445 conn(0x7f1ac80b0df0 msgr2=0x7f1ac815dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 -- 172.21.15.6:0/3590338445 shutdown_connections 2023-11-28T13:02:16.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.331+0000 7f1acf198700 1 -- 172.21.15.6:0/3590338445 wait complete. 2023-11-28T13:02:16.344 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:02:16.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.471+0000 7fdc0c141700 1 Processor -- start 2023-11-28T13:02:16.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.471+0000 7fdc0c141700 1 -- start start 2023-11-28T13:02:16.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc0c141700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdc040b62d0 0x7fdc040cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc0c141700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc040cd2a0 con 0x7fdc040b2d90 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc0c141700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc040d0400 con 0x7fdc0406f8b0 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc0c141700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc040d06e0 con 0x7fdc040b62d0 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc0a6de700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fdc040b2d90 0x7fdc040ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48308/0 (socket says 172.21.15.6:48308) 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc0a6de700 1 -- 172.21.15.6:0/4272416366 learned_addr learned my addr 172.21.15.6:0/4272416366 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc09edd700 1 --2- 172.21.15.6:0/4272416366 >> v2:172.21.15.61:3300/0 conn(0x7fdc040b62d0 0x7fdc040cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc08edb700 1 -- 172.21.15.6:0/4272416366 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581412468 0 0) 0x7fdc040cd2a0 con 0x7fdc040b2d90 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc08edb700 1 -- 172.21.15.6:0/4272416366 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbf00035b0 con 0x7fdc040b2d90 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc09edd700 1 -- 172.21.15.6:0/4272416366 >> v1:172.21.15.61:6789/0 conn(0x7fdc0406f8b0 legacy=0x7fdc040b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc08edb700 1 -- 172.21.15.6:0/4272416366 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 250759431 0 0) 0x7fdbf00035b0 con 0x7fdc040b2d90 2023-11-28T13:02:16.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc09edd700 1 -- 172.21.15.6:0/4272416366 >> v1:172.21.15.6:6789/0 conn(0x7fdc040b2d90 legacy=0x7fdc040ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.472+0000 7fdc09edd700 1 -- 172.21.15.6:0/4272416366 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc040cffb0 con 0x7fdc040b62d0 2023-11-28T13:02:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc09edd700 1 --2- 172.21.15.6:0/4272416366 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc040b62d0 0x7fdc040cb630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009840 tx=0x7fdbf4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f705f988f66f0c1f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc08edb700 1 -- 172.21.15.6:0/4272416366 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0019f0 con 0x7fdc040b62d0 2023-11-28T13:02:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc08edb700 1 -- 172.21.15.6:0/4272416366 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf4004030 con 0x7fdc040b62d0 2023-11-28T13:02:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc08edb700 1 -- 172.21.15.6:0/4272416366 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf40043c0 con 0x7fdc040b62d0 2023-11-28T13:02:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc0c141700 1 -- 172.21.15.6:0/4272416366 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc040b62d0 msgr2=0x7fdc040cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc0c141700 1 --2- 172.21.15.6:0/4272416366 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc040b62d0 0x7fdc040cb630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009840 tx=0x7fdbf4009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc0c141700 1 -- 172.21.15.6:0/4272416366 shutdown_connections 2023-11-28T13:02:16.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc0c141700 1 --2- 172.21.15.6:0/4272416366 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc040b62d0 0x7fdc040cb630 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc0c141700 1 -- 172.21.15.6:0/4272416366 >> 172.21.15.6:0/4272416366 conn(0x7fdc040cd680 msgr2=0x7fdc040cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc0c141700 1 -- 172.21.15.6:0/4272416366 shutdown_connections 2023-11-28T13:02:16.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.473+0000 7fdc0c141700 1 -- 172.21.15.6:0/4272416366 wait complete. 2023-11-28T13:02:16.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.475+0000 7fdc0c141700 1 Processor -- start 2023-11-28T13:02:16.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.475+0000 7fdc0c141700 1 -- start start 2023-11-28T13:02:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.475+0000 7fdc0c141700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc0406f8b0 0x7fdc040c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.475+0000 7fdc0c141700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc040b2d90 0x7fdc04153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.475+0000 7fdc0c141700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc040b62d0 0x7fdc040b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.475+0000 7fdc0c141700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040c5f50 con 0x7fdc0406f8b0 2023-11-28T13:02:16.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.475+0000 7fdc0c141700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040c6230 con 0x7fdc040b62d0 2023-11-28T13:02:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.475+0000 7fdc0c141700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040c6510 con 0x7fdc040b2d90 2023-11-28T13:02:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.475+0000 7fdc09edd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc0406f8b0 0x7fdc040c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc09edd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc0406f8b0 0x7fdc040c6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:60990/0 (socket says 172.21.15.6:60990) 2023-11-28T13:02:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc09edd700 1 -- 172.21.15.6:0/3732155707 learned_addr learned my addr 172.21.15.6:0/3732155707 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:16.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc096dc700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc040b2d90 0x7fdc04153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc0a6de700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc040b62d0 0x7fdc040b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc09edd700 1 -- 172.21.15.6:0/3732155707 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc040b2d90 msgr2=0x7fdc04153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc09edd700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc040b2d90 0x7fdc04153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc09edd700 1 -- 172.21.15.6:0/3732155707 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc040b62d0 msgr2=0x7fdc040b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc09edd700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc040b62d0 0x7fdc040b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc09edd700 1 -- 172.21.15.6:0/3732155707 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf40094a0 con 0x7fdc0406f8b0 2023-11-28T13:02:16.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc096dc700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc040b2d90 0x7fdc04153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:16.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc09edd700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc0406f8b0 0x7fdc040c6ff0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009b90 tx=0x7fdbf4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:16.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdbfa7fc700 1 -- 172.21.15.6:0/3732155707 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4011830 con 0x7fdc0406f8b0 2023-11-28T13:02:16.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdbfa7fc700 1 -- 172.21.15.6:0/3732155707 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf401f070 con 0x7fdc0406f8b0 2023-11-28T13:02:16.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc0c141700 1 -- 172.21.15.6:0/3732155707 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc040b66a0 con 0x7fdc0406f8b0 2023-11-28T13:02:16.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdbfa7fc700 1 -- 172.21.15.6:0/3732155707 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4011bd0 con 0x7fdc0406f8b0 2023-11-28T13:02:16.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.476+0000 7fdc0c141700 1 -- 172.21.15.6:0/3732155707 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0406fd30 con 0x7fdc0406f8b0 2023-11-28T13:02:16.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.477+0000 7fdc0c141700 1 -- 172.21.15.6:0/3732155707 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc040cce50 con 0x7fdc0406f8b0 2023-11-28T13:02:16.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.477+0000 7fdbfa7fc700 1 -- 172.21.15.6:0/3732155707 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdbf4017020 con 0x7fdc0406f8b0 2023-11-28T13:02:16.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.478+0000 7fdbfa7fc700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbe80682f0 0x7fdbe806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.478+0000 7fdbfa7fc700 1 -- 172.21.15.6:0/3732155707 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fdbf4021070 con 0x7fdc0406f8b0 2023-11-28T13:02:16.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.478+0000 7fdc096dc700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbe80682f0 0x7fdbe806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.488 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.478+0000 7fdc096dc700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbe80682f0 0x7fdbe806a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdc00000f60 tx=0x7fdc00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:16.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.482+0000 7fdbfa7fc700 1 -- 172.21.15.6:0/3732155707 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdbf4053ef0 con 0x7fdc0406f8b0 2023-11-28T13:02:16.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.674+0000 7fdc0c141700 1 -- 172.21.15.6:0/3732155707 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdc040afa10 con 0x7fdc0406f8b0 2023-11-28T13:02:16.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.675+0000 7fdbfa7fc700 1 -- 172.21.15.6:0/3732155707 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdbf402b020 con 0x7fdc0406f8b0 2023-11-28T13:02:16.677 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:02:16.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 -- 172.21.15.6:0/3732155707 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbe80682f0 msgr2=0x7fdbe806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbe80682f0 0x7fdbe806a7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdc00000f60 tx=0x7fdc00009040 comp rx=0 tx=0).stop 2023-11-28T13:02:16.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 -- 172.21.15.6:0/3732155707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc0406f8b0 msgr2=0x7fdc040c6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc0406f8b0 0x7fdc040c6ff0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009b90 tx=0x7fdbf4004750 comp rx=0 tx=0).stop 2023-11-28T13:02:16.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 -- 172.21.15.6:0/3732155707 shutdown_connections 2023-11-28T13:02:16.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdc040b62d0 0x7fdc040b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdc0406f8b0 0x7fdc040c6ff0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdbe80682f0 0x7fdbe806a7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 --2- 172.21.15.6:0/3732155707 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdc040b2d90 0x7fdc04153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 -- 172.21.15.6:0/3732155707 >> 172.21.15.6:0/3732155707 conn(0x7fdc040cd680 msgr2=0x7fdc040b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:16.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 -- 172.21.15.6:0/3732155707 shutdown_connections 2023-11-28T13:02:16.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.677+0000 7fdc0c141700 1 -- 172.21.15.6:0/3732155707 wait complete. 2023-11-28T13:02:16.691 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:02:16.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.817+0000 7f614820c700 1 Processor -- start 2023-11-28T13:02:16.819 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.817+0000 7f614820c700 1 -- start start 2023-11-28T13:02:16.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.818+0000 7f614820c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f61400b1d40 0x7f61400cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.818+0000 7f614820c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f614015dbf0 0x7f61400ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.818+0000 7f614820c700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61400ceff0 con 0x7f6140159500 2023-11-28T13:02:16.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.818+0000 7f614820c700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614015e090 con 0x7f61400b1d40 2023-11-28T13:02:16.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.818+0000 7f614820c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61400cede0 con 0x7f614015dbf0 2023-11-28T13:02:16.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f61457a7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f61400b1d40 0x7f61400cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f6145fa8700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6140159500 0x7f61400b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48318/0 (socket says 172.21.15.6:48318) 2023-11-28T13:02:16.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f6145fa8700 1 -- 172.21.15.6:0/4084734411 learned_addr learned my addr 172.21.15.6:0/4084734411 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:16.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f61457a7700 1 -- 172.21.15.6:0/4084734411 >> v2:172.21.15.61:3300/0 conn(0x7f614015dbf0 msgr2=0x7f61400ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:16.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f61467a9700 1 --2- 172.21.15.6:0/4084734411 >> v2:172.21.15.61:3300/0 conn(0x7f614015dbf0 0x7f61400ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f61457a7700 1 --2- 172.21.15.6:0/4084734411 >> v2:172.21.15.61:3300/0 conn(0x7f614015dbf0 0x7f61400ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f6144fa6700 1 -- 172.21.15.6:0/4084734411 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924155743 0 0) 0x7f61400ceff0 con 0x7f6140159500 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f61457a7700 1 -- 172.21.15.6:0/4084734411 >> v1:172.21.15.6:6789/0 conn(0x7f6140159500 legacy=0x7f61400b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f61457a7700 1 -- 172.21.15.6:0/4084734411 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400cf2d0 con 0x7f61400b1d40 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f61457a7700 1 --2- 172.21.15.6:0/4084734411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 0x7f61400cd140 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f61400718b0 tx=0x7f613c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e429c5b555d459 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f6144fa6700 1 -- 172.21.15.6:0/4084734411 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c011830 con 0x7f61400b1d40 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f6144fa6700 1 -- 172.21.15.6:0/4084734411 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c011990 con 0x7f61400b1d40 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.819+0000 7f6144fa6700 1 -- 172.21.15.6:0/4084734411 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c011d20 con 0x7f61400b1d40 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.820+0000 7f614820c700 1 -- 172.21.15.6:0/4084734411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 msgr2=0x7f61400cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.820+0000 7f614820c700 1 --2- 172.21.15.6:0/4084734411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 0x7f61400cd140 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f61400718b0 tx=0x7f613c009b20 comp rx=0 tx=0).stop 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.820+0000 7f614820c700 1 -- 172.21.15.6:0/4084734411 shutdown_connections 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.820+0000 7f614820c700 1 --2- 172.21.15.6:0/4084734411 >> v2:172.21.15.61:3300/0 conn(0x7f614015dbf0 0x7f61400ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.820+0000 7f614820c700 1 --2- 172.21.15.6:0/4084734411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 0x7f61400cd140 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.820+0000 7f614820c700 1 -- 172.21.15.6:0/4084734411 >> 172.21.15.6:0/4084734411 conn(0x7f61400cc630 msgr2=0x7f61400cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.820+0000 7f614820c700 1 -- 172.21.15.6:0/4084734411 shutdown_connections 2023-11-28T13:02:16.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.820+0000 7f614820c700 1 -- 172.21.15.6:0/4084734411 wait complete. 2023-11-28T13:02:16.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.822+0000 7f614820c700 1 Processor -- start 2023-11-28T13:02:16.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.822+0000 7f614820c700 1 -- start start 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f614820c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 0x7f61401f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f614820c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6140159500 0x7f61400b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f614820c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f614015dbf0 0x7f6140153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f614820c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400b58f0 con 0x7f61400b1d40 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f614820c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400b56c0 con 0x7f6140159500 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f614820c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61401f37c0 con 0x7f614015dbf0 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f6145fa8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 0x7f61401f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f6145fa8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 0x7f61401f0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32772/0 (socket says 172.21.15.6:32772) 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f6145fa8700 1 -- 172.21.15.6:0/3973820983 learned_addr learned my addr 172.21.15.6:0/3973820983 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f61467a9700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f614015dbf0 0x7f6140153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f61457a7700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6140159500 0x7f61400b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f6145fa8700 1 -- 172.21.15.6:0/3973820983 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f614015dbf0 msgr2=0x7f6140153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f6145fa8700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f614015dbf0 0x7f6140153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f6145fa8700 1 -- 172.21.15.6:0/3973820983 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6140159500 msgr2=0x7f61400b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f6145fa8700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6140159500 0x7f61400b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f6145fa8700 1 -- 172.21.15.6:0/3973820983 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0094a0 con 0x7f61400b1d40 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f61467a9700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f614015dbf0 0x7f6140153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f6145fa8700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 0x7f61401f0b10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f61300011e0 tx=0x7f613000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f61367fc700 1 -- 172.21.15.6:0/3973820983 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130004c80 con 0x7f61400b1d40 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f61367fc700 1 -- 172.21.15.6:0/3973820983 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6130004e00 con 0x7f61400b1d40 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f614820c700 1 -- 172.21.15.6:0/3973820983 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614015d3f0 con 0x7f61400b1d40 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f61367fc700 1 -- 172.21.15.6:0/3973820983 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130007310 con 0x7f61400b1d40 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.823+0000 7f614820c700 1 -- 172.21.15.6:0/3973820983 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614015d110 con 0x7f61400b1d40 2023-11-28T13:02:16.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.824+0000 7f614820c700 1 -- 172.21.15.6:0/3973820983 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614015d6d0 con 0x7f61400b1d40 2023-11-28T13:02:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.825+0000 7f61367fc700 1 -- 172.21.15.6:0/3973820983 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f613001a020 con 0x7f61400b1d40 2023-11-28T13:02:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.825+0000 7f61367fc700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61240682f0 0x7f612406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.825+0000 7f61367fc700 1 -- 172.21.15.6:0/3973820983 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6130085a60 con 0x7f61400b1d40 2023-11-28T13:02:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.825+0000 7f61457a7700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61240682f0 0x7f612406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:16.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.826+0000 7f61457a7700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61240682f0 0x7f612406a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f61400718b0 tx=0x7f613c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:16.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:16.830+0000 7f61367fc700 1 -- 172.21.15.6:0/3973820983 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6130052140 con 0x7f61400b1d40 2023-11-28T13:02:17.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.021+0000 7f614820c700 1 -- 172.21.15.6:0/3973820983 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61400cc000 con 0x7f61400b1d40 2023-11-28T13:02:17.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.022+0000 7f61367fc700 1 -- 172.21.15.6:0/3973820983 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6130055760 con 0x7f61400b1d40 2023-11-28T13:02:17.024 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:02:17.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 -- 172.21.15.6:0/3973820983 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61240682f0 msgr2=0x7f612406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61240682f0 0x7f612406a7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f61400718b0 tx=0x7f613c008030 comp rx=0 tx=0).stop 2023-11-28T13:02:17.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 -- 172.21.15.6:0/3973820983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 msgr2=0x7f61401f0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 0x7f61401f0b10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f61300011e0 tx=0x7f613000cae0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 -- 172.21.15.6:0/3973820983 shutdown_connections 2023-11-28T13:02:17.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61240682f0 0x7f612406a7a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f614015dbf0 0x7f6140153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61400b1d40 0x7f61401f0b10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 --2- 172.21.15.6:0/3973820983 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6140159500 0x7f61400b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 -- 172.21.15.6:0/3973820983 >> 172.21.15.6:0/3973820983 conn(0x7f61400cc630 msgr2=0x7f61400b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:17.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 -- 172.21.15.6:0/3973820983 shutdown_connections 2023-11-28T13:02:17.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.024+0000 7f614820c700 1 -- 172.21.15.6:0/3973820983 wait complete. 2023-11-28T13:02:17.038 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:17.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.161+0000 7f9ad4410700 1 Processor -- start 2023-11-28T13:02:17.163 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.161+0000 7f9ad4410700 1 -- start start 2023-11-28T13:02:17.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.162+0000 7f9ad4410700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9acc159590 0x7f9acc06e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.162+0000 7f9ad4410700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ce890 0x7f9acc0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.162+0000 7f9ad4410700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc15d120 con 0x7f9acc15db70 2023-11-28T13:02:17.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.162+0000 7f9ad4410700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc0cf250 con 0x7f9acc159590 2023-11-28T13:02:17.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.162+0000 7f9ad4410700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc15d6e0 con 0x7f9acc0ce890 2023-11-28T13:02:17.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.162+0000 7f9ad21ac700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9acc159590 0x7f9acc06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.162+0000 7f9ad19ab700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f9acc15db70 0x7f9acc0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48332/0 (socket says 172.21.15.6:48332) 2023-11-28T13:02:17.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.162+0000 7f9ad19ab700 1 -- 172.21.15.6:0/1584817567 learned_addr learned my addr 172.21.15.6:0/1584817567 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad29ad700 1 --2- 172.21.15.6:0/1584817567 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ce890 0x7f9acc0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad11aa700 1 -- 172.21.15.6:0/1584817567 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355831418 0 0) 0x7f9acc15d120 con 0x7f9acc15db70 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad11aa700 1 -- 172.21.15.6:0/1584817567 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab80035b0 con 0x7f9acc15db70 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad21ac700 1 -- 172.21.15.6:0/1584817567 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ce890 msgr2=0x7f9acc0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad21ac700 1 --2- 172.21.15.6:0/1584817567 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ce890 0x7f9acc0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad21ac700 1 -- 172.21.15.6:0/1584817567 >> v1:172.21.15.6:6789/0 conn(0x7f9acc15db70 legacy=0x7f9acc0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad21ac700 1 -- 172.21.15.6:0/1584817567 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0aef60 con 0x7f9acc159590 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad21ac700 1 --2- 172.21.15.6:0/1584817567 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc159590 0x7f9acc06e400 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009840 tx=0x7f9abc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2102da698097c428 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad11aa700 1 -- 172.21.15.6:0/1584817567 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc011830 con 0x7f9acc159590 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad11aa700 1 -- 172.21.15.6:0/1584817567 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc011990 con 0x7f9acc159590 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.163+0000 7f9ad11aa700 1 -- 172.21.15.6:0/1584817567 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc011d20 con 0x7f9acc159590 2023-11-28T13:02:17.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.164+0000 7f9ad4410700 1 -- 172.21.15.6:0/1584817567 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc159590 msgr2=0x7f9acc06e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.164+0000 7f9ad4410700 1 --2- 172.21.15.6:0/1584817567 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc159590 0x7f9acc06e400 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009840 tx=0x7f9abc009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:17.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.164+0000 7f9ad4410700 1 -- 172.21.15.6:0/1584817567 shutdown_connections 2023-11-28T13:02:17.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.164+0000 7f9ad4410700 1 --2- 172.21.15.6:0/1584817567 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc0ce890 0x7f9acc0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.164+0000 7f9ad4410700 1 --2- 172.21.15.6:0/1584817567 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc159590 0x7f9acc06e400 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.164+0000 7f9ad4410700 1 -- 172.21.15.6:0/1584817567 >> 172.21.15.6:0/1584817567 conn(0x7f9acc0b1650 msgr2=0x7f9acc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:17.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.164+0000 7f9ad4410700 1 -- 172.21.15.6:0/1584817567 shutdown_connections 2023-11-28T13:02:17.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.164+0000 7f9ad4410700 1 -- 172.21.15.6:0/1584817567 wait complete. 2023-11-28T13:02:17.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.165+0000 7f9ad4410700 1 Processor -- start 2023-11-28T13:02:17.169 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.165+0000 7f9ad4410700 1 -- start start 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad4410700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0ce890 0x7f9acc154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad4410700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9acc159590 0x7f9acc159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad4410700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc15db70 0x7f9acc0c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad4410700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1f67a0 con 0x7f9acc0ce890 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad4410700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0b3a00 con 0x7f9acc159590 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad4410700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc154550 con 0x7f9acc15db70 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad21ac700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0ce890 0x7f9acc154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad21ac700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0ce890 0x7f9acc154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32784/0 (socket says 172.21.15.6:32784) 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad21ac700 1 -- 172.21.15.6:0/3816788058 learned_addr learned my addr 172.21.15.6:0/3816788058 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad19ab700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9acc159590 0x7f9acc159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad29ad700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc15db70 0x7f9acc0c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad21ac700 1 -- 172.21.15.6:0/3816788058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc15db70 msgr2=0x7f9acc0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad21ac700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc15db70 0x7f9acc0c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad21ac700 1 -- 172.21.15.6:0/3816788058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9acc159590 msgr2=0x7f9acc159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad21ac700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9acc159590 0x7f9acc159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad21ac700 1 -- 172.21.15.6:0/3816788058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc0094a0 con 0x7f9acc0ce890 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad21ac700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0ce890 0x7f9acc154010 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009fd0 tx=0x7f9abc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ac2ffd700 1 -- 172.21.15.6:0/3816788058 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0036a0 con 0x7f9acc0ce890 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ac2ffd700 1 -- 172.21.15.6:0/3816788058 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc01f070 con 0x7f9acc0ce890 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ad4410700 1 -- 172.21.15.6:0/3816788058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc15d370 con 0x7f9acc0ce890 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.166+0000 7f9ac2ffd700 1 -- 172.21.15.6:0/3816788058 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc003a60 con 0x7f9acc0ce890 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.167+0000 7f9ad4410700 1 -- 172.21.15.6:0/3816788058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc0af240 con 0x7f9acc0ce890 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.168+0000 7f9ad4410700 1 -- 172.21.15.6:0/3816788058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc15e010 con 0x7f9acc0ce890 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.168+0000 7f9ac2ffd700 1 -- 172.21.15.6:0/3816788058 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9abc00f050 con 0x7f9acc0ce890 2023-11-28T13:02:17.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.168+0000 7f9ac2ffd700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ab00682f0 0x7f9ab006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.168+0000 7f9ac2ffd700 1 -- 172.21.15.6:0/3816788058 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9abc021070 con 0x7f9acc0ce890 2023-11-28T13:02:17.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.168+0000 7f9ad19ab700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ab00682f0 0x7f9ab006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.169+0000 7f9ad19ab700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ab00682f0 0x7f9ab006a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9acc1e5460 tx=0x7f9ac8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:17.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.173+0000 7f9ac2ffd700 1 -- 172.21.15.6:0/3816788058 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9abc053c50 con 0x7f9acc0ce890 2023-11-28T13:02:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.414+0000 7f9ad4410700 1 -- 172.21.15.6:0/3816788058 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9acc0675a0 con 0x7f9acc0ce890 2023-11-28T13:02:17.419 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.415+0000 7f9ac2ffd700 1 -- 172.21.15.6:0/3816788058 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f9abc057270 con 0x7f9acc0ce890 2023-11-28T13:02:17.420 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:17.420 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4026920958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4026920958},{"type":"v1","addr":"172.21.15.61:6833","nonce":4026920958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2212733196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2212733196},{"type":"v1","addr":"172.21.15.61:6835","nonce":2212733196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1674709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1674709721},{"type":"v1","addr":"172.21.15.6:6836","nonce":1674709721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:42.595217+0000","modified":"2023-11-28T13:01:45.602537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/2789212407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":2789212407},{"type":"v1","addr":"172.21.15.6:6837","nonce":2789212407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:02:17.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 -- 172.21.15.6:0/3816788058 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ab00682f0 msgr2=0x7f9ab006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ab00682f0 0x7f9ab006a7a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9acc1e5460 tx=0x7f9ac8009040 comp rx=0 tx=0).stop 2023-11-28T13:02:17.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 -- 172.21.15.6:0/3816788058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0ce890 msgr2=0x7f9acc154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0ce890 0x7f9acc154010 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009fd0 tx=0x7f9abc004750 comp rx=0 tx=0).stop 2023-11-28T13:02:17.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 -- 172.21.15.6:0/3816788058 shutdown_connections 2023-11-28T13:02:17.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ab00682f0 0x7f9ab006a7a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9acc15db70 0x7f9acc0c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc0ce890 0x7f9acc154010 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 --2- 172.21.15.6:0/3816788058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9acc159590 0x7f9acc159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 -- 172.21.15.6:0/3816788058 >> 172.21.15.6:0/3816788058 conn(0x7f9acc0b1650 msgr2=0x7f9acc0d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 -- 172.21.15.6:0/3816788058 shutdown_connections 2023-11-28T13:02:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.417+0000 7f9ad4410700 1 -- 172.21.15.6:0/3816788058 wait complete. 2023-11-28T13:02:17.422 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2023-11-28T13:02:17.430 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:17.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.553+0000 7fa40aa79700 1 Processor -- start 2023-11-28T13:02:17.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.553+0000 7fa40aa79700 1 -- start start 2023-11-28T13:02:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.554+0000 7fa40aa79700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 0x7fa4040cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.554+0000 7fa40aa79700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa404161090 0x7fa4040ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.554+0000 7fa40aa79700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4040ce340 con 0x7fa4040caeb0 2023-11-28T13:02:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.554+0000 7fa40aa79700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4040cebe0 con 0x7fa404161090 2023-11-28T13:02:17.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.554+0000 7fa40aa79700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040cb3f0 con 0x7fa40415c990 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.554+0000 7fa408815700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 0x7fa4040cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.554+0000 7fa403fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa404161090 0x7fa4040ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.554+0000 7fa408815700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 0x7fa4040cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:50728/0 (socket says 172.21.15.6:50728) 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.554+0000 7fa408815700 1 -- 172.21.15.6:0/812395903 learned_addr learned my addr 172.21.15.6:0/812395903 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.554+0000 7fa403fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa404161090 0x7fa4040ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:36512/0 (socket says 172.21.15.6:36512) 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.555+0000 7fa4037fe700 1 -- 172.21.15.6:0/812395903 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984509152 0 0) 0x7fa4040ce340 con 0x7fa4040caeb0 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.555+0000 7fa408815700 1 -- 172.21.15.6:0/812395903 >> v2:172.21.15.61:3300/0 conn(0x7fa404161090 msgr2=0x7fa4040ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.555+0000 7fa408815700 1 --2- 172.21.15.6:0/812395903 >> v2:172.21.15.61:3300/0 conn(0x7fa404161090 0x7fa4040ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.555+0000 7fa408815700 1 -- 172.21.15.6:0/812395903 >> v1:172.21.15.61:6789/0 conn(0x7fa4040caeb0 legacy=0x7fa4040cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.555+0000 7fa408815700 1 -- 172.21.15.6:0/812395903 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040ce590 con 0x7fa40415c990 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.555+0000 7fa403fff700 1 --2- 172.21.15.6:0/812395903 >> v2:172.21.15.61:3300/0 conn(0x7fa404161090 0x7fa4040ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:17.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.555+0000 7fa408815700 1 --2- 172.21.15.6:0/812395903 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 0x7fa4040cf070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80098f0 tx=0x7fa3f8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97d24b327f46fb5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:17.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.555+0000 7fa4037fe700 1 -- 172.21.15.6:0/812395903 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8011830 con 0x7fa40415c990 2023-11-28T13:02:17.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.555+0000 7fa4037fe700 1 -- 172.21.15.6:0/812395903 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f8011990 con 0x7fa40415c990 2023-11-28T13:02:17.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.555+0000 7fa4037fe700 1 -- 172.21.15.6:0/812395903 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8011d20 con 0x7fa40415c990 2023-11-28T13:02:17.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.556+0000 7fa40aa79700 1 -- 172.21.15.6:0/812395903 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 msgr2=0x7fa4040cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.556+0000 7fa40aa79700 1 --2- 172.21.15.6:0/812395903 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 0x7fa4040cf070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80098f0 tx=0x7fa3f8009c00 comp rx=0 tx=0).stop 2023-11-28T13:02:17.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.556+0000 7fa40aa79700 1 -- 172.21.15.6:0/812395903 shutdown_connections 2023-11-28T13:02:17.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.556+0000 7fa40aa79700 1 --2- 172.21.15.6:0/812395903 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 0x7fa4040cf070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.556+0000 7fa40aa79700 1 --2- 172.21.15.6:0/812395903 >> v2:172.21.15.61:3300/0 conn(0x7fa404161090 0x7fa4040ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.556+0000 7fa40aa79700 1 -- 172.21.15.6:0/812395903 >> 172.21.15.6:0/812395903 conn(0x7fa4040b1650 msgr2=0x7fa4040b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.556+0000 7fa40aa79700 1 -- 172.21.15.6:0/812395903 shutdown_connections 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.556+0000 7fa40aa79700 1 -- 172.21.15.6:0/812395903 wait complete. 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.557+0000 7fa40aa79700 1 Processor -- start 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.557+0000 7fa40aa79700 1 -- start start 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa40aa79700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4040caeb0 0x7fa4040c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa40aa79700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 0x7fa404148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa40aa79700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa404161090 0x7fa404148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa40aa79700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4041e65b0 con 0x7fa4040caeb0 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa40aa79700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040abf10 con 0x7fa404161090 2023-11-28T13:02:17.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa40aa79700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4041e89a0 con 0x7fa40415c990 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa408815700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4040caeb0 0x7fa4040c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa408815700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4040caeb0 0x7fa4040c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32800/0 (socket says 172.21.15.6:32800) 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa408815700 1 -- 172.21.15.6:0/1004020072 learned_addr learned my addr 172.21.15.6:0/1004020072 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa403fff700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 0x7fa404148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa409016700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa404161090 0x7fa404148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa408815700 1 -- 172.21.15.6:0/1004020072 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 msgr2=0x7fa404148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa408815700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 0x7fa404148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa408815700 1 -- 172.21.15.6:0/1004020072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa404161090 msgr2=0x7fa404148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa408815700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa404161090 0x7fa404148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa408815700 1 -- 172.21.15.6:0/1004020072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00094a0 con 0x7fa4040caeb0 2023-11-28T13:02:17.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.558+0000 7fa408815700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4040caeb0 0x7fa4040c8fb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009c40 tx=0x7fa3f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:17.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.559+0000 7fa4017fa700 1 -- 172.21.15.6:0/1004020072 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8004550 con 0x7fa4040caeb0 2023-11-28T13:02:17.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.559+0000 7fa4017fa700 1 -- 172.21.15.6:0/1004020072 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f801f070 con 0x7fa4040caeb0 2023-11-28T13:02:17.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.559+0000 7fa40aa79700 1 -- 172.21.15.6:0/1004020072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f8009550 con 0x7fa4040caeb0 2023-11-28T13:02:17.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.559+0000 7fa4017fa700 1 -- 172.21.15.6:0/1004020072 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f80038e0 con 0x7fa4040caeb0 2023-11-28T13:02:17.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.559+0000 7fa40aa79700 1 -- 172.21.15.6:0/1004020072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4040cf5b0 con 0x7fa4040caeb0 2023-11-28T13:02:17.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.560+0000 7fa40aa79700 1 -- 172.21.15.6:0/1004020072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4040ce870 con 0x7fa4040caeb0 2023-11-28T13:02:17.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.560+0000 7fa4017fa700 1 -- 172.21.15.6:0/1004020072 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa3f800f050 con 0x7fa4040caeb0 2023-11-28T13:02:17.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.561+0000 7fa4017fa700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa3e80682f0 0x7fa3e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.561+0000 7fa4017fa700 1 -- 172.21.15.6:0/1004020072 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa3f8021070 con 0x7fa4040caeb0 2023-11-28T13:02:17.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.561+0000 7fa403fff700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa3e80682f0 0x7fa3e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.561+0000 7fa403fff700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa3e80682f0 0x7fa3e806a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa4041e7c80 tx=0x7fa3f0009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:17.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.565+0000 7fa4017fa700 1 -- 172.21.15.6:0/1004020072 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa3f8053d00 con 0x7fa4040caeb0 2023-11-28T13:02:17.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.759+0000 7fa40aa79700 1 -- 172.21.15.6:0/1004020072 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4040cf740 con 0x7fa4040caeb0 2023-11-28T13:02:17.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.760+0000 7fa4017fa700 1 -- 172.21.15.6:0/1004020072 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18066 (secure 0 0 0) 0x7fa3f8057320 con 0x7fa4040caeb0 2023-11-28T13:02:17.762 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:17.762 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":42,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:11.026507+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:41.144348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:41.554568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.18","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 -- 172.21.15.6:0/1004020072 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa3e80682f0 msgr2=0x7fa3e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa3e80682f0 0x7fa3e806a7a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa4041e7c80 tx=0x7fa3f0009210 comp rx=0 tx=0).stop 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 -- 172.21.15.6:0/1004020072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4040caeb0 msgr2=0x7fa4040c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4040caeb0 0x7fa4040c8fb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009c40 tx=0x7fa3f8004750 comp rx=0 tx=0).stop 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 -- 172.21.15.6:0/1004020072 shutdown_connections 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa404161090 0x7fa404148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4040caeb0 0x7fa4040c8fb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa3e80682f0 0x7fa3e806a7a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 --2- 172.21.15.6:0/1004020072 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa40415c990 0x7fa404148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 -- 172.21.15.6:0/1004020072 >> 172.21.15.6:0/1004020072 conn(0x7fa4040b1650 msgr2=0x7fa40406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 -- 172.21.15.6:0/1004020072 shutdown_connections 2023-11-28T13:02:17.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.762+0000 7fa40aa79700 1 -- 172.21.15.6:0/1004020072 wait complete. 2023-11-28T13:02:17.776 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:02:17.776 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:17.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.898+0000 7fd067a3c700 1 Processor -- start 2023-11-28T13:02:17.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.899+0000 7fd067a3c700 1 -- start start 2023-11-28T13:02:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.899+0000 7fd067a3c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd060161090 0x7fd0600ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.899+0000 7fd067a3c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0600caeb0 0x7fd0600cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.899+0000 7fd067a3c700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0600ce340 con 0x7fd06015c990 2023-11-28T13:02:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.899+0000 7fd067a3c700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0600cebe0 con 0x7fd0600caeb0 2023-11-28T13:02:17.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.899+0000 7fd067a3c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0600cf780 con 0x7fd060161090 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd065fd9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0600caeb0 0x7fd0600cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd0657d8700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd06015c990 0x7fd0600cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48358/0 (socket says 172.21.15.6:48358) 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd0657d8700 1 -- 172.21.15.6:0/2077818416 learned_addr learned my addr 172.21.15.6:0/2077818416 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd064fd7700 1 --2- 172.21.15.6:0/2077818416 >> v2:172.21.15.61:3300/0 conn(0x7fd060161090 0x7fd0600ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd065fd9700 1 -- 172.21.15.6:0/2077818416 >> v2:172.21.15.61:3300/0 conn(0x7fd060161090 msgr2=0x7fd0600ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd065fd9700 1 --2- 172.21.15.6:0/2077818416 >> v2:172.21.15.61:3300/0 conn(0x7fd060161090 0x7fd0600ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd065fd9700 1 -- 172.21.15.6:0/2077818416 >> v1:172.21.15.6:6789/0 conn(0x7fd06015c990 legacy=0x7fd0600cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd053fff700 1 -- 172.21.15.6:0/2077818416 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749756438 0 0) 0x7fd0600ce340 con 0x7fd06015c990 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd065fd9700 1 -- 172.21.15.6:0/2077818416 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600cb3f0 con 0x7fd0600caeb0 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd065fd9700 1 --2- 172.21.15.6:0/2077818416 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 0x7fd0600cc310 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd06006ab20 tx=0x7fd05800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6478d088e05cd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd053fff700 1 -- 172.21.15.6:0/2077818416 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05800e070 con 0x7fd0600caeb0 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd053fff700 1 -- 172.21.15.6:0/2077818416 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05800bbd0 con 0x7fd0600caeb0 2023-11-28T13:02:17.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.900+0000 7fd053fff700 1 -- 172.21.15.6:0/2077818416 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058012430 con 0x7fd0600caeb0 2023-11-28T13:02:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.901+0000 7fd067a3c700 1 -- 172.21.15.6:0/2077818416 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 msgr2=0x7fd0600cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.901+0000 7fd067a3c700 1 --2- 172.21.15.6:0/2077818416 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 0x7fd0600cc310 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd06006ab20 tx=0x7fd05800b330 comp rx=0 tx=0).stop 2023-11-28T13:02:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.901+0000 7fd067a3c700 1 -- 172.21.15.6:0/2077818416 shutdown_connections 2023-11-28T13:02:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.901+0000 7fd067a3c700 1 --2- 172.21.15.6:0/2077818416 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 0x7fd0600cc310 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.901+0000 7fd067a3c700 1 --2- 172.21.15.6:0/2077818416 >> v2:172.21.15.61:3300/0 conn(0x7fd060161090 0x7fd0600ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.901+0000 7fd067a3c700 1 -- 172.21.15.6:0/2077818416 >> 172.21.15.6:0/2077818416 conn(0x7fd0600b1650 msgr2=0x7fd0600b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.901+0000 7fd067a3c700 1 -- 172.21.15.6:0/2077818416 shutdown_connections 2023-11-28T13:02:17.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.901+0000 7fd067a3c700 1 -- 172.21.15.6:0/2077818416 wait complete. 2023-11-28T13:02:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.902+0000 7fd067a3c700 1 Processor -- start 2023-11-28T13:02:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.902+0000 7fd067a3c700 1 -- start start 2023-11-28T13:02:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd067a3c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 0x7fd0600c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd067a3c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06015c990 0x7fd06014ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd067a3c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060161090 0x7fd0601600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd067a3c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600c5730 con 0x7fd0600caeb0 2023-11-28T13:02:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd067a3c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600c4e00 con 0x7fd060161090 2023-11-28T13:02:17.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd067a3c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600c50e0 con 0x7fd06015c990 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd0657d8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 0x7fd0600c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd0657d8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 0x7fd0600c3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32812/0 (socket says 172.21.15.6:32812) 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd0657d8700 1 -- 172.21.15.6:0/2471370677 learned_addr learned my addr 172.21.15.6:0/2471370677 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd064fd7700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06015c990 0x7fd06014ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd065fd9700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060161090 0x7fd0601600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd0657d8700 1 -- 172.21.15.6:0/2471370677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06015c990 msgr2=0x7fd06014ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd0657d8700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06015c990 0x7fd06014ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd0657d8700 1 -- 172.21.15.6:0/2471370677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060161090 msgr2=0x7fd0601600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd0657d8700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060161090 0x7fd0601600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd0657d8700 1 -- 172.21.15.6:0/2471370677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058009a70 con 0x7fd0600caeb0 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.903+0000 7fd0657d8700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 0x7fd0600c3b60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0011e0 tx=0x7fd05c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:17.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.904+0000 7fd051ffb700 1 -- 172.21.15.6:0/2471370677 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c004c80 con 0x7fd0600caeb0 2023-11-28T13:02:17.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.904+0000 7fd051ffb700 1 -- 172.21.15.6:0/2471370677 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c004e00 con 0x7fd0600caeb0 2023-11-28T13:02:17.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.904+0000 7fd067a3c700 1 -- 172.21.15.6:0/2471370677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0600ce870 con 0x7fd0600caeb0 2023-11-28T13:02:17.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.904+0000 7fd051ffb700 1 -- 172.21.15.6:0/2471370677 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c007310 con 0x7fd0600caeb0 2023-11-28T13:02:17.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.904+0000 7fd067a3c700 1 -- 172.21.15.6:0/2471370677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0600ce590 con 0x7fd0600caeb0 2023-11-28T13:02:17.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.905+0000 7fd067a3c700 1 -- 172.21.15.6:0/2471370677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0600cc850 con 0x7fd0600caeb0 2023-11-28T13:02:17.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.905+0000 7fd051ffb700 1 -- 172.21.15.6:0/2471370677 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd05c01a020 con 0x7fd0600caeb0 2023-11-28T13:02:17.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.905+0000 7fd051ffb700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0440682f0 0x7fd04406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:17.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.905+0000 7fd051ffb700 1 -- 172.21.15.6:0/2471370677 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fd05c085b80 con 0x7fd0600caeb0 2023-11-28T13:02:17.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.906+0000 7fd064fd7700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0440682f0 0x7fd04406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:17.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.906+0000 7fd064fd7700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0440682f0 0x7fd04406a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd0600c62a0 tx=0x7fd054008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:17.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:17.910+0000 7fd051ffb700 1 -- 172.21.15.6:0/2471370677 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd05c052170 con 0x7fd0600caeb0 2023-11-28T13:02:18.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.149+0000 7fd067a3c700 1 -- 172.21.15.6:0/2471370677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0600675a0 con 0x7fd0600caeb0 2023-11-28T13:02:18.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.149+0000 7fd051ffb700 1 -- 172.21.15.6:0/2471370677 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4642 (secure 0 0 0) 0x7fd05c055790 con 0x7fd0600caeb0 2023-11-28T13:02:18.151 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:18.151 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4026920958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4026920958},{"type":"v1","addr":"172.21.15.61:6833","nonce":4026920958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2212733196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2212733196},{"type":"v1","addr":"172.21.15.61:6835","nonce":2212733196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1674709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1674709721},{"type":"v1","addr":"172.21.15.6:6836","nonce":1674709721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:42.595217+0000","modified":"2023-11-28T13:01:45.602537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/2789212407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":2789212407},{"type":"v1","addr":"172.21.15.6:6837","nonce":2789212407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:02:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.151+0000 7fd067a3c700 1 -- 172.21.15.6:0/2471370677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0440682f0 msgr2=0x7fd04406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.151+0000 7fd067a3c700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0440682f0 0x7fd04406a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd0600c62a0 tx=0x7fd054008040 comp rx=0 tx=0).stop 2023-11-28T13:02:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.151+0000 7fd067a3c700 1 -- 172.21.15.6:0/2471370677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 msgr2=0x7fd0600c3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.151+0000 7fd067a3c700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 0x7fd0600c3b60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0011e0 tx=0x7fd05c00cae0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.151+0000 7fd067a3c700 1 -- 172.21.15.6:0/2471370677 shutdown_connections 2023-11-28T13:02:18.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.151+0000 7fd067a3c700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd060161090 0x7fd0601600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.151+0000 7fd067a3c700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0600caeb0 0x7fd0600c3b60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.151+0000 7fd067a3c700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0440682f0 0x7fd04406a7a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.151+0000 7fd067a3c700 1 --2- 172.21.15.6:0/2471370677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd06015c990 0x7fd06014ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.151+0000 7fd067a3c700 1 -- 172.21.15.6:0/2471370677 >> 172.21.15.6:0/2471370677 conn(0x7fd0600b1650 msgr2=0x7fd0600c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.152+0000 7fd067a3c700 1 -- 172.21.15.6:0/2471370677 shutdown_connections 2023-11-28T13:02:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.152+0000 7fd067a3c700 1 -- 172.21.15.6:0/2471370677 wait complete. 2023-11-28T13:02:18.154 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2023-11-28T13:02:18.165 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:18.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.287+0000 7f9138e33700 1 Processor -- start 2023-11-28T13:02:18.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.288+0000 7f9138e33700 1 -- start start 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.288+0000 7f9138e33700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f913415c990 0x7f91340cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.288+0000 7f9138e33700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9134161090 0x7f91340ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f9138e33700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91340ce340 con 0x7f91340caeb0 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f9138e33700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91340cebe0 con 0x7f9134161090 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f9138e33700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91340cb3f0 con 0x7f913415c990 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f9132d9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f91340caeb0 0x7f91340cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48376/0 (socket says 172.21.15.6:48376) 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f9132d9d700 1 -- 172.21.15.6:0/263455945 learned_addr learned my addr 172.21.15.6:0/263455945 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f913259c700 1 --2- 172.21.15.6:0/263455945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f913415c990 0x7f91340cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f9131d9b700 1 --2- 172.21.15.6:0/263455945 >> v2:172.21.15.61:3300/0 conn(0x7f9134161090 0x7f91340ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f913159a700 1 -- 172.21.15.6:0/263455945 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284327937 0 0) 0x7f91340ce340 con 0x7f91340caeb0 2023-11-28T13:02:18.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f913159a700 1 -- 172.21.15.6:0/263455945 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f911c0035b0 con 0x7f91340caeb0 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f913159a700 1 -- 172.21.15.6:0/263455945 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1178605070 0 0) 0x7f911c0035b0 con 0x7f91340caeb0 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f913259c700 1 -- 172.21.15.6:0/263455945 >> v2:172.21.15.61:3300/0 conn(0x7f9134161090 msgr2=0x7f91340ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f913259c700 1 --2- 172.21.15.6:0/263455945 >> v2:172.21.15.61:3300/0 conn(0x7f9134161090 0x7f91340ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f913259c700 1 -- 172.21.15.6:0/263455945 >> v1:172.21.15.6:6789/0 conn(0x7f91340caeb0 legacy=0x7f91340cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f913259c700 1 -- 172.21.15.6:0/263455945 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340ce590 con 0x7f913415c990 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f9131d9b700 1 --2- 172.21.15.6:0/263455945 >> v2:172.21.15.61:3300/0 conn(0x7f9134161090 0x7f91340ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.289+0000 7f913259c700 1 --2- 172.21.15.6:0/263455945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f913415c990 0x7f91340cf070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9128009840 tx=0x7f9128009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a26cf984b741c98 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f913159a700 1 -- 172.21.15.6:0/263455945 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c0019f0 con 0x7f913415c990 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f913159a700 1 -- 172.21.15.6:0/263455945 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9128004030 con 0x7f913415c990 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f913159a700 1 -- 172.21.15.6:0/263455945 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280043c0 con 0x7f913415c990 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f9138e33700 1 -- 172.21.15.6:0/263455945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f913415c990 msgr2=0x7f91340cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f9138e33700 1 --2- 172.21.15.6:0/263455945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f913415c990 0x7f91340cf070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9128009840 tx=0x7f9128009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:18.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f9138e33700 1 -- 172.21.15.6:0/263455945 shutdown_connections 2023-11-28T13:02:18.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f9138e33700 1 --2- 172.21.15.6:0/263455945 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f913415c990 0x7f91340cf070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f9138e33700 1 --2- 172.21.15.6:0/263455945 >> v2:172.21.15.61:3300/0 conn(0x7f9134161090 0x7f91340ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f9138e33700 1 -- 172.21.15.6:0/263455945 >> 172.21.15.6:0/263455945 conn(0x7f91340b1650 msgr2=0x7f91340b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:18.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f9138e33700 1 -- 172.21.15.6:0/263455945 shutdown_connections 2023-11-28T13:02:18.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.290+0000 7f9138e33700 1 -- 172.21.15.6:0/263455945 wait complete. 2023-11-28T13:02:18.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.292+0000 7f9138e33700 1 Processor -- start 2023-11-28T13:02:18.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.292+0000 7f9138e33700 1 -- start start 2023-11-28T13:02:18.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.292+0000 7f9138e33700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340caeb0 0x7f913414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9138e33700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f913415c990 0x7f91340c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9138e33700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9134161090 0x7f91340c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9138e33700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913414d3e0 con 0x7f91340caeb0 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9138e33700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91341e7050 con 0x7f9134161090 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9138e33700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91341e7330 con 0x7f913415c990 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f913259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340caeb0 0x7f913414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f913259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340caeb0 0x7f913414cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32826/0 (socket says 172.21.15.6:32826) 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f913259c700 1 -- 172.21.15.6:0/3543066128 learned_addr learned my addr 172.21.15.6:0/3543066128 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f913259c700 1 -- 172.21.15.6:0/3543066128 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f913415c990 msgr2=0x7f91340c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f913259c700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f913415c990 0x7f91340c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9132d9d700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9134161090 0x7f91340c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f913259c700 1 -- 172.21.15.6:0/3543066128 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9134161090 msgr2=0x7f91340c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f913259c700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9134161090 0x7f91340c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f913259c700 1 -- 172.21.15.6:0/3543066128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91240094a0 con 0x7f91340caeb0 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f913259c700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340caeb0 0x7f913414cea0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f91280090b0 tx=0x7f9128004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9122ffd700 1 -- 172.21.15.6:0/3543066128 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128003e20 con 0x7f91340caeb0 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9122ffd700 1 -- 172.21.15.6:0/3543066128 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912801f070 con 0x7f91340caeb0 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9138e33700 1 -- 172.21.15.6:0/3543066128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91280094a0 con 0x7f91340caeb0 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9122ffd700 1 -- 172.21.15.6:0/3543066128 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9128011870 con 0x7f91340caeb0 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.293+0000 7f9138e33700 1 -- 172.21.15.6:0/3543066128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91340cf5b0 con 0x7f91340caeb0 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.294+0000 7f9138e33700 1 -- 172.21.15.6:0/3543066128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91340ce870 con 0x7f91340caeb0 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.295+0000 7f9122ffd700 1 -- 172.21.15.6:0/3543066128 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9128017020 con 0x7f91340caeb0 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.295+0000 7f9122ffd700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91140682f0 0x7f911406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.295+0000 7f9122ffd700 1 -- 172.21.15.6:0/3543066128 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9128021070 con 0x7f91340caeb0 2023-11-28T13:02:18.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.295+0000 7f9131d9b700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91140682f0 0x7f911406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.296+0000 7f9131d9b700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91140682f0 0x7f911406a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9124006f10 tx=0x7f9124006e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:18.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.300+0000 7f9122ffd700 1 -- 172.21.15.6:0/3543066128 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9128053e10 con 0x7f91340caeb0 2023-11-28T13:02:18.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.535+0000 7f9138e33700 1 -- 172.21.15.6:0/3543066128 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91340cf740 con 0x7f91340caeb0 2023-11-28T13:02:18.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.536+0000 7f9122ffd700 1 -- 172.21.15.6:0/3543066128 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4642 (secure 0 0 0) 0x7f9128058e60 con 0x7f91340caeb0 2023-11-28T13:02:18.537 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:18.538 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4026920958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4026920958},{"type":"v1","addr":"172.21.15.61:6833","nonce":4026920958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2212733196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2212733196},{"type":"v1","addr":"172.21.15.61:6835","nonce":2212733196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1674709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1674709721},{"type":"v1","addr":"172.21.15.6:6836","nonce":1674709721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:01:42.595217+0000","modified":"2023-11-28T13:01:45.602537+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5505":{"gid":5505,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/2789212407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":2789212407},{"type":"v1","addr":"172.21.15.6:6837","nonce":2789212407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T13:02:18.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 -- 172.21.15.6:0/3543066128 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91140682f0 msgr2=0x7f911406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91140682f0 0x7f911406a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9124006f10 tx=0x7f9124006e60 comp rx=0 tx=0).stop 2023-11-28T13:02:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 -- 172.21.15.6:0/3543066128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340caeb0 msgr2=0x7f913414cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340caeb0 0x7f913414cea0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f91280090b0 tx=0x7f9128004750 comp rx=0 tx=0).stop 2023-11-28T13:02:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 -- 172.21.15.6:0/3543066128 shutdown_connections 2023-11-28T13:02:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9134161090 0x7f91340c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91340caeb0 0x7f913414cea0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91140682f0 0x7f911406a7a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 --2- 172.21.15.6:0/3543066128 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f913415c990 0x7f91340c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 -- 172.21.15.6:0/3543066128 >> 172.21.15.6:0/3543066128 conn(0x7f91340b1650 msgr2=0x7f913406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:18.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 -- 172.21.15.6:0/3543066128 shutdown_connections 2023-11-28T13:02:18.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.538+0000 7f9138e33700 1 -- 172.21.15.6:0/3543066128 wait complete. 2023-11-28T13:02:18.541 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 16 2023-11-28T13:02:18.551 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:18.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.674+0000 7f0e0e580700 1 Processor -- start 2023-11-28T13:02:18.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.674+0000 7f0e0e580700 1 -- start start 2023-11-28T13:02:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.675+0000 7f0e0e580700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e080cb470 0x7f0e080ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.675+0000 7f0e0e580700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0806c140 con 0x7f0e080cb0a0 2023-11-28T13:02:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.675+0000 7f0e0e580700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e080c9ab0 con 0x7f0e0806bb80 2023-11-28T13:02:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.675+0000 7f0e0e580700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080caad0 con 0x7f0e080cb470 2023-11-28T13:02:18.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.675+0000 7f0e07fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f0e080cb0a0 0x7f0e0815a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48386/0 (socket says 172.21.15.6:48386) 2023-11-28T13:02:18.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.675+0000 7f0e07fff700 1 -- 172.21.15.6:0/1339567190 learned_addr learned my addr 172.21.15.6:0/1339567190 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:18.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e077fe700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e080cb470 0x7f0e080ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17723227 0 0) 0x7f0e0806c140 con 0x7f0e080cb0a0 2023-11-28T13:02:18.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0df40035b0 con 0x7f0e080cb0a0 2023-11-28T13:02:18.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4104741128 0 0) 0x7f0df40035b0 con 0x7f0e080cb0a0 2023-11-28T13:02:18.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e080cb470 msgr2=0x7f0e080ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:18.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e080cb470 0x7f0e080ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 >> v1:172.21.15.61:6789/0 conn(0x7f0e0806bb80 legacy=0x7f0e0814fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e080c9d90 con 0x7f0e080cb0a0 2023-11-28T13:02:18.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f0df8001670 con 0x7f0e080cb0a0 2023-11-28T13:02:18.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 >> v1:172.21.15.6:6789/0 conn(0x7f0e080cb0a0 legacy=0x7f0e0815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e080cb470 0x7f0df4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0df4003fb0 0x7f0df4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0df4004d90 0x7f0df4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df400bba0 con 0x7f0df4004d90 2023-11-28T13:02:18.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df400bce0 con 0x7f0df4003fb0 2023-11-28T13:02:18.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df400e0b0 con 0x7f0e080cb470 2023-11-28T13:02:18.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e0cb1d700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0df4004d90 0x7f0df4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e07fff700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e080cb470 0x7f0df4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e077fe700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0df4003fb0 0x7f0df4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e0cb1d700 1 -- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e080cb470 msgr2=0x7f0df4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:18.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e0cb1d700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0e080cb470 0x7f0df4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e0cb1d700 1 -- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0df4003fb0 msgr2=0x7f0df4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:18.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e0cb1d700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0df4003fb0 0x7f0df4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.676+0000 7f0e0cb1d700 1 -- 172.21.15.6:0/1339567190 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df40035b0 con 0x7f0df4004d90 2023-11-28T13:02:18.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e0cb1d700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0df4004d90 0x7f0df4005200 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000c160 tx=0x7f0df4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cebe410b1e50fa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:18.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00001db0 con 0x7f0df4004d90 2023-11-28T13:02:18.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0000cca0 con 0x7f0df4004d90 2023-11-28T13:02:18.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e06ffd700 1 -- 172.21.15.6:0/1339567190 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00003c40 con 0x7f0df4004d90 2023-11-28T13:02:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e0e580700 1 -- 172.21.15.6:0/1339567190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0df4004d90 msgr2=0x7f0df4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e0e580700 1 --2- 172.21.15.6:0/1339567190 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0df4004d90 0x7f0df4005200 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000c160 tx=0x7f0df4005880 comp rx=0 tx=0).stop 2023-11-28T13:02:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e0cb1d700 1 -- 172.21.15.6:0/1339567190 reap_dead start 2023-11-28T13:02:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e0e580700 1 -- 172.21.15.6:0/1339567190 shutdown_connections 2023-11-28T13:02:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e0e580700 1 -- 172.21.15.6:0/1339567190 >> 172.21.15.6:0/1339567190 conn(0x7f0e080b0dd0 msgr2=0x7f0e080b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e0e580700 1 -- 172.21.15.6:0/1339567190 shutdown_connections 2023-11-28T13:02:18.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.677+0000 7f0e0e580700 1 -- 172.21.15.6:0/1339567190 wait complete. 2023-11-28T13:02:18.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.678+0000 7f0e0e580700 1 Processor -- start 2023-11-28T13:02:18.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.679+0000 7f0e0e580700 1 -- start start 2023-11-28T13:02:18.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.679+0000 7f0e0e580700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0df4004d90 0x7f0e080afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.679+0000 7f0e0e580700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080c4df0 0x7f0e0814e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.679+0000 7f0e0e580700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e081f0fd0 0x7f0e081f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.679+0000 7f0e0e580700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080c7c10 con 0x7f0e081f0fd0 2023-11-28T13:02:18.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.679+0000 7f0e0e580700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081f1980 con 0x7f0e080c4df0 2023-11-28T13:02:18.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.679+0000 7f0e0e580700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080c5260 con 0x7f0df4004d90 2023-11-28T13:02:18.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0cb1d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e081f0fd0 0x7f0e081f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0cb1d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e081f0fd0 0x7f0e081f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32850/0 (socket says 172.21.15.6:32850) 2023-11-28T13:02:18.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0cb1d700 1 -- 172.21.15.6:0/1434977263 learned_addr learned my addr 172.21.15.6:0/1434977263 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:18.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e077fe700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080c4df0 0x7f0e0814e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e07fff700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0df4004d90 0x7f0e080afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0cb1d700 1 -- 172.21.15.6:0/1434977263 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0df4004d90 msgr2=0x7f0e080afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0cb1d700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0df4004d90 0x7f0e080afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0cb1d700 1 -- 172.21.15.6:0/1434977263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080c4df0 msgr2=0x7f0e0814e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0cb1d700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080c4df0 0x7f0e0814e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0cb1d700 1 -- 172.21.15.6:0/1434977263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e081505b0 con 0x7f0e081f0fd0 2023-11-28T13:02:18.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e07fff700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0df4004d90 0x7f0e080afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:18.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e077fe700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080c4df0 0x7f0e0814e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:18.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0cb1d700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e081f0fd0 0x7f0e081f1440 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0e00001220 tx=0x7f0e000014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:18.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e04ff9700 1 -- 172.21.15.6:0/1434977263 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00003c10 con 0x7f0e081f0fd0 2023-11-28T13:02:18.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e04ff9700 1 -- 172.21.15.6:0/1434977263 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e00003d70 con 0x7f0e081f0fd0 2023-11-28T13:02:18.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0e580700 1 -- 172.21.15.6:0/1434977263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e080ca070 con 0x7f0e081f0fd0 2023-11-28T13:02:18.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e04ff9700 1 -- 172.21.15.6:0/1434977263 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00005310 con 0x7f0e081f0fd0 2023-11-28T13:02:18.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.680+0000 7f0e0e580700 1 -- 172.21.15.6:0/1434977263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e08150740 con 0x7f0e081f0fd0 2023-11-28T13:02:18.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.681+0000 7f0e0e580700 1 -- 172.21.15.6:0/1434977263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e080675a0 con 0x7f0e081f0fd0 2023-11-28T13:02:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.682+0000 7f0e04ff9700 1 -- 172.21.15.6:0/1434977263 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0e0002e070 con 0x7f0e081f0fd0 2023-11-28T13:02:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.682+0000 7f0e04ff9700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0de80682f0 0x7f0de806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.682+0000 7f0e04ff9700 1 -- 172.21.15.6:0/1434977263 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f0e00086d20 con 0x7f0e081f0fd0 2023-11-28T13:02:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.682+0000 7f0e07fff700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0de80682f0 0x7f0de806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.682+0000 7f0e07fff700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0de80682f0 0x7f0de806a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0df8007e50 tx=0x7f0df8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:18.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.686+0000 7f0e04ff9700 1 -- 172.21.15.6:0/1434977263 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0e00053400 con 0x7f0e081f0fd0 2023-11-28T13:02:18.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.880+0000 7f0e0e580700 1 -- 172.21.15.6:0/1434977263 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e080b5260 con 0x7f0e081f0fd0 2023-11-28T13:02:18.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.882+0000 7f0e04ff9700 1 -- 172.21.15.6:0/1434977263 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18066 (secure 0 0 0) 0x7f0e00056a20 con 0x7f0e081f0fd0 2023-11-28T13:02:18.883 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:18.883 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":42,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:11.026507+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:01:41.144348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T13:01:41.554568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.18","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.883+0000 7f0e0e580700 1 -- 172.21.15.6:0/1434977263 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0de80682f0 msgr2=0x7f0de806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.883+0000 7f0e0e580700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0de80682f0 0x7f0de806a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0df8007e50 tx=0x7f0df8009040 comp rx=0 tx=0).stop 2023-11-28T13:02:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.883+0000 7f0e0e580700 1 -- 172.21.15.6:0/1434977263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e081f0fd0 msgr2=0x7f0e081f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.883+0000 7f0e0e580700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e081f0fd0 0x7f0e081f1440 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0e00001220 tx=0x7f0e000014a0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.884+0000 7f0e0e580700 1 -- 172.21.15.6:0/1434977263 shutdown_connections 2023-11-28T13:02:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.884+0000 7f0e0e580700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e081f0fd0 0x7f0e081f1440 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.884+0000 7f0e0e580700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0de80682f0 0x7f0de806a7a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.884+0000 7f0e0e580700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0df4004d90 0x7f0e080afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.884+0000 7f0e0e580700 1 --2- 172.21.15.6:0/1434977263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e080c4df0 0x7f0e0814e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:18.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.884+0000 7f0e0e580700 1 -- 172.21.15.6:0/1434977263 >> 172.21.15.6:0/1434977263 conn(0x7f0e080b0dd0 msgr2=0x7f0e08158200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.884+0000 7f0e0e580700 1 -- 172.21.15.6:0/1434977263 shutdown_connections 2023-11-28T13:02:18.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:18.884+0000 7f0e0e580700 1 -- 172.21.15.6:0/1434977263 wait complete. 2023-11-28T13:02:18.897 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:02:19.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.023+0000 7f5303698700 1 Processor -- start 2023-11-28T13:02:19.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.023+0000 7f5303698700 1 -- start start 2023-11-28T13:02:19.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.024+0000 7f5303698700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f52fc0cc310 0x7f52fc0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:19.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.024+0000 7f5303698700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc0ce340 con 0x7f52fc15c990 2023-11-28T13:02:19.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.024+0000 7f5303698700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc0cebe0 con 0x7f52fc161090 2023-11-28T13:02:19.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.024+0000 7f5303698700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc0cf780 con 0x7f52fc0cc310 2023-11-28T13:02:19.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.024+0000 7f5301434700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f52fc15c990 0x7f52fc0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48406/0 (socket says 172.21.15.6:48406) 2023-11-28T13:02:19.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.024+0000 7f5301434700 1 -- 172.21.15.6:0/1139475045 learned_addr learned my addr 172.21.15.6:0/1139475045 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:19.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.024+0000 7f5301c35700 1 --2- 172.21.15.6:0/1139475045 >> v2:172.21.15.61:3300/0 conn(0x7f52fc0cc310 0x7f52fc0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.024+0000 7f52effff700 1 -- 172.21.15.6:0/1139475045 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419998320 0 0) 0x7f52fc0ce340 con 0x7f52fc15c990 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.024+0000 7f52effff700 1 -- 172.21.15.6:0/1139475045 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e80035b0 con 0x7f52fc15c990 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.025+0000 7f52effff700 1 -- 172.21.15.6:0/1139475045 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458816614 0 0) 0x7f52fc0cebe0 con 0x7f52fc161090 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.025+0000 7f52effff700 1 -- 172.21.15.6:0/1139475045 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52fc0ce340 con 0x7f52fc161090 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.025+0000 7f5301c35700 1 -- 172.21.15.6:0/1139475045 >> v1:172.21.15.61:6789/0 conn(0x7f52fc161090 legacy=0x7f52fc0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.025+0000 7f5301c35700 1 -- 172.21.15.6:0/1139475045 >> v1:172.21.15.6:6789/0 conn(0x7f52fc15c990 legacy=0x7f52fc0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.025+0000 7f52effff700 1 -- 172.21.15.6:0/1139475045 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3047738996 0 0) 0x7f52e80035b0 con 0x7f52fc15c990 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.025+0000 7f5301c35700 1 -- 172.21.15.6:0/1139475045 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc0cccb0 con 0x7f52fc0cc310 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.025+0000 7f5301c35700 1 --2- 172.21.15.6:0/1139475045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52fc0cc310 0x7f52fc0cc770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009d90 tx=0x7f52f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e2e991f71d5befe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.025+0000 7f52effff700 1 -- 172.21.15.6:0/1139475045 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400e070 con 0x7f52fc0cc310 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.025+0000 7f52effff700 1 -- 172.21.15.6:0/1139475045 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f4003a20 con 0x7f52fc0cc310 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.025+0000 7f52effff700 1 -- 172.21.15.6:0/1139475045 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0001670 con 0x7f52fc0cc310 2023-11-28T13:02:19.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.026+0000 7f5303698700 1 -- 172.21.15.6:0/1139475045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52fc0cc310 msgr2=0x7f52fc0cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:19.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.026+0000 7f5303698700 1 --2- 172.21.15.6:0/1139475045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52fc0cc310 0x7f52fc0cc770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009d90 tx=0x7f52f400b330 comp rx=0 tx=0).stop 2023-11-28T13:02:19.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.026+0000 7f5303698700 1 -- 172.21.15.6:0/1139475045 shutdown_connections 2023-11-28T13:02:19.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.026+0000 7f5303698700 1 --2- 172.21.15.6:0/1139475045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52fc0cc310 0x7f52fc0cc770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:19.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.026+0000 7f5303698700 1 -- 172.21.15.6:0/1139475045 >> 172.21.15.6:0/1139475045 conn(0x7f52fc0b1650 msgr2=0x7f52fc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:19.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.026+0000 7f5303698700 1 -- 172.21.15.6:0/1139475045 shutdown_connections 2023-11-28T13:02:19.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.026+0000 7f5303698700 1 -- 172.21.15.6:0/1139475045 wait complete. 2023-11-28T13:02:19.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.027+0000 7f5303698700 1 Processor -- start 2023-11-28T13:02:19.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.027+0000 7f5303698700 1 -- start start 2023-11-28T13:02:19.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5303698700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52fc0cc310 0x7f52fc15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:19.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5303698700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52fc15c990 0x7f52fc0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:19.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5303698700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52fc161090 0x7f52fc0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:19.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5303698700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc15fbb0 con 0x7f52fc161090 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5303698700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1e8950 con 0x7f52fc15c990 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5303698700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc15f160 con 0x7f52fc0cc310 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5301c35700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52fc161090 0x7f52fc0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5301c35700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52fc161090 0x7f52fc0b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32862/0 (socket says 172.21.15.6:32862) 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5301c35700 1 -- 172.21.15.6:0/2774527337 learned_addr learned my addr 172.21.15.6:0/2774527337 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5301434700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52fc0cc310 0x7f52fc15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5300c33700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52fc15c990 0x7f52fc0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5301c35700 1 -- 172.21.15.6:0/2774527337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52fc0cc310 msgr2=0x7f52fc15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5301c35700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52fc0cc310 0x7f52fc15cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5301c35700 1 -- 172.21.15.6:0/2774527337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52fc15c990 msgr2=0x7f52fc0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5301c35700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52fc15c990 0x7f52fc0c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:19.032 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.028+0000 7f5301c35700 1 -- 172.21.15.6:0/2774527337 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4009a70 con 0x7f52fc161090 2023-11-28T13:02:19.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.029+0000 7f5301c35700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52fc161090 0x7f52fc0b39a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f52f400b680 tx=0x7f52f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:19.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.030+0000 7f52edffb700 1 -- 172.21.15.6:0/2774527337 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4004440 con 0x7f52fc161090 2023-11-28T13:02:19.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.030+0000 7f52edffb700 1 -- 172.21.15.6:0/2774527337 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f40045a0 con 0x7f52fc161090 2023-11-28T13:02:19.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.030+0000 7f5303698700 1 -- 172.21.15.6:0/2774527337 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc0ce590 con 0x7f52fc161090 2023-11-28T13:02:19.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.030+0000 7f52edffb700 1 -- 172.21.15.6:0/2774527337 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4004950 con 0x7f52fc161090 2023-11-28T13:02:19.033 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.030+0000 7f5303698700 1 -- 172.21.15.6:0/2774527337 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc0cb3f0 con 0x7f52fc161090 2023-11-28T13:02:19.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.031+0000 7f5303698700 1 -- 172.21.15.6:0/2774527337 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc0ce870 con 0x7f52fc161090 2023-11-28T13:02:19.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.031+0000 7f52edffb700 1 -- 172.21.15.6:0/2774527337 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f52f4018070 con 0x7f52fc161090 2023-11-28T13:02:19.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.032+0000 7f52edffb700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52e00682f0 0x7f52e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:19.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.032+0000 7f52edffb700 1 -- 172.21.15.6:0/2774527337 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f52f40878a0 con 0x7f52fc161090 2023-11-28T13:02:19.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.032+0000 7f5301434700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52e00682f0 0x7f52e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:19.034 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.032+0000 7f5301434700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52e00682f0 0x7f52e006a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f52f0000fd0 tx=0x7f52f0000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:19.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.038+0000 7f52edffb700 1 -- 172.21.15.6:0/2774527337 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52f4053f80 con 0x7f52fc161090 2023-11-28T13:02:19.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:19.276+0000 7f5303698700 1 -- 172.21.15.6:0/2774527337 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f52fc067600 con 0x7f52fc161090 2023-11-28T13:02:20.033 INFO:tasks.ceph.mon.a.smithi006.stderr:2023-11-28T13:02:20.032+0000 7fa6275a1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:02:20.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.036+0000 7f52edffb700 1 -- 172.21.15.6:0/2774527337 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f52f40575a0 con 0x7f52fc161090 2023-11-28T13:02:20.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.039+0000 7f5303698700 1 -- 172.21.15.6:0/2774527337 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52e00682f0 msgr2=0x7f52e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:20.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.039+0000 7f5303698700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52e00682f0 0x7f52e006a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f52f0000fd0 tx=0x7f52f0000ed0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.039+0000 7f5303698700 1 -- 172.21.15.6:0/2774527337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52fc161090 msgr2=0x7f52fc0b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:20.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.039+0000 7f5303698700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52fc161090 0x7f52fc0b39a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f52f400b680 tx=0x7f52f4004ad0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.039+0000 7f5303698700 1 -- 172.21.15.6:0/2774527337 shutdown_connections 2023-11-28T13:02:20.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.039+0000 7f5303698700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52fc161090 0x7f52fc0b39a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.039+0000 7f5303698700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52e00682f0 0x7f52e006a7a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.039+0000 7f5303698700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f52fc0cc310 0x7f52fc15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.039+0000 7f5303698700 1 --2- 172.21.15.6:0/2774527337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52fc15c990 0x7f52fc0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.039+0000 7f5303698700 1 -- 172.21.15.6:0/2774527337 >> 172.21.15.6:0/2774527337 conn(0x7f52fc0b1650 msgr2=0x7f52fc0ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:20.041 INFO:tasks.ceph.mds.c.smithi006.stderr: -5722> 2023-11-28T13:01:46.159+0000 7f6c26fd7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:02:20.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.042+0000 7f5303698700 1 -- 172.21.15.6:0/2774527337 shutdown_connections 2023-11-28T13:02:20.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.042+0000 7f5303698700 1 -- 172.21.15.6:0/2774527337 wait complete. 2023-11-28T13:02:20.044 INFO:teuthology.orchestra.run.smithi006.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:02:20.059 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:02:20.076 INFO:tasks.ceph.mds.c.smithi006.stdout:starting mds.c at 2023-11-28T13:02:20.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.192+0000 7fceb3c3f700 1 Processor -- start 2023-11-28T13:02:20.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.192+0000 7fceb3c3f700 1 -- start start 2023-11-28T13:02:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.193+0000 7fceb3c3f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fceac0b1630 0x7fceac0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.193+0000 7fceb3c3f700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac0ce550 con 0x7fceac06afc0 2023-11-28T13:02:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.193+0000 7fceb3c3f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac0ce830 con 0x7fceac0b1a00 2023-11-28T13:02:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.193+0000 7fceb3c3f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceac0ceb10 con 0x7fceac0b1630 2023-11-28T13:02:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.193+0000 7fceb21dc700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fceac06afc0 0x7fceac06b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48436/0 (socket says 172.21.15.6:48436) 2023-11-28T13:02:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.193+0000 7fceb21dc700 1 -- 172.21.15.6:0/4263043623 learned_addr learned my addr 172.21.15.6:0/4263043623 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.193+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534145153 0 0) 0x7fceac0ce550 con 0x7fceac06afc0 2023-11-28T13:02:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.193+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce980035b0 con 0x7fceac06afc0 2023-11-28T13:02:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.193+0000 7fceb19db700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fceac0b1630 0x7fceac0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:20.197 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3955616177 0 0) 0x7fce980035b0 con 0x7fceac06afc0 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fceac0b1630 msgr2=0x7fceac0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fceac0b1630 0x7fceac0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 >> v1:172.21.15.61:6789/0 conn(0x7fceac0b1a00 legacy=0x7fceac0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac0cedf0 con 0x7fceac06afc0 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fcea40019f0 con 0x7fceac06afc0 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 >> v1:172.21.15.6:6789/0 conn(0x7fceac06afc0 legacy=0x7fceac06b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fceac0b1630 0x7fce98003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98003fb0 0x7fce98004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce98004d90 0x7fce98005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac0ceb10 con 0x7fce98003fb0 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9800baf0 con 0x7fce98004d90 2023-11-28T13:02:20.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9800bcb0 con 0x7fceac0b1630 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb11da700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98003fb0 0x7fce98004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb19db700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fceac0b1630 0x7fce98003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb21dc700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce98004d90 0x7fce98005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb11da700 1 -- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fceac0b1630 msgr2=0x7fce98003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb11da700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fceac0b1630 0x7fce98003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb11da700 1 -- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce98004d90 msgr2=0x7fce98005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb11da700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fce98004d90 0x7fce98005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb11da700 1 -- 172.21.15.6:0/4263043623 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce980035b0 con 0x7fce98003fb0 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.194+0000 7fceb11da700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98003fb0 0x7fce98004850 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcea8012da0 tx=0x7fce98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fe6c984f1fe051 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.195+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8017070 con 0x7fce98003fb0 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.195+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea8012e90 con 0x7fce98003fb0 2023-11-28T13:02:20.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.195+0000 7fceb09d9700 1 -- 172.21.15.6:0/4263043623 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8012910 con 0x7fce98003fb0 2023-11-28T13:02:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.195+0000 7fceb3c3f700 1 -- 172.21.15.6:0/4263043623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98003fb0 msgr2=0x7fce98004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.195+0000 7fceb3c3f700 1 --2- 172.21.15.6:0/4263043623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fce98003fb0 0x7fce98004850 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcea8012da0 tx=0x7fce98005880 comp rx=0 tx=0).stop 2023-11-28T13:02:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.195+0000 7fceb21dc700 1 -- 172.21.15.6:0/4263043623 reap_dead start 2023-11-28T13:02:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.195+0000 7fceb3c3f700 1 -- 172.21.15.6:0/4263043623 shutdown_connections 2023-11-28T13:02:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.195+0000 7fceb3c3f700 1 -- 172.21.15.6:0/4263043623 >> 172.21.15.6:0/4263043623 conn(0x7fceac0cc630 msgr2=0x7fceac0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.195+0000 7fceb3c3f700 1 -- 172.21.15.6:0/4263043623 shutdown_connections 2023-11-28T13:02:20.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.195+0000 7fceb3c3f700 1 -- 172.21.15.6:0/4263043623 wait complete. 2023-11-28T13:02:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.196+0000 7fceb3c3f700 1 Processor -- start 2023-11-28T13:02:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.196+0000 7fceb3c3f700 1 -- start start 2023-11-28T13:02:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.197+0000 7fceb3c3f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce98003fb0 0x7fceac15b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.197+0000 7fceb3c3f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fceac0c6400 0x7fceac15f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.197+0000 7fceb3c3f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fceac15caa0 0x7fceac15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.197+0000 7fceb3c3f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac1ee180 con 0x7fceac0c6400 2023-11-28T13:02:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.197+0000 7fceb3c3f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac15bf50 con 0x7fceac15caa0 2023-11-28T13:02:20.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.197+0000 7fceb3c3f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceac0a7f80 con 0x7fce98003fb0 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.197+0000 7fceb11da700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fceac0c6400 0x7fceac15f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.197+0000 7fceb11da700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fceac0c6400 0x7fceac15f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32894/0 (socket says 172.21.15.6:32894) 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.197+0000 7fceb11da700 1 -- 172.21.15.6:0/426729938 learned_addr learned my addr 172.21.15.6:0/426729938 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb19db700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce98003fb0 0x7fceac15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb21dc700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fceac15caa0 0x7fceac15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb11da700 1 -- 172.21.15.6:0/426729938 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce98003fb0 msgr2=0x7fceac15b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb11da700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce98003fb0 0x7fceac15b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb11da700 1 -- 172.21.15.6:0/426729938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fceac15caa0 msgr2=0x7fceac15cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb11da700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fceac15caa0 0x7fceac15cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb11da700 1 -- 172.21.15.6:0/426729938 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac0b1e70 con 0x7fceac0c6400 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb21dc700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fceac15caa0 0x7fceac15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:20.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb11da700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fceac0c6400 0x7fceac15f100 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcea8004000 tx=0x7fcea8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fcea27fc700 1 -- 172.21.15.6:0/426729938 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8020070 con 0x7fceac0c6400 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fcea27fc700 1 -- 172.21.15.6:0/426729938 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea8001d40 con 0x7fceac0c6400 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb3c3f700 1 -- 172.21.15.6:0/426729938 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceac0cf0d0 con 0x7fceac0c6400 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fcea27fc700 1 -- 172.21.15.6:0/426729938 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8003ad0 con 0x7fceac0c6400 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.198+0000 7fceb3c3f700 1 -- 172.21.15.6:0/426729938 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceac0cb5b0 con 0x7fceac0c6400 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.199+0000 7fceb3c3f700 1 -- 172.21.15.6:0/426729938 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceac0a8130 con 0x7fceac0c6400 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.201+0000 7fcea27fc700 1 -- 172.21.15.6:0/426729938 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcea8003c30 con 0x7fceac0c6400 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.201+0000 7fcea27fc700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce900682f0 0x7fce9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.201+0000 7fcea27fc700 1 -- 172.21.15.6:0/426729938 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6526+0+0 (secure 0 0 0) 0x7fcea8023040 con 0x7fceac0c6400 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.201+0000 7fceb19db700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce900682f0 0x7fce9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:20.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.201+0000 7fceb19db700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce900682f0 0x7fce9006a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fceac15ee20 tx=0x7fce9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:20.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.204+0000 7fcea27fc700 1 -- 172.21.15.6:0/426729938 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcea805b760 con 0x7fceac0c6400 2023-11-28T13:02:20.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:20.449+0000 7fceb3c3f700 1 -- 172.21.15.6:0/426729938 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fceac0a80f0 con 0x7fceac0c6400 2023-11-28T13:02:21.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.041+0000 7fcea27fc700 1 -- 172.21.15.6:0/426729938 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcea8010e60 con 0x7fceac0c6400 2023-11-28T13:02:21.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 -- 172.21.15.6:0/426729938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce900682f0 msgr2=0x7fce9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:21.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce900682f0 0x7fce9006a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fceac15ee20 tx=0x7fce9c008040 comp rx=0 tx=0).stop 2023-11-28T13:02:21.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 -- 172.21.15.6:0/426729938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fceac0c6400 msgr2=0x7fceac15f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:21.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fceac0c6400 0x7fceac15f100 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcea8004000 tx=0x7fcea8001120 comp rx=0 tx=0).stop 2023-11-28T13:02:21.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 -- 172.21.15.6:0/426729938 shutdown_connections 2023-11-28T13:02:21.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fceac15caa0 0x7fceac15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:21.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fce900682f0 0x7fce9006a7a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:21.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fce98003fb0 0x7fceac15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:21.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 --2- 172.21.15.6:0/426729938 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fceac0c6400 0x7fceac15f100 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:21.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 -- 172.21.15.6:0/426729938 >> 172.21.15.6:0/426729938 conn(0x7fceac0cc630 msgr2=0x7fceac06c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:21.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 -- 172.21.15.6:0/426729938 shutdown_connections 2023-11-28T13:02:21.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.045+0000 7fceb3c3f700 1 -- 172.21.15.6:0/426729938 wait complete. 2023-11-28T13:02:21.063 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:02:21.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.248+0000 7f58448a7700 1 Processor -- start 2023-11-28T13:02:21.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.249+0000 7f58448a7700 1 -- start start 2023-11-28T13:02:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.249+0000 7f58448a7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5840159390 0x7f584015d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.249+0000 7f58448a7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58400cdf80 0x7f58400ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.249+0000 7f58448a7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58400cee50 con 0x7f58400cea10 2023-11-28T13:02:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.249+0000 7f58448a7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5840159760 con 0x7f5840159390 2023-11-28T13:02:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f58448a7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584006c3a0 con 0x7f58400cdf80 2023-11-28T13:02:21.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583e59c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5840159390 0x7f584015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:21.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583dd9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58400cdf80 0x7f58400ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:21.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583dd9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58400cdf80 0x7f58400ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:50838/0 (socket says 172.21.15.6:50838) 2023-11-28T13:02:21.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583e59c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5840159390 0x7f584015d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:36620/0 (socket says 172.21.15.6:36620) 2023-11-28T13:02:21.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583dd9b700 1 -- 172.21.15.6:0/1105307652 learned_addr learned my addr 172.21.15.6:0/1105307652 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:21.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583ed9d700 1 --1- 172.21.15.6:0/1105307652 >> v1:172.21.15.61:6789/0 conn(0x7f58400cea10 0x7f58400b1530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:38488/0 (socket says 172.21.15.6:38488) 2023-11-28T13:02:21.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583d59a700 1 -- 172.21.15.6:0/1105307652 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279505175 0 0) 0x7f58400cee50 con 0x7f58400cea10 2023-11-28T13:02:21.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583d59a700 1 -- 172.21.15.6:0/1105307652 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58280035b0 con 0x7f58400cea10 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583dd9b700 1 -- 172.21.15.6:0/1105307652 >> v2:172.21.15.61:3300/0 conn(0x7f5840159390 msgr2=0x7f584015d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583dd9b700 1 --2- 172.21.15.6:0/1105307652 >> v2:172.21.15.61:3300/0 conn(0x7f5840159390 0x7f584015d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583dd9b700 1 -- 172.21.15.6:0/1105307652 >> v1:172.21.15.61:6789/0 conn(0x7f58400cea10 legacy=0x7f58400b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583dd9b700 1 -- 172.21.15.6:0/1105307652 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584015dd50 con 0x7f58400cdf80 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.250+0000 7f583e59c700 1 --2- 172.21.15.6:0/1105307652 >> v2:172.21.15.61:3300/0 conn(0x7f5840159390 0x7f584015d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f583dd9b700 1 --2- 172.21.15.6:0/1105307652 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58400cdf80 0x7f58400ce4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f58300098f0 tx=0x7f5830009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e215fdf49b782da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f583d59a700 1 -- 172.21.15.6:0/1105307652 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830011830 con 0x7f58400cdf80 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f583d59a700 1 -- 172.21.15.6:0/1105307652 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5830011990 con 0x7f58400cdf80 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f583d59a700 1 -- 172.21.15.6:0/1105307652 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830011d00 con 0x7f58400cdf80 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f58448a7700 1 -- 172.21.15.6:0/1105307652 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58400cdf80 msgr2=0x7f58400ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:21.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f58448a7700 1 --2- 172.21.15.6:0/1105307652 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58400cdf80 0x7f58400ce4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f58300098f0 tx=0x7f5830009c00 comp rx=0 tx=0).stop 2023-11-28T13:02:21.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f58448a7700 1 -- 172.21.15.6:0/1105307652 shutdown_connections 2023-11-28T13:02:21.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f58448a7700 1 --2- 172.21.15.6:0/1105307652 >> v2:172.21.15.61:3300/0 conn(0x7f5840159390 0x7f584015d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:21.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f58448a7700 1 --2- 172.21.15.6:0/1105307652 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f58400cdf80 0x7f58400ce4d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:21.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f58448a7700 1 -- 172.21.15.6:0/1105307652 >> 172.21.15.6:0/1105307652 conn(0x7f584006dae0 msgr2=0x7f584006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:21.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f58448a7700 1 -- 172.21.15.6:0/1105307652 shutdown_connections 2023-11-28T13:02:21.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.251+0000 7f58448a7700 1 -- 172.21.15.6:0/1105307652 wait complete. 2023-11-28T13:02:21.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.253+0000 7f58448a7700 1 Processor -- start 2023-11-28T13:02:21.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.253+0000 7f58448a7700 1 -- start start 2023-11-28T13:02:21.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f58448a7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58400cdf80 0x7f5840153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:21.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f58448a7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58400cea10 0x7f58400aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:21.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f58448a7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5840159390 0x7f58400acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:21.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f58448a7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58401f2ed0 con 0x7f58400cea10 2023-11-28T13:02:21.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f58448a7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58400abc70 con 0x7f58400cdf80 2023-11-28T13:02:21.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f58448a7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58400abf50 con 0x7f5840159390 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58400cea10 0x7f58400aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583dd9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58400cea10 0x7f58400aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32910/0 (socket says 172.21.15.6:32910) 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583dd9b700 1 -- 172.21.15.6:0/4156714785 learned_addr learned my addr 172.21.15.6:0/4156714785 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583ed9d700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5840159390 0x7f58400acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583e59c700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58400cdf80 0x7f5840153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583dd9b700 1 -- 172.21.15.6:0/4156714785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5840159390 msgr2=0x7f58400acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583dd9b700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5840159390 0x7f58400acf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583dd9b700 1 -- 172.21.15.6:0/4156714785 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58400cdf80 msgr2=0x7f5840153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583dd9b700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58400cdf80 0x7f5840153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583dd9b700 1 -- 172.21.15.6:0/4156714785 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340094a0 con 0x7f58400cea10 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583ed9d700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5840159390 0x7f58400acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583e59c700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58400cdf80 0x7f5840153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f583dd9b700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58400cea10 0x7f58400aca30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5830009c40 tx=0x7f5830004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:21.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f582effd700 1 -- 172.21.15.6:0/4156714785 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300044d0 con 0x7f58400cea10 2023-11-28T13:02:21.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f582effd700 1 -- 172.21.15.6:0/4156714785 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583001e070 con 0x7f58400cea10 2023-11-28T13:02:21.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f582effd700 1 -- 172.21.15.6:0/4156714785 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300038e0 con 0x7f58400cea10 2023-11-28T13:02:21.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f58448a7700 1 -- 172.21.15.6:0/4156714785 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5830009550 con 0x7f58400cea10 2023-11-28T13:02:21.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.254+0000 7f58448a7700 1 -- 172.21.15.6:0/4156714785 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584006c920 con 0x7f58400cea10 2023-11-28T13:02:21.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.255+0000 7f58448a7700 1 -- 172.21.15.6:0/4156714785 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584006c640 con 0x7f58400cea10 2023-11-28T13:02:21.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.256+0000 7f582effd700 1 -- 172.21.15.6:0/4156714785 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f583000f050 con 0x7f58400cea10 2023-11-28T13:02:21.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.256+0000 7f582effd700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58200682f0 0x7f582006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:21.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.257+0000 7f582effd700 1 -- 172.21.15.6:0/4156714785 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f5830086e50 con 0x7f58400cea10 2023-11-28T13:02:21.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.257+0000 7f583e59c700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58200682f0 0x7f582006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:21.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.257+0000 7f583e59c700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58200682f0 0x7f582006a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5834009dc0 tx=0x7f5834008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:21.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.261+0000 7f582effd700 1 -- 172.21.15.6:0/4156714785 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f58300533f0 con 0x7f58400cea10 2023-11-28T13:02:21.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:21.454+0000 7f58448a7700 1 -- 172.21.15.6:0/4156714785 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f58400af280 con 0x7f58400cea10 2023-11-28T13:02:22.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.100+0000 7f582effd700 1 -- 172.21.15.6:0/4156714785 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7f58300150d0 con 0x7f58400cea10 2023-11-28T13:02:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 -- 172.21.15.6:0/4156714785 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58200682f0 msgr2=0x7f582006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58200682f0 0x7f582006a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5834009dc0 tx=0x7f5834008040 comp rx=0 tx=0).stop 2023-11-28T13:02:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 -- 172.21.15.6:0/4156714785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58400cea10 msgr2=0x7f58400aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58400cea10 0x7f58400aca30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5830009c40 tx=0x7f5830004750 comp rx=0 tx=0).stop 2023-11-28T13:02:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 -- 172.21.15.6:0/4156714785 shutdown_connections 2023-11-28T13:02:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f58200682f0 0x7f582006a7a0 secure :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5834009dc0 tx=0x7f5834008040 comp rx=0 tx=0).stop 2023-11-28T13:02:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5840159390 0x7f58400acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:22.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58400cdf80 0x7f5840153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:22.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 --2- 172.21.15.6:0/4156714785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f58400cea10 0x7f58400aca30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:22.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 -- 172.21.15.6:0/4156714785 >> 172.21.15.6:0/4156714785 conn(0x7f584006dae0 msgr2=0x7f58400b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:22.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 -- 172.21.15.6:0/4156714785 shutdown_connections 2023-11-28T13:02:22.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.105+0000 7f58448a7700 1 -- 172.21.15.6:0/4156714785 wait complete. 2023-11-28T13:02:22.108 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:02:22.122 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:02:22.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.258+0000 7fe400fec700 1 Processor -- start 2023-11-28T13:02:22.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.258+0000 7fe400fec700 1 -- start start 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe400fec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 0x7fe3fc06cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe400fec700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe3fc15e040 0x7fe3fc15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe400fec700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3fc06d4e0 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe400fec700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3fc15ee70 con 0x7fe3fc15e040 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe400fec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc15a8a0 con 0x7fe3fc0af140 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe3f9d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe3fc15e040 0x7fe3fc15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe3f9d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe3fc15e040 0x7fe3fc15e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32916/0 (socket says 172.21.15.6:32916) 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe3f9d9b700 1 -- 172.21.15.6:0/1427831888 learned_addr learned my addr 172.21.15.6:0/1427831888 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe3fa59c700 1 --2- 172.21.15.6:0/1427831888 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 0x7fe3fc06cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe3f9d9b700 1 -- 172.21.15.6:0/1427831888 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 msgr2=0x7fe3fc06cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:22.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe3f9d9b700 1 --2- 172.21.15.6:0/1427831888 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 0x7fe3fc06cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe3f9d9b700 1 -- 172.21.15.6:0/1427831888 >> v1:172.21.15.61:6789/0 conn(0x7fe3fc15e9f0 legacy=0x7fe3fc0b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe3f9d9b700 1 -- 172.21.15.6:0/1427831888 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc06ba30 con 0x7fe3fc15e040 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe3fa59c700 1 --2- 172.21.15.6:0/1427831888 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 0x7fe3fc06cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.259+0000 7fe3f9d9b700 1 --2- 172.21.15.6:0/1427831888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e040 0x7fe3fc15e4b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00098f0 tx=0x7fe3f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a7393b8aa15b21 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe3f959a700 1 -- 172.21.15.6:0/1427831888 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000b8a0 con 0x7fe3fc15e040 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe3f959a700 1 -- 172.21.15.6:0/1427831888 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f000ba00 con 0x7fe3fc15e040 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe3f959a700 1 -- 172.21.15.6:0/1427831888 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000bd90 con 0x7fe3fc15e040 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe400fec700 1 -- 172.21.15.6:0/1427831888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e040 msgr2=0x7fe3fc15e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe400fec700 1 --2- 172.21.15.6:0/1427831888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e040 0x7fe3fc15e4b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00098f0 tx=0x7fe3f0009c00 comp rx=0 tx=0).stop 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe400fec700 1 -- 172.21.15.6:0/1427831888 shutdown_connections 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe400fec700 1 --2- 172.21.15.6:0/1427831888 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 0x7fe3fc06cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:22.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe400fec700 1 --2- 172.21.15.6:0/1427831888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e040 0x7fe3fc15e4b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:22.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe400fec700 1 -- 172.21.15.6:0/1427831888 >> 172.21.15.6:0/1427831888 conn(0x7fe3fc06dae0 msgr2=0x7fe3fc06dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:22.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe400fec700 1 -- 172.21.15.6:0/1427831888 shutdown_connections 2023-11-28T13:02:22.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.260+0000 7fe400fec700 1 -- 172.21.15.6:0/1427831888 wait complete. 2023-11-28T13:02:22.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.262+0000 7fe400fec700 1 Processor -- start 2023-11-28T13:02:22.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.262+0000 7fe400fec700 1 -- start start 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.262+0000 7fe400fec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 0x7fe3fc157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.262+0000 7fe400fec700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3fc15e040 0x7fe3fc0c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.262+0000 7fe400fec700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e9f0 0x7fe3fc1e5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.262+0000 7fe400fec700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc158150 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.262+0000 7fe400fec700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc0c3aa0 con 0x7fe3fc15e040 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.262+0000 7fe400fec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc0c3d80 con 0x7fe3fc0af140 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.262+0000 7fe3fad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e9f0 0x7fe3fc1e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3fad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e9f0 0x7fe3fc1e5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32926/0 (socket says 172.21.15.6:32926) 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3fad9d700 1 -- 172.21.15.6:0/2708775656 learned_addr learned my addr 172.21.15.6:0/2708775656 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3fa59c700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 0x7fe3fc157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3f9d9b700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3fc15e040 0x7fe3fc0c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:22.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3fad9d700 1 -- 172.21.15.6:0/2708775656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 msgr2=0x7fe3fc157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3fad9d700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 0x7fe3fc157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3fad9d700 1 -- 172.21.15.6:0/2708775656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3fc15e040 msgr2=0x7fe3fc0c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3fad9d700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3fc15e040 0x7fe3fc0c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3fad9d700 1 -- 172.21.15.6:0/2708775656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0009550 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3fad9d700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e9f0 0x7fe3fc1e5890 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400cb50 tx=0x7fe3f400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3eaffd700 1 -- 172.21.15.6:0/2708775656 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4003c00 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3eaffd700 1 -- 172.21.15.6:0/2708775656 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f4003d60 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe400fec700 1 -- 172.21.15.6:0/2708775656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc06b750 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe3eaffd700 1 -- 172.21.15.6:0/2708775656 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400ab70 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.263+0000 7fe400fec700 1 -- 172.21.15.6:0/2708775656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc06b470 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.264+0000 7fe400fec700 1 -- 172.21.15.6:0/2708775656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3fc0cb2d0 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.265+0000 7fe3eaffd700 1 -- 172.21.15.6:0/2708775656 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe3f4010050 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.265+0000 7fe3eaffd700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3dc0682f0 0x7fe3dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:22.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.265+0000 7fe3eaffd700 1 -- 172.21.15.6:0/2708775656 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6088+0+0 (secure 0 0 0) 0x7fe3f4085c20 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.265+0000 7fe3fa59c700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3dc0682f0 0x7fe3dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:22.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.266+0000 7fe3fa59c700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3dc0682f0 0x7fe3dc06a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc1e66b0 tx=0x7fe3ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:22.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.269+0000 7fe3eaffd700 1 -- 172.21.15.6:0/2708775656 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe3f40523e0 con 0x7fe3fc15e9f0 2023-11-28T13:02:22.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:22.466+0000 7fe400fec700 1 -- 172.21.15.6:0/2708775656 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe3fc0739f0 con 0x7fe3fc15e9f0 2023-11-28T13:02:23.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.103+0000 7fe3eaffd700 1 -- 172.21.15.6:0/2708775656 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe3f4055a00 con 0x7fe3fc15e9f0 2023-11-28T13:02:23.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 -- 172.21.15.6:0/2708775656 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3dc0682f0 msgr2=0x7fe3dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3dc0682f0 0x7fe3dc06a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc1e66b0 tx=0x7fe3ec008040 comp rx=0 tx=0).stop 2023-11-28T13:02:23.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 -- 172.21.15.6:0/2708775656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e9f0 msgr2=0x7fe3fc1e5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e9f0 0x7fe3fc1e5890 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400cb50 tx=0x7fe3f400cf10 comp rx=0 tx=0).stop 2023-11-28T13:02:23.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 -- 172.21.15.6:0/2708775656 shutdown_connections 2023-11-28T13:02:23.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3fc15e9f0 0x7fe3fc1e5890 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3dc0682f0 0x7fe3dc06a7a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe3fc0af140 0x7fe3fc157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 --2- 172.21.15.6:0/2708775656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3fc15e040 0x7fe3fc0c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 -- 172.21.15.6:0/2708775656 >> 172.21.15.6:0/2708775656 conn(0x7fe3fc06dae0 msgr2=0x7fe3fc0b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:23.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 -- 172.21.15.6:0/2708775656 shutdown_connections 2023-11-28T13:02:23.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.108+0000 7fe400fec700 1 -- 172.21.15.6:0/2708775656 wait complete. 2023-11-28T13:02:23.111 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' removed 2023-11-28T13:02:23.125 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T13:02:23.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.257+0000 7f549ab75700 1 Processor -- start 2023-11-28T13:02:23.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.257+0000 7f549ab75700 1 -- start start 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f549ab75700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f54940b1d40 0x7f54940cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f549ab75700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54940ceff0 con 0x7f549415dbf0 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f549ab75700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54940cf2d0 con 0x7f5494159500 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f549ab75700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549415e090 con 0x7f54940b1d40 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f5493fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f54940b1d40 0x7f54940cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f5493fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f54940b1d40 0x7f54940cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32932/0 (socket says 172.21.15.6:32932) 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f5493fff700 1 -- 172.21.15.6:0/149616119 learned_addr learned my addr 172.21.15.6:0/149616119 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f5499112700 1 --1- 172.21.15.6:0/149616119 >> v1:172.21.15.6:6789/0 conn(0x7f549415dbf0 0x7f549406bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48486/0 (socket says 172.21.15.6:48486) 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f5493fff700 1 -- 172.21.15.6:0/149616119 >> v1:172.21.15.61:6789/0 conn(0x7f5494159500 legacy=0x7f54940b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f5493fff700 1 -- 172.21.15.6:0/149616119 >> v1:172.21.15.6:6789/0 conn(0x7f549415dbf0 legacy=0x7f549406bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f54937fe700 1 -- 172.21.15.6:0/149616119 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341050442 0 0) 0x7f54940ceff0 con 0x7f549415dbf0 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f5493fff700 1 -- 172.21.15.6:0/149616119 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549415d110 con 0x7f54940b1d40 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f5493fff700 1 --2- 172.21.15.6:0/149616119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54940b1d40 0x7f54940cd140 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f548000bb90 tx=0x7f548000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15a3cf7f14c5164 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f54937fe700 1 -- 172.21.15.6:0/149616119 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548000dd10 con 0x7f54940b1d40 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f54937fe700 1 -- 172.21.15.6:0/149616119 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548000de70 con 0x7f54940b1d40 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.258+0000 7f54937fe700 1 -- 172.21.15.6:0/149616119 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480004230 con 0x7f54940b1d40 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.259+0000 7f549ab75700 1 -- 172.21.15.6:0/149616119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54940b1d40 msgr2=0x7f54940cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.259+0000 7f549ab75700 1 --2- 172.21.15.6:0/149616119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54940b1d40 0x7f54940cd140 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f548000bb90 tx=0x7f548000bea0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.259+0000 7f549ab75700 1 -- 172.21.15.6:0/149616119 shutdown_connections 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.259+0000 7f549ab75700 1 --2- 172.21.15.6:0/149616119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f54940b1d40 0x7f54940cd140 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.259+0000 7f549ab75700 1 -- 172.21.15.6:0/149616119 >> 172.21.15.6:0/149616119 conn(0x7f54940cc630 msgr2=0x7f54940cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.259+0000 7f549ab75700 1 -- 172.21.15.6:0/149616119 shutdown_connections 2023-11-28T13:02:23.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.259+0000 7f549ab75700 1 -- 172.21.15.6:0/149616119 wait complete. 2023-11-28T13:02:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.260+0000 7f549ab75700 1 Processor -- start 2023-11-28T13:02:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.260+0000 7f549ab75700 1 -- start start 2023-11-28T13:02:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f549ab75700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54940b1d40 0x7f5494154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f549ab75700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5494159500 0x7f5494159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f549ab75700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f549415dbf0 0x7f549415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f549ab75700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54940c70c0 con 0x7f5494159500 2023-11-28T13:02:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f549ab75700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54940c7360 con 0x7f549415dbf0 2023-11-28T13:02:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f549ab75700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549415c700 con 0x7f54940b1d40 2023-11-28T13:02:23.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f5493fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5494159500 0x7f5494159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f5493fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5494159500 0x7f5494159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32938/0 (socket says 172.21.15.6:32938) 2023-11-28T13:02:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f5493fff700 1 -- 172.21.15.6:0/481231595 learned_addr learned my addr 172.21.15.6:0/481231595 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f5499112700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f549415dbf0 0x7f549415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f5493fff700 1 -- 172.21.15.6:0/481231595 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54940b1d40 msgr2=0x7f5494154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.261+0000 7f5493fff700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54940b1d40 0x7f5494154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.262+0000 7f5493fff700 1 -- 172.21.15.6:0/481231595 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f549415dbf0 msgr2=0x7f549415b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.262+0000 7f5493fff700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f549415dbf0 0x7f549415b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.262+0000 7f5493fff700 1 -- 172.21.15.6:0/481231595 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548000b7f0 con 0x7f5494159500 2023-11-28T13:02:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.262+0000 7f5499112700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f549415dbf0 0x7f549415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:23.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.262+0000 7f5493fff700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5494159500 0x7f5494159c10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5480004000 tx=0x7f5480004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.262+0000 7f54917fa700 1 -- 172.21.15.6:0/481231595 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480020070 con 0x7f5494159500 2023-11-28T13:02:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.262+0000 7f54917fa700 1 -- 172.21.15.6:0/481231595 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54800043e0 con 0x7f5494159500 2023-11-28T13:02:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.262+0000 7f54917fa700 1 -- 172.21.15.6:0/481231595 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480010470 con 0x7f5494159500 2023-11-28T13:02:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.262+0000 7f549ab75700 1 -- 172.21.15.6:0/481231595 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549415d6d0 con 0x7f5494159500 2023-11-28T13:02:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.262+0000 7f549ab75700 1 -- 172.21.15.6:0/481231595 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549415d3f0 con 0x7f5494159500 2023-11-28T13:02:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.263+0000 7f549ab75700 1 -- 172.21.15.6:0/481231595 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54941598d0 con 0x7f5494159500 2023-11-28T13:02:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.263+0000 7f54917fa700 1 -- 172.21.15.6:0/481231595 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f54800193f0 con 0x7f5494159500 2023-11-28T13:02:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.264+0000 7f54917fa700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5478068320 0x7f547806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.264+0000 7f54917fa700 1 -- 172.21.15.6:0/481231595 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f54800892f0 con 0x7f5494159500 2023-11-28T13:02:23.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.264+0000 7f5498911700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5478068320 0x7f547806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:23.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.264+0000 7f5498911700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5478068320 0x7f547806a7d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5488003df0 tx=0x7f5488007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:23.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.268+0000 7f54917fa700 1 -- 172.21.15.6:0/481231595 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5480055c30 con 0x7f5494159500 2023-11-28T13:02:23.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.466+0000 7f549ab75700 1 -- 172.21.15.6:0/481231595 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f54940c6940 con 0x7f5494159500 2023-11-28T13:02:23.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.470+0000 7f54917fa700 1 -- 172.21.15.6:0/481231595 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5480059250 con 0x7f5494159500 2023-11-28T13:02:23.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 -- 172.21.15.6:0/481231595 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5478068320 msgr2=0x7f547806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5478068320 0x7f547806a7d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5488003df0 tx=0x7f5488007a40 comp rx=0 tx=0).stop 2023-11-28T13:02:23.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 -- 172.21.15.6:0/481231595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5494159500 msgr2=0x7f5494159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5494159500 0x7f5494159c10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5480004000 tx=0x7f5480004750 comp rx=0 tx=0).stop 2023-11-28T13:02:23.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 -- 172.21.15.6:0/481231595 shutdown_connections 2023-11-28T13:02:23.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f549415dbf0 0x7f549415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5478068320 0x7f547806a7d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f54940b1d40 0x7f5494154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 --2- 172.21.15.6:0/481231595 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5494159500 0x7f5494159c10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 -- 172.21.15.6:0/481231595 >> 172.21.15.6:0/481231595 conn(0x7f54940cc630 msgr2=0x7f5494157650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:23.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 -- 172.21.15.6:0/481231595 shutdown_connections 2023-11-28T13:02:23.474 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.472+0000 7f549ab75700 1 -- 172.21.15.6:0/481231595 wait complete. 2023-11-28T13:02:23.485 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T13:02:23.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.608+0000 7fa504297700 1 Processor -- start 2023-11-28T13:02:23.610 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.609+0000 7fa504297700 1 -- start start 2023-11-28T13:02:23.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.609+0000 7fa504297700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa4fc15d1e0 0x7fa4fc15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.609+0000 7fa504297700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa4fc06a560 0x7fa4fc06aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.609+0000 7fa504297700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4fc0cdec0 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa504297700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4fc06af40 con 0x7fa4fc06a560 2023-11-28T13:02:23.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa504297700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4fc0c9cd0 con 0x7fa4fc15d1e0 2023-11-28T13:02:23.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501832700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa4fc06a560 0x7fa4fc06aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:23.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501832700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa4fc06a560 0x7fa4fc06aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32946/0 (socket says 172.21.15.6:32946) 2023-11-28T13:02:23.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501832700 1 -- 172.21.15.6:0/1332138179 learned_addr learned my addr 172.21.15.6:0/1332138179 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:23.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa502033700 1 --2- 172.21.15.6:0/1332138179 >> v2:172.21.15.61:3300/0 conn(0x7fa4fc15d1e0 0x7fa4fc15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:23.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501832700 1 -- 172.21.15.6:0/1332138179 >> v2:172.21.15.61:3300/0 conn(0x7fa4fc15d1e0 msgr2=0x7fa4fc15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:23.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501832700 1 --2- 172.21.15.6:0/1332138179 >> v2:172.21.15.61:3300/0 conn(0x7fa4fc15d1e0 0x7fa4fc15d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501832700 1 -- 172.21.15.6:0/1332138179 >> v1:172.21.15.61:6789/0 conn(0x7fa4fc0cceb0 legacy=0x7fa4fc0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501832700 1 -- 172.21.15.6:0/1332138179 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc0c9fb0 con 0x7fa4fc06a560 2023-11-28T13:02:23.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501832700 1 --2- 172.21.15.6:0/1332138179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc06a560 0x7fa4fc06aa00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0b1ce0 tx=0x7fa4f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30577eb4bf21f435 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:23.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501031700 1 -- 172.21.15.6:0/1332138179 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000b870 con 0x7fa4fc06a560 2023-11-28T13:02:23.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501031700 1 -- 172.21.15.6:0/1332138179 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f000b9d0 con 0x7fa4fc06a560 2023-11-28T13:02:23.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.610+0000 7fa501031700 1 -- 172.21.15.6:0/1332138179 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000bd60 con 0x7fa4fc06a560 2023-11-28T13:02:23.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.611+0000 7fa504297700 1 -- 172.21.15.6:0/1332138179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc06a560 msgr2=0x7fa4fc06aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.611+0000 7fa504297700 1 --2- 172.21.15.6:0/1332138179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc06a560 0x7fa4fc06aa00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0b1ce0 tx=0x7fa4f0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.611+0000 7fa504297700 1 -- 172.21.15.6:0/1332138179 shutdown_connections 2023-11-28T13:02:23.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.611+0000 7fa504297700 1 --2- 172.21.15.6:0/1332138179 >> v2:172.21.15.61:3300/0 conn(0x7fa4fc15d1e0 0x7fa4fc15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.611+0000 7fa504297700 1 --2- 172.21.15.6:0/1332138179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc06a560 0x7fa4fc06aa00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.611+0000 7fa504297700 1 -- 172.21.15.6:0/1332138179 >> 172.21.15.6:0/1332138179 conn(0x7fa4fc0b0dd0 msgr2=0x7fa4fc0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:23.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.611+0000 7fa504297700 1 -- 172.21.15.6:0/1332138179 shutdown_connections 2023-11-28T13:02:23.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.611+0000 7fa504297700 1 -- 172.21.15.6:0/1332138179 wait complete. 2023-11-28T13:02:23.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.612+0000 7fa504297700 1 Processor -- start 2023-11-28T13:02:23.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.612+0000 7fa504297700 1 -- start start 2023-11-28T13:02:23.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa504297700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4fc06a560 0x7fa4fc1ef570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa504297700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc0cceb0 0x7fa4fc1f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa504297700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4fc15d1e0 0x7fa4fc0ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa504297700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc1efab0 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa504297700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc1f3010 con 0x7fa4fc06a560 2023-11-28T13:02:23.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa504297700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc0c6800 con 0x7fa4fc15d1e0 2023-11-28T13:02:23.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa501832700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc0cceb0 0x7fa4fc1f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:23.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa501832700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc0cceb0 0x7fa4fc1f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32952/0 (socket says 172.21.15.6:32952) 2023-11-28T13:02:23.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa501832700 1 -- 172.21.15.6:0/897579261 learned_addr learned my addr 172.21.15.6:0/897579261 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:23.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa502033700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4fc06a560 0x7fa4fc1ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:23.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa502834700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4fc15d1e0 0x7fa4fc0ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:23.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa501832700 1 -- 172.21.15.6:0/897579261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4fc15d1e0 msgr2=0x7fa4fc0ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa501832700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4fc15d1e0 0x7fa4fc0ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa501832700 1 -- 172.21.15.6:0/897579261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4fc06a560 msgr2=0x7fa4fc1ef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:23.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa501832700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4fc06a560 0x7fa4fc1ef570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa501832700 1 -- 172.21.15.6:0/897579261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f0009550 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.613+0000 7fa502834700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4fc15d1e0 0x7fa4fc0ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.614+0000 7fa501832700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc0cceb0 0x7fa4fc1f2ad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0000c00 tx=0x7fa4f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.614+0000 7fa4eeffd700 1 -- 172.21.15.6:0/897579261 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0004520 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.614+0000 7fa4eeffd700 1 -- 172.21.15.6:0/897579261 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f001e070 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.614+0000 7fa4eeffd700 1 -- 172.21.15.6:0/897579261 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00038a0 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.614+0000 7fa504297700 1 -- 172.21.15.6:0/897579261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc0ca570 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.614+0000 7fa504297700 1 -- 172.21.15.6:0/897579261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc0ca290 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.615+0000 7fa4ecff9700 1 -- 172.21.15.6:0/897579261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d80052f0 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.615+0000 7fa4eeffd700 1 -- 172.21.15.6:0/897579261 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa4f000f050 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.616+0000 7fa4eeffd700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4e00682f0 0x7fa4e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:23.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.616+0000 7fa4eeffd700 1 -- 172.21.15.6:0/897579261 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa4f0020070 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.616+0000 7fa502033700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4e00682f0 0x7fa4e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:23.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.616+0000 7fa502033700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4e00682f0 0x7fa4e006a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80099d0 tx=0x7fa4f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:23.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.620+0000 7fa4eeffd700 1 -- 172.21.15.6:0/897579261 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa4f0053af0 con 0x7fa4fc0cceb0 2023-11-28T13:02:23.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:23.894+0000 7fa4ecff9700 1 -- 172.21.15.6:0/897579261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fa4d80055a0 con 0x7fa4fc0cceb0 2023-11-28T13:02:24.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.112+0000 7fa4eeffd700 1 -- 172.21.15.6:0/897579261 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v127) v1 ==== 142+0+0 (secure 0 0 0) 0x7fa4f0057110 con 0x7fa4fc0cceb0 2023-11-28T13:02:24.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 -- 172.21.15.6:0/897579261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4e00682f0 msgr2=0x7fa4e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:24.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4e00682f0 0x7fa4e006a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80099d0 tx=0x7fa4f8008040 comp rx=0 tx=0).stop 2023-11-28T13:02:24.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 -- 172.21.15.6:0/897579261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc0cceb0 msgr2=0x7fa4fc1f2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:24.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc0cceb0 0x7fa4fc1f2ad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0000c00 tx=0x7fa4f0004750 comp rx=0 tx=0).stop 2023-11-28T13:02:24.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 -- 172.21.15.6:0/897579261 shutdown_connections 2023-11-28T13:02:24.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa4e00682f0 0x7fa4e006a7a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:24.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa4fc15d1e0 0x7fa4fc0ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:24.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4fc06a560 0x7fa4fc1ef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:24.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 --2- 172.21.15.6:0/897579261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa4fc0cceb0 0x7fa4fc1f2ad0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:24.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 -- 172.21.15.6:0/897579261 >> 172.21.15.6:0/897579261 conn(0x7fa4fc0b0dd0 msgr2=0x7fa4fc15b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:24.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 -- 172.21.15.6:0/897579261 shutdown_connections 2023-11-28T13:02:24.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.114+0000 7fa4ecff9700 1 -- 172.21.15.6:0/897579261 wait complete. 2023-11-28T13:02:24.128 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-28T13:02:24.128 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-28T13:02:24.129 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:02:24.129 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T13:02:24.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.254+0000 7f5932123700 1 Processor -- start 2023-11-28T13:02:24.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.254+0000 7f5932123700 1 -- start start 2023-11-28T13:02:24.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f5932123700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f592c15c990 0x7f592c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:24.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f5932123700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f592c161090 0x7f592c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f5932123700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c0ce340 con 0x7f592c0caeb0 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f5932123700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c0cebe0 con 0x7f592c15c990 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f5932123700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c0cb3f0 con 0x7f592c161090 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592b7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f592c15c990 0x7f592c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592bfff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f592c0caeb0 0x7f592c0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48512/0 (socket says 172.21.15.6:48512) 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592bfff700 1 -- 172.21.15.6:0/559672147 learned_addr learned my addr 172.21.15.6:0/559672147 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592affd700 1 --2- 172.21.15.6:0/559672147 >> v2:172.21.15.61:3300/0 conn(0x7f592c161090 0x7f592c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592b7fe700 1 -- 172.21.15.6:0/559672147 >> v2:172.21.15.61:3300/0 conn(0x7f592c161090 msgr2=0x7f592c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592b7fe700 1 --2- 172.21.15.6:0/559672147 >> v2:172.21.15.61:3300/0 conn(0x7f592c161090 0x7f592c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592b7fe700 1 -- 172.21.15.6:0/559672147 >> v1:172.21.15.6:6789/0 conn(0x7f592c0caeb0 legacy=0x7f592c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592b7fe700 1 -- 172.21.15.6:0/559672147 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c0ce590 con 0x7f592c15c990 2023-11-28T13:02:24.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592affd700 1 --2- 172.21.15.6:0/559672147 >> v2:172.21.15.61:3300/0 conn(0x7f592c161090 0x7f592c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592b7fe700 1 --2- 172.21.15.6:0/559672147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c15c990 0x7f592c0cf070 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f592c06ab20 tx=0x7f591c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b345ef249cdf83 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592a7fc700 1 -- 172.21.15.6:0/559672147 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c011830 con 0x7f592c15c990 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592a7fc700 1 -- 172.21.15.6:0/559672147 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c011990 con 0x7f592c15c990 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.255+0000 7f592a7fc700 1 -- 172.21.15.6:0/559672147 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c011d20 con 0x7f592c15c990 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.256+0000 7f5932123700 1 -- 172.21.15.6:0/559672147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c15c990 msgr2=0x7f592c0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.256+0000 7f5932123700 1 --2- 172.21.15.6:0/559672147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c15c990 0x7f592c0cf070 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f592c06ab20 tx=0x7f591c009b20 comp rx=0 tx=0).stop 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.256+0000 7f5932123700 1 -- 172.21.15.6:0/559672147 shutdown_connections 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.256+0000 7f5932123700 1 --2- 172.21.15.6:0/559672147 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c15c990 0x7f592c0cf070 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.256+0000 7f5932123700 1 --2- 172.21.15.6:0/559672147 >> v2:172.21.15.61:3300/0 conn(0x7f592c161090 0x7f592c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.256+0000 7f5932123700 1 -- 172.21.15.6:0/559672147 >> 172.21.15.6:0/559672147 conn(0x7f592c0b1650 msgr2=0x7f592c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.256+0000 7f5932123700 1 -- 172.21.15.6:0/559672147 shutdown_connections 2023-11-28T13:02:24.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.256+0000 7f5932123700 1 -- 172.21.15.6:0/559672147 wait complete. 2023-11-28T13:02:24.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.257+0000 7f5932123700 1 Processor -- start 2023-11-28T13:02:24.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.257+0000 7f5932123700 1 -- start start 2023-11-28T13:02:24.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.258+0000 7f5932123700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f592c0caeb0 0x7f592c14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:24.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.258+0000 7f5932123700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c161090 0x7f592c14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.258+0000 7f5932123700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c14eeb0 0x7f592c0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.258+0000 7f5932123700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c0c3810 con 0x7f592c161090 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.258+0000 7f5932123700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c0b5610 con 0x7f592c14eeb0 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.258+0000 7f5932123700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c0b58f0 con 0x7f592c0caeb0 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.258+0000 7f592affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c161090 0x7f592c14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.258+0000 7f592affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c161090 0x7f592c14e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32964/0 (socket says 172.21.15.6:32964) 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.258+0000 7f592affd700 1 -- 172.21.15.6:0/2321344429 learned_addr learned my addr 172.21.15.6:0/2321344429 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f592b7fe700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f592c0caeb0 0x7f592c14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f592bfff700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c14eeb0 0x7f592c0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f592affd700 1 -- 172.21.15.6:0/2321344429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f592c0caeb0 msgr2=0x7f592c14deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f592affd700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f592c0caeb0 0x7f592c14deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f592affd700 1 -- 172.21.15.6:0/2321344429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c14eeb0 msgr2=0x7f592c0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:24.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f592affd700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c14eeb0 0x7f592c0b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f592affd700 1 -- 172.21.15.6:0/2321344429 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0094a0 con 0x7f592c161090 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f592bfff700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c14eeb0 0x7f592c0b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f592affd700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c161090 0x7f592c14e970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f592000c750 tx=0x7f592000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f5913fff700 1 -- 172.21.15.6:0/2321344429 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920011800 con 0x7f592c161090 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f5913fff700 1 -- 172.21.15.6:0/2321344429 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5920011960 con 0x7f592c161090 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f5932123700 1 -- 172.21.15.6:0/2321344429 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c0cf5b0 con 0x7f592c161090 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f5913fff700 1 -- 172.21.15.6:0/2321344429 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920011cd0 con 0x7f592c161090 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.259+0000 7f5932123700 1 -- 172.21.15.6:0/2321344429 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c0ce870 con 0x7f592c161090 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.260+0000 7f5932123700 1 -- 172.21.15.6:0/2321344429 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c0cf740 con 0x7f592c161090 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.260+0000 7f5913fff700 1 -- 172.21.15.6:0/2321344429 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5920020070 con 0x7f592c161090 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.261+0000 7f5913fff700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f59180682f0 0x7f591806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.261+0000 7f5913fff700 1 -- 172.21.15.6:0/2321344429 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f5920085a20 con 0x7f592c161090 2023-11-28T13:02:24.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.261+0000 7f592b7fe700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f59180682f0 0x7f591806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:24.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.261+0000 7f592b7fe700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f59180682f0 0x7f591806a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f592c06f7d0 tx=0x7f591c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:24.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.265+0000 7f5913fff700 1 -- 172.21.15.6:0/2321344429 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5920052370 con 0x7f592c161090 2023-11-28T13:02:24.538 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:24.537+0000 7f5932123700 1 -- 172.21.15.6:0/2321344429 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f592c0675a0 con 0x7f592c161090 2023-11-28T13:02:25.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.117+0000 7f5913fff700 1 -- 172.21.15.6:0/2321344429 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v128) v1 ==== 152+0+0 (secure 0 0 0) 0x7f5920055990 con 0x7f592c161090 2023-11-28T13:02:25.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.119+0000 7f5932123700 1 -- 172.21.15.6:0/2321344429 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f59180682f0 msgr2=0x7f591806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.119+0000 7f5932123700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f59180682f0 0x7f591806a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f592c06f7d0 tx=0x7f591c008030 comp rx=0 tx=0).stop 2023-11-28T13:02:25.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.119+0000 7f5932123700 1 -- 172.21.15.6:0/2321344429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c161090 msgr2=0x7f592c14e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.119+0000 7f5932123700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c161090 0x7f592c14e970 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f592000c750 tx=0x7f592000cb10 comp rx=0 tx=0).stop 2023-11-28T13:02:25.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.120+0000 7f5932123700 1 -- 172.21.15.6:0/2321344429 shutdown_connections 2023-11-28T13:02:25.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.120+0000 7f5932123700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c14eeb0 0x7f592c0b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.120+0000 7f5932123700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f59180682f0 0x7f591806a7a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.120+0000 7f5932123700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f592c0caeb0 0x7f592c14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.120+0000 7f5932123700 1 --2- 172.21.15.6:0/2321344429 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f592c161090 0x7f592c14e970 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.120+0000 7f5932123700 1 -- 172.21.15.6:0/2321344429 >> 172.21.15.6:0/2321344429 conn(0x7f592c0b1650 msgr2=0x7f592c1603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:25.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.120+0000 7f5932123700 1 -- 172.21.15.6:0/2321344429 shutdown_connections 2023-11-28T13:02:25.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.120+0000 7f5932123700 1 -- 172.21.15.6:0/2321344429 wait complete. 2023-11-28T13:02:25.136 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T13:02:25.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.270+0000 7f6c5e54a700 1 Processor -- start 2023-11-28T13:02:25.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.270+0000 7f6c5e54a700 1 -- start start 2023-11-28T13:02:25.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c5e54a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6c58159500 0x7f6c580cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c5e54a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c5815dbf0 0x7f6c580ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c5e54a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c580b1630 0x7f6c580b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c5e54a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c580b2010 con 0x7f6c5815dbf0 2023-11-28T13:02:25.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c5e54a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c580cede0 con 0x7f6c58159500 2023-11-28T13:02:25.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c5e54a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5815e040 con 0x7f6c580b1630 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c577fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c5815dbf0 0x7f6c580ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c577fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c5815dbf0 0x7f6c580ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32970/0 (socket says 172.21.15.6:32970) 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c577fe700 1 -- 172.21.15.6:0/704780790 learned_addr learned my addr 172.21.15.6:0/704780790 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c5cae7700 1 --2- 172.21.15.6:0/704780790 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c580b1630 0x7f6c580b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c57fff700 1 --2- 172.21.15.6:0/704780790 >> v2:172.21.15.61:3300/0 conn(0x7f6c58159500 0x7f6c580cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c577fe700 1 -- 172.21.15.6:0/704780790 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c580b1630 msgr2=0x7f6c580b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c577fe700 1 --2- 172.21.15.6:0/704780790 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c580b1630 0x7f6c580b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c577fe700 1 -- 172.21.15.6:0/704780790 >> v2:172.21.15.61:3300/0 conn(0x7f6c58159500 msgr2=0x7f6c580cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c577fe700 1 --2- 172.21.15.6:0/704780790 >> v2:172.21.15.61:3300/0 conn(0x7f6c58159500 0x7f6c580cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c577fe700 1 -- 172.21.15.6:0/704780790 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580ceff0 con 0x7f6c5815dbf0 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c577fe700 1 --2- 172.21.15.6:0/704780790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 0x7f6c580ce8a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6c580d0bc0 tx=0x7f6c4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37589102d1cf2ff0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:25.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.271+0000 7f6c56ffd700 1 -- 172.21.15.6:0/704780790 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00b860 con 0x7f6c5815dbf0 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c56ffd700 1 -- 172.21.15.6:0/704780790 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c00b9c0 con 0x7f6c5815dbf0 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c56ffd700 1 -- 172.21.15.6:0/704780790 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00bd50 con 0x7f6c5815dbf0 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c5e54a700 1 -- 172.21.15.6:0/704780790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 msgr2=0x7f6c580ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c5e54a700 1 --2- 172.21.15.6:0/704780790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 0x7f6c580ce8a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6c580d0bc0 tx=0x7f6c4c009c00 comp rx=0 tx=0).stop 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c5e54a700 1 -- 172.21.15.6:0/704780790 shutdown_connections 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c5e54a700 1 --2- 172.21.15.6:0/704780790 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c580b1630 0x7f6c580b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c5e54a700 1 --2- 172.21.15.6:0/704780790 >> v2:172.21.15.61:3300/0 conn(0x7f6c58159500 0x7f6c580cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c5e54a700 1 --2- 172.21.15.6:0/704780790 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 0x7f6c580ce8a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c5e54a700 1 -- 172.21.15.6:0/704780790 >> 172.21.15.6:0/704780790 conn(0x7f6c580cc630 msgr2=0x7f6c580cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c5e54a700 1 -- 172.21.15.6:0/704780790 shutdown_connections 2023-11-28T13:02:25.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.272+0000 7f6c5e54a700 1 -- 172.21.15.6:0/704780790 wait complete. 2023-11-28T13:02:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.274+0000 7f6c5e54a700 1 Processor -- start 2023-11-28T13:02:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.274+0000 7f6c5e54a700 1 -- start start 2023-11-28T13:02:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5e54a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c580b1630 0x7f6c5814dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5e54a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6c58159500 0x7f6c58154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5e54a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 0x7f6c58155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5e54a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58153820 con 0x7f6c5815dbf0 2023-11-28T13:02:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5e54a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58153ac0 con 0x7f6c58159500 2023-11-28T13:02:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5e54a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58153da0 con 0x7f6c580b1630 2023-11-28T13:02:25.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5cae7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 0x7f6c58155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5cae7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 0x7f6c58155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32980/0 (socket says 172.21.15.6:32980) 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5cae7700 1 -- 172.21.15.6:0/224137496 learned_addr learned my addr 172.21.15.6:0/224137496 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c57fff700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c580b1630 0x7f6c5814dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c577fe700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6c58159500 0x7f6c58154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5cae7700 1 -- 172.21.15.6:0/224137496 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c580b1630 msgr2=0x7f6c5814dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5cae7700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c580b1630 0x7f6c5814dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5cae7700 1 -- 172.21.15.6:0/224137496 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6c58159500 msgr2=0x7f6c58154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5cae7700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6c58159500 0x7f6c58154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.275+0000 7f6c5cae7700 1 -- 172.21.15.6:0/224137496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c009550 con 0x7f6c5815dbf0 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.276+0000 7f6c5cae7700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 0x7f6c58155350 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000cb80 tx=0x7f6c5000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.276+0000 7f6c54ff9700 1 -- 172.21.15.6:0/224137496 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000a8e0 con 0x7f6c5815dbf0 2023-11-28T13:02:25.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.276+0000 7f6c54ff9700 1 -- 172.21.15.6:0/224137496 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5000aa40 con 0x7f6c5815dbf0 2023-11-28T13:02:25.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.276+0000 7f6c5e54a700 1 -- 172.21.15.6:0/224137496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5815d110 con 0x7f6c5815dbf0 2023-11-28T13:02:25.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.276+0000 7f6c54ff9700 1 -- 172.21.15.6:0/224137496 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000ae30 con 0x7f6c5815dbf0 2023-11-28T13:02:25.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.276+0000 7f6c5e54a700 1 -- 172.21.15.6:0/224137496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c580cf2d0 con 0x7f6c5815dbf0 2023-11-28T13:02:25.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.277+0000 7f6c5e54a700 1 -- 172.21.15.6:0/224137496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5815d3f0 con 0x7f6c5815dbf0 2023-11-28T13:02:25.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.277+0000 7f6c54ff9700 1 -- 172.21.15.6:0/224137496 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6c50021070 con 0x7f6c5815dbf0 2023-11-28T13:02:25.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.278+0000 7f6c54ff9700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c44068320 0x7f6c4406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.278+0000 7f6c54ff9700 1 -- 172.21.15.6:0/224137496 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6c50085a30 con 0x7f6c5815dbf0 2023-11-28T13:02:25.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.278+0000 7f6c57fff700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c44068320 0x7f6c4406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.278+0000 7f6c57fff700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c44068320 0x7f6c4406a7d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6c580b3d10 tx=0x7f6c48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:25.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.282+0000 7f6c54ff9700 1 -- 172.21.15.6:0/224137496 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6c50087e60 con 0x7f6c5815dbf0 2023-11-28T13:02:25.480 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.478+0000 7f6c5e54a700 1 -- 172.21.15.6:0/224137496 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6c580739f0 con 0x7f6c5815dbf0 2023-11-28T13:02:25.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.482+0000 7f6c54ff9700 1 -- 172.21.15.6:0/224137496 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6c5001f0c0 con 0x7f6c5815dbf0 2023-11-28T13:02:25.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.484+0000 7f6c5e54a700 1 -- 172.21.15.6:0/224137496 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c44068320 msgr2=0x7f6c4406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.484+0000 7f6c5e54a700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c44068320 0x7f6c4406a7d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6c580b3d10 tx=0x7f6c48008040 comp rx=0 tx=0).stop 2023-11-28T13:02:25.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.484+0000 7f6c5e54a700 1 -- 172.21.15.6:0/224137496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 msgr2=0x7f6c58155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.484+0000 7f6c5e54a700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 0x7f6c58155350 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000cb80 tx=0x7f6c5000cf40 comp rx=0 tx=0).stop 2023-11-28T13:02:25.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.484+0000 7f6c5e54a700 1 -- 172.21.15.6:0/224137496 shutdown_connections 2023-11-28T13:02:25.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.484+0000 7f6c5e54a700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c5815dbf0 0x7f6c58155350 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.484+0000 7f6c5e54a700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c44068320 0x7f6c4406a7d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.484+0000 7f6c5e54a700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c580b1630 0x7f6c5814dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.484+0000 7f6c5e54a700 1 --2- 172.21.15.6:0/224137496 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6c58159500 0x7f6c58154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.484+0000 7f6c5e54a700 1 -- 172.21.15.6:0/224137496 >> 172.21.15.6:0/224137496 conn(0x7f6c580cc630 msgr2=0x7f6c580cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:25.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.485+0000 7f6c5e54a700 1 -- 172.21.15.6:0/224137496 shutdown_connections 2023-11-28T13:02:25.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.485+0000 7f6c5e54a700 1 -- 172.21.15.6:0/224137496 wait complete. 2023-11-28T13:02:25.498 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:02:25.498 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:02:25.539 DEBUG:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:02:25.585 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:02:25.585 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:02:25.585 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:25.603 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3446 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:02:25.603 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:02:25.603 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:25.623 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:02:25.623 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:02:25.623 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:25.660 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3446 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:02:25.660 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:02:25.661 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:25.682 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:02:25.682 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T13:02:25.682 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:25.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.823+0000 7f08d8860700 1 Processor -- start 2023-11-28T13:02:25.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.823+0000 7f08d8860700 1 -- start start 2023-11-28T13:02:25.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.824+0000 7f08d8860700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415a650 0x7f08d40b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.824+0000 7f08d8860700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f08d415aa20 0x7f08d40ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.824+0000 7f08d8860700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d40ca990 con 0x7f08d40b5500 2023-11-28T13:02:25.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.824+0000 7f08d8860700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d40caf50 con 0x7f08d415aa20 2023-11-28T13:02:25.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.824+0000 7f08d8860700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40aec70 con 0x7f08d415a650 2023-11-28T13:02:25.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d2d9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f08d40b5500 0x7f08d40cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48538/0 (socket says 172.21.15.6:48538) 2023-11-28T13:02:25.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d2d9d700 1 -- 172.21.15.6:0/3428556870 learned_addr learned my addr 172.21.15.6:0/3428556870 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:25.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d259c700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415a650 0x7f08d40b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d1d9b700 1 --2- 172.21.15.6:0/3428556870 >> v2:172.21.15.61:3300/0 conn(0x7f08d415aa20 0x7f08d40ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352325974 0 0) 0x7f08d40ca990 con 0x7f08d40b5500 2023-11-28T13:02:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08bc0035b0 con 0x7f08d40b5500 2023-11-28T13:02:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1144435487 0 0) 0x7f08bc0035b0 con 0x7f08d40b5500 2023-11-28T13:02:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415a650 msgr2=0x7f08d40b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415a650 0x7f08d40b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 >> v2:172.21.15.61:3300/0 conn(0x7f08d415aa20 msgr2=0x7f08d40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 --2- 172.21.15.6:0/3428556870 >> v2:172.21.15.61:3300/0 conn(0x7f08d415aa20 0x7f08d40ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d40aef10 con 0x7f08d40b5500 2023-11-28T13:02:25.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f08cc0019f0 con 0x7f08d40b5500 2023-11-28T13:02:25.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d1d9b700 1 --2- 172.21.15.6:0/3428556870 >> v2:172.21.15.61:3300/0 conn(0x7f08d415aa20 0x7f08d40ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:25.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d259c700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415a650 0x7f08d40b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:25.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 >> v1:172.21.15.6:6789/0 conn(0x7f08d40b5500 legacy=0x7f08d40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 0x7f08bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08bc004430 0x7f08bc0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415a650 0x7f08bc004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40caf50 con 0x7f08bc003a00 2023-11-28T13:02:25.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40aec70 con 0x7f08bc004430 2023-11-28T13:02:25.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc00ba10 con 0x7f08d415a650 2023-11-28T13:02:25.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d259c700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 0x7f08bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d1d9b700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08bc004430 0x7f08bc0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.825+0000 7f08d2d9d700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415a650 0x7f08bc004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d259c700 1 -- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415a650 msgr2=0x7f08bc004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:25.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d259c700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415a650 0x7f08bc004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d259c700 1 -- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08bc004430 msgr2=0x7f08bc0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:25.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d259c700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08bc004430 0x7f08bc0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d259c700 1 -- 172.21.15.6:0/3428556870 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c40094a0 con 0x7f08bc003a00 2023-11-28T13:02:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d1d9b700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08bc004430 0x7f08bc0048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d259c700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 0x7f08bc003e10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009d30 tx=0x7f08bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a0938aadee4022 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4009e80 con 0x7f08bc003a00 2023-11-28T13:02:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c400e9a0 con 0x7f08bc003a00 2023-11-28T13:02:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d159a700 1 -- 172.21.15.6:0/3428556870 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400ed10 con 0x7f08bc003a00 2023-11-28T13:02:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d8860700 1 -- 172.21.15.6:0/3428556870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 msgr2=0x7f08bc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d8860700 1 --2- 172.21.15.6:0/3428556870 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 0x7f08bc003e10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009d30 tx=0x7f08bc005880 comp rx=0 tx=0).stop 2023-11-28T13:02:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d2d9d700 1 -- 172.21.15.6:0/3428556870 reap_dead start 2023-11-28T13:02:25.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d8860700 1 -- 172.21.15.6:0/3428556870 shutdown_connections 2023-11-28T13:02:25.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.826+0000 7f08d8860700 1 -- 172.21.15.6:0/3428556870 >> 172.21.15.6:0/3428556870 conn(0x7f08d40b1650 msgr2=0x7f08d40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:25.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.827+0000 7f08d8860700 1 -- 172.21.15.6:0/3428556870 shutdown_connections 2023-11-28T13:02:25.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.827+0000 7f08d8860700 1 -- 172.21.15.6:0/3428556870 wait complete. 2023-11-28T13:02:25.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.828+0000 7f08d8860700 1 Processor -- start 2023-11-28T13:02:25.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.828+0000 7f08d8860700 1 -- start start 2023-11-28T13:02:25.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d8860700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 0x7f08d40c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d8860700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08d40cb280 0x7f08d40cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d8860700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415b140 0x7f08d415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d8860700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40cbc30 con 0x7f08bc003a00 2023-11-28T13:02:25.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d8860700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d41f0150 con 0x7f08d40cb280 2023-11-28T13:02:25.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d8860700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d41f0430 con 0x7f08d415b140 2023-11-28T13:02:25.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 0x7f08d40c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 0x7f08d40c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:32994/0 (socket says 172.21.15.6:32994) 2023-11-28T13:02:25.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d259c700 1 -- 172.21.15.6:0/2857569514 learned_addr learned my addr 172.21.15.6:0/2857569514 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:25.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d1d9b700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08d40cb280 0x7f08d40cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d2d9d700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415b140 0x7f08d415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d259c700 1 -- 172.21.15.6:0/2857569514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415b140 msgr2=0x7f08d415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d259c700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415b140 0x7f08d415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d259c700 1 -- 172.21.15.6:0/2857569514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08d40cb280 msgr2=0x7f08d40cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:25.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d259c700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08d40cb280 0x7f08d40cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:25.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d259c700 1 -- 172.21.15.6:0/2857569514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d40cabe0 con 0x7f08bc003a00 2023-11-28T13:02:25.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08d259c700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 0x7f08d40c5f50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f08bc005690 tx=0x7f08c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:25.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.829+0000 7f08c2ffd700 1 -- 172.21.15.6:0/2857569514 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4009e80 con 0x7f08bc003a00 2023-11-28T13:02:25.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.830+0000 7f08c2ffd700 1 -- 172.21.15.6:0/2857569514 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c4028450 con 0x7f08bc003a00 2023-11-28T13:02:25.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.830+0000 7f08d8860700 1 -- 172.21.15.6:0/2857569514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d40af1f0 con 0x7f08bc003a00 2023-11-28T13:02:25.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.830+0000 7f08c2ffd700 1 -- 172.21.15.6:0/2857569514 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4028800 con 0x7f08bc003a00 2023-11-28T13:02:25.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.830+0000 7f08d8860700 1 -- 172.21.15.6:0/2857569514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d40ccbc0 con 0x7f08bc003a00 2023-11-28T13:02:25.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.831+0000 7f08d8860700 1 -- 172.21.15.6:0/2857569514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d406cb30 con 0x7f08bc003a00 2023-11-28T13:02:25.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.831+0000 7f08c2ffd700 1 -- 172.21.15.6:0/2857569514 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f08c4025070 con 0x7f08bc003a00 2023-11-28T13:02:25.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.831+0000 7f08c2ffd700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08b40682f0 0x7f08b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:25.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.831+0000 7f08c2ffd700 1 -- 172.21.15.6:0/2857569514 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f08c4087420 con 0x7f08bc003a00 2023-11-28T13:02:25.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.831+0000 7f08d1d9b700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08b40682f0 0x7f08b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:25.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.832+0000 7f08d1d9b700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08b40682f0 0x7f08b406a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f08d40cbf40 tx=0x7f08c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:25.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:25.836+0000 7f08c2ffd700 1 -- 172.21.15.6:0/2857569514 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f08c4053d70 con 0x7f08bc003a00 2023-11-28T13:02:26.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.071+0000 7f08d8860700 1 -- 172.21.15.6:0/2857569514 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08d40675a0 con 0x7f08bc003a00 2023-11-28T13:02:26.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.071+0000 7f08c2ffd700 1 -- 172.21.15.6:0/2857569514 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f08c4017de0 con 0x7f08bc003a00 2023-11-28T13:02:26.073 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:26.073 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4026920958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4026920958},{"type":"v1","addr":"172.21.15.61:6833","nonce":4026920958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4967,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2212733196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2212733196},{"type":"v1","addr":"172.21.15.61:6835","nonce":2212733196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1674709721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1674709721},{"type":"v1","addr":"172.21.15.6:6836","nonce":1674709721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5937,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/1633618731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":1633618731},{"type":"v1","addr":"172.21.15.6:6837","nonce":1633618731}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-11-28T13:02:26.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.073+0000 7f08d8860700 1 -- 172.21.15.6:0/2857569514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08b40682f0 msgr2=0x7f08b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:26.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.073+0000 7f08d8860700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08b40682f0 0x7f08b406a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f08d40cbf40 tx=0x7f08c8008040 comp rx=0 tx=0).stop 2023-11-28T13:02:26.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.073+0000 7f08d8860700 1 -- 172.21.15.6:0/2857569514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 msgr2=0x7f08d40c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:26.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.073+0000 7f08d8860700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 0x7f08d40c5f50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f08bc005690 tx=0x7f08c40047c0 comp rx=0 tx=0).stop 2023-11-28T13:02:26.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.073+0000 7f08d8860700 1 -- 172.21.15.6:0/2857569514 shutdown_connections 2023-11-28T13:02:26.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.073+0000 7f08d8860700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08b40682f0 0x7f08b406a7a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:26.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.073+0000 7f08d8860700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f08d415b140 0x7f08d415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:26.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.073+0000 7f08d8860700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f08bc003a00 0x7f08d40c5f50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:26.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.073+0000 7f08d8860700 1 --2- 172.21.15.6:0/2857569514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f08d40cb280 0x7f08d40cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:26.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.073+0000 7f08d8860700 1 -- 172.21.15.6:0/2857569514 >> 172.21.15.6:0/2857569514 conn(0x7f08d40b1650 msgr2=0x7f08d4157b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:26.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.074+0000 7f08d8860700 1 -- 172.21.15.6:0/2857569514 shutdown_connections 2023-11-28T13:02:26.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.074+0000 7f08d8860700 1 -- 172.21.15.6:0/2857569514 wait complete. 2023-11-28T13:02:26.076 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 18 2023-11-28T13:02:26.087 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:02:26.087 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T13:02:26.087 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:02:26.087 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:02:26.088 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:02:26.088 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T13:02:26.088 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:02:26.088 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:02:26.088 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:02:26.088 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T13:02:26.089 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:02:26.089 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:02:26.089 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:02:26.089 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T13:02:26.089 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:02:26.089 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:02:26.097 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:02:26.097 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:02:26.098 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:02:26.098 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:02:26.100 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:02:26.100 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:02:26.101 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:02:26.101 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:02:26.140 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:02:26.140 DEBUG:teuthology.parallel:result is None 2023-11-28T13:02:26.140 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:02:26.141 DEBUG:teuthology.parallel:result is None 2023-11-28T13:02:26.143 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:02:26.143 DEBUG:teuthology.parallel:result is None 2023-11-28T13:02:26.143 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:02:26.144 DEBUG:teuthology.parallel:result is None 2023-11-28T13:02:26.144 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T13:02:26.222 INFO:tasks.ceph.mds.c.smithi006.stdout:starting mds.c at 2023-11-28T13:02:26.222 INFO:tasks.ceph.mds.a.smithi006.stdout:starting mds.a at 2023-11-28T13:02:26.231 INFO:tasks.ceph.mds.d.smithi061.stdout:starting mds.d at 2023-11-28T13:02:26.238 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2023-11-28T13:02:26.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.280+0000 7f1f0fcf2700 1 Processor -- start 2023-11-28T13:02:26.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.280+0000 7f1f0fcf2700 1 -- start start 2023-11-28T13:02:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.281+0000 7f1f0fcf2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1f0815c640 0x7f1f080cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.281+0000 7f1f0fcf2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1f080d2820 0x7f1f080b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.281+0000 7f1f0fcf2700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f080d1470 con 0x7f1f0806bf30 2023-11-28T13:02:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.281+0000 7f1f0fcf2700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0815ca10 con 0x7f1f080d2820 2023-11-28T13:02:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.281+0000 7f1f0fcf2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f080cd4e0 con 0x7f1f0815c640 2023-11-28T13:02:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.281+0000 7f1f0d28d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1f080d2820 0x7f1f080b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.281+0000 7f1f0e28f700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1f0806bf30 0x7f1f0806c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48582/0 (socket says 172.21.15.6:48582) 2023-11-28T13:02:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.281+0000 7f1f0d28d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1f080d2820 0x7f1f080b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33026/0 (socket says 172.21.15.6:33026) 2023-11-28T13:02:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.281+0000 7f1f0e28f700 1 -- 172.21.15.6:0/2593948593 learned_addr learned my addr 172.21.15.6:0/2593948593 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.282+0000 7f1f0da8e700 1 --2- 172.21.15.6:0/2593948593 >> v2:172.21.15.61:3300/0 conn(0x7f1f0815c640 0x7f1f080cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.282+0000 7f1f0d28d700 1 -- 172.21.15.6:0/2593948593 >> v2:172.21.15.61:3300/0 conn(0x7f1f0815c640 msgr2=0x7f1f080cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.282+0000 7f1f0d28d700 1 --2- 172.21.15.6:0/2593948593 >> v2:172.21.15.61:3300/0 conn(0x7f1f0815c640 0x7f1f080cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:26.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.282+0000 7f1f0d28d700 1 -- 172.21.15.6:0/2593948593 >> v1:172.21.15.6:6789/0 conn(0x7f1f0806bf30 legacy=0x7f1f0806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:26.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.282+0000 7f1f0d28d700 1 -- 172.21.15.6:0/2593948593 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f080cd7c0 con 0x7f1f080d2820 2023-11-28T13:02:26.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.282+0000 7f1f0ca8c700 1 -- 172.21.15.6:0/2593948593 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897108804 0 0) 0x7f1f080d1470 con 0x7f1f0806bf30 2023-11-28T13:02:26.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.282+0000 7f1f0d28d700 1 --2- 172.21.15.6:0/2593948593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f080d2820 0x7f1f080b5c40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1f0806d420 tx=0x7f1ef8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327c54d5375a7d4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:26.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.282+0000 7f1f0ca8c700 1 -- 172.21.15.6:0/2593948593 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8011830 con 0x7f1f080d2820 2023-11-28T13:02:26.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.282+0000 7f1f0ca8c700 1 -- 172.21.15.6:0/2593948593 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef8011990 con 0x7f1f080d2820 2023-11-28T13:02:26.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.282+0000 7f1f0ca8c700 1 -- 172.21.15.6:0/2593948593 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8011d20 con 0x7f1f080d2820 2023-11-28T13:02:26.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.283+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/2593948593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f080d2820 msgr2=0x7f1f080b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:26.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.283+0000 7f1f0fcf2700 1 --2- 172.21.15.6:0/2593948593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f080d2820 0x7f1f080b5c40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1f0806d420 tx=0x7f1ef8009b20 comp rx=0 tx=0).stop 2023-11-28T13:02:26.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.283+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/2593948593 shutdown_connections 2023-11-28T13:02:26.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.283+0000 7f1f0fcf2700 1 --2- 172.21.15.6:0/2593948593 >> v2:172.21.15.61:3300/0 conn(0x7f1f0815c640 0x7f1f080cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:26.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.283+0000 7f1f0fcf2700 1 --2- 172.21.15.6:0/2593948593 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f080d2820 0x7f1f080b5c40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:26.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.283+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/2593948593 >> 172.21.15.6:0/2593948593 conn(0x7f1f080d2df0 msgr2=0x7f1f080d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:26.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.283+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/2593948593 shutdown_connections 2023-11-28T13:02:26.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.283+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/2593948593 wait complete. 2023-11-28T13:02:26.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.284+0000 7f1f0fcf2700 1 Processor -- start 2023-11-28T13:02:26.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.284+0000 7f1f0fcf2700 1 -- start start 2023-11-28T13:02:26.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0fcf2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f0806bf30 0x7f1f080ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:26.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0fcf2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1f080d2820 0x7f1f080b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:26.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0fcf2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f0815c640 0x7f1f080cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:26.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0fcf2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080cae10 con 0x7f1f0806bf30 2023-11-28T13:02:26.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0fcf2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0814b1f0 con 0x7f1f0815c640 2023-11-28T13:02:26.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0fcf2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080cca50 con 0x7f1f080d2820 2023-11-28T13:02:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0da8e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f0806bf30 0x7f1f080ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0da8e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f0806bf30 0x7f1f080ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33032/0 (socket says 172.21.15.6:33032) 2023-11-28T13:02:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0da8e700 1 -- 172.21.15.6:0/700671382 learned_addr learned my addr 172.21.15.6:0/700671382 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0e28f700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f0815c640 0x7f1f080cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:26.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0d28d700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1f080d2820 0x7f1f080b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0da8e700 1 -- 172.21.15.6:0/700671382 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1f080d2820 msgr2=0x7f1f080b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0da8e700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1f080d2820 0x7f1f080b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0da8e700 1 -- 172.21.15.6:0/700671382 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f0815c640 msgr2=0x7f1f080cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0da8e700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f0815c640 0x7f1f080cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:26.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0da8e700 1 -- 172.21.15.6:0/700671382 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef80094a0 con 0x7f1f0806bf30 2023-11-28T13:02:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0e28f700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f0815c640 0x7f1f080cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0d28d700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1f080d2820 0x7f1f080b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1f0da8e700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f0806bf30 0x7f1f080ca8d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1f00007ef0 tx=0x7f1f0000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.285+0000 7f1efe7fc700 1 -- 172.21.15.6:0/700671382 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00014420 con 0x7f1f0806bf30 2023-11-28T13:02:26.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.286+0000 7f1efe7fc700 1 -- 172.21.15.6:0/700671382 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f00014580 con 0x7f1f0806bf30 2023-11-28T13:02:26.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.286+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/700671382 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f080cf580 con 0x7f1f0806bf30 2023-11-28T13:02:26.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.286+0000 7f1efe7fc700 1 -- 172.21.15.6:0/700671382 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00014930 con 0x7f1f0806bf30 2023-11-28T13:02:26.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.286+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/700671382 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f080d1710 con 0x7f1f0806bf30 2023-11-28T13:02:26.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.286+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/700671382 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f080cdaa0 con 0x7f1f0806bf30 2023-11-28T13:02:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.287+0000 7f1efe7fc700 1 -- 172.21.15.6:0/700671382 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1f00012050 con 0x7f1f0806bf30 2023-11-28T13:02:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.287+0000 7f1efe7fc700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1eec0682f0 0x7f1eec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.287+0000 7f1efe7fc700 1 -- 172.21.15.6:0/700671382 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f1f00086210 con 0x7f1f0806bf30 2023-11-28T13:02:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.287+0000 7f1f0d28d700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1eec0682f0 0x7f1eec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:26.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.288+0000 7f1f0d28d700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1eec0682f0 0x7f1eec06a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1f0806b4d0 tx=0x7f1ef8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:26.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.292+0000 7f1efe7fc700 1 -- 172.21.15.6:0/700671382 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1f00052c30 con 0x7f1f0806bf30 2023-11-28T13:02:26.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:26.501+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/700671382 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1f0814b540 con 0x7f1f0806bf30 2023-11-28T13:02:26.774 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:02:26.777 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:02:27.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.279+0000 7f1efe7fc700 1 -- 172.21.15.6:0/700671382 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1f00056250 con 0x7f1f0806bf30 2023-11-28T13:02:27.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/700671382 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1eec0682f0 msgr2=0x7f1eec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1eec0682f0 0x7f1eec06a7a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1f0806b4d0 tx=0x7f1ef8008030 comp rx=0 tx=0).stop 2023-11-28T13:02:27.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/700671382 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f0806bf30 msgr2=0x7f1f080ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f0806bf30 0x7f1f080ca8d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1f00007ef0 tx=0x7f1f0000e3c0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/700671382 shutdown_connections 2023-11-28T13:02:27.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f0815c640 0x7f1f080cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1f0806bf30 0x7f1f080ca8d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1eec0682f0 0x7f1eec06a7a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 --2- 172.21.15.6:0/700671382 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1f080d2820 0x7f1f080b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/700671382 >> 172.21.15.6:0/700671382 conn(0x7f1f080d2df0 msgr2=0x7f1f080b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:27.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/700671382 shutdown_connections 2023-11-28T13:02:27.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.282+0000 7f1f0fcf2700 1 -- 172.21.15.6:0/700671382 wait complete. 2023-11-28T13:02:27.285 INFO:teuthology.orchestra.run.smithi006.stderr: removed all blocklist entries 2023-11-28T13:02:27.298 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T13:02:27.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.429+0000 7fdb8acd3700 1 Processor -- start 2023-11-28T13:02:27.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.429+0000 7fdb8acd3700 1 -- start start 2023-11-28T13:02:27.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb8acd3700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdb840b4fe0 0x7fdb8415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb8acd3700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8406abb0 con 0x7fdb840cd4a0 2023-11-28T13:02:27.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb8acd3700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8406a280 con 0x7fdb8415a230 2023-11-28T13:02:27.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb8acd3700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8415a600 con 0x7fdb840b4fe0 2023-11-28T13:02:27.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb83fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdb840b4fe0 0x7fdb8415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb89270700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fdb840cd4a0 0x7fdb840d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48600/0 (socket says 172.21.15.6:48600) 2023-11-28T13:02:27.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb89270700 1 -- 172.21.15.6:0/58688137 learned_addr learned my addr 172.21.15.6:0/58688137 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:27.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb83fff700 1 -- 172.21.15.6:0/58688137 >> v1:172.21.15.61:6789/0 conn(0x7fdb8415a230 legacy=0x7fdb840b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:27.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb83fff700 1 -- 172.21.15.6:0/58688137 >> v1:172.21.15.6:6789/0 conn(0x7fdb840cd4a0 legacy=0x7fdb840d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb83fff700 1 -- 172.21.15.6:0/58688137 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8406a560 con 0x7fdb840b4fe0 2023-11-28T13:02:27.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.430+0000 7fdb837fe700 1 -- 172.21.15.6:0/58688137 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403064098 0 0) 0x7fdb8406abb0 con 0x7fdb840cd4a0 2023-11-28T13:02:27.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb83fff700 1 --2- 172.21.15.6:0/58688137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840b4fe0 0x7fdb8415d6b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009840 tx=0x7fdb70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7f20c21e83c321 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:27.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb837fe700 1 -- 172.21.15.6:0/58688137 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70004230 con 0x7fdb840b4fe0 2023-11-28T13:02:27.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb837fe700 1 -- 172.21.15.6:0/58688137 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb70004390 con 0x7fdb840b4fe0 2023-11-28T13:02:27.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb837fe700 1 -- 172.21.15.6:0/58688137 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb700036a0 con 0x7fdb840b4fe0 2023-11-28T13:02:27.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb8acd3700 1 -- 172.21.15.6:0/58688137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840b4fe0 msgr2=0x7fdb8415d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb8acd3700 1 --2- 172.21.15.6:0/58688137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840b4fe0 0x7fdb8415d6b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009840 tx=0x7fdb70009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:27.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb8acd3700 1 -- 172.21.15.6:0/58688137 shutdown_connections 2023-11-28T13:02:27.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb8acd3700 1 --2- 172.21.15.6:0/58688137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840b4fe0 0x7fdb8415d6b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb8acd3700 1 -- 172.21.15.6:0/58688137 >> 172.21.15.6:0/58688137 conn(0x7fdb840cf8d0 msgr2=0x7fdb840cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:27.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb8acd3700 1 -- 172.21.15.6:0/58688137 shutdown_connections 2023-11-28T13:02:27.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.431+0000 7fdb8acd3700 1 -- 172.21.15.6:0/58688137 wait complete. 2023-11-28T13:02:27.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.432+0000 7fdb8acd3700 1 Processor -- start 2023-11-28T13:02:27.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.433+0000 7fdb8acd3700 1 -- start start 2023-11-28T13:02:27.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.433+0000 7fdb8acd3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb840b4fe0 0x7fdb840c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.433+0000 7fdb8acd3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840cd4a0 0x7fdb841578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.433+0000 7fdb8acd3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb8415a230 0x7fdb840b8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.433+0000 7fdb8acd3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb840c7ec0 con 0x7fdb840cd4a0 2023-11-28T13:02:27.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.433+0000 7fdb8acd3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb840c7240 con 0x7fdb8415a230 2023-11-28T13:02:27.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.433+0000 7fdb8acd3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb840c7520 con 0x7fdb840b4fe0 2023-11-28T13:02:27.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb83fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840cd4a0 0x7fdb841578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb83fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840cd4a0 0x7fdb841578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33052/0 (socket says 172.21.15.6:33052) 2023-11-28T13:02:27.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb83fff700 1 -- 172.21.15.6:0/4056348882 learned_addr learned my addr 172.21.15.6:0/4056348882 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:27.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb89270700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb8415a230 0x7fdb840b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb88a6f700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb840b4fe0 0x7fdb840c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb83fff700 1 -- 172.21.15.6:0/4056348882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb840b4fe0 msgr2=0x7fdb840c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb83fff700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb840b4fe0 0x7fdb840c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb83fff700 1 -- 172.21.15.6:0/4056348882 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb8415a230 msgr2=0x7fdb840b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb83fff700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb8415a230 0x7fdb840b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb83fff700 1 -- 172.21.15.6:0/4056348882 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb700094a0 con 0x7fdb840cd4a0 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb83fff700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840cd4a0 0x7fdb841578f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdb700090b0 tx=0x7fdb70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb817fa700 1 -- 172.21.15.6:0/4056348882 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70011830 con 0x7fdb840cd4a0 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb817fa700 1 -- 172.21.15.6:0/4056348882 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7001e070 con 0x7fdb840cd4a0 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb8acd3700 1 -- 172.21.15.6:0/4056348882 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb840cd8c0 con 0x7fdb840cd4a0 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb817fa700 1 -- 172.21.15.6:0/4056348882 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70011bb0 con 0x7fdb840cd4a0 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.434+0000 7fdb8acd3700 1 -- 172.21.15.6:0/4056348882 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8406a840 con 0x7fdb840cd4a0 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.435+0000 7fdb8acd3700 1 -- 172.21.15.6:0/4056348882 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb840c8000 con 0x7fdb840cd4a0 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.436+0000 7fdb817fa700 1 -- 172.21.15.6:0/4056348882 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdb70017020 con 0x7fdb840cd4a0 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.436+0000 7fdb817fa700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdb680682f0 0x7fdb6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.436+0000 7fdb817fa700 1 -- 172.21.15.6:0/4056348882 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fdb70086ea0 con 0x7fdb840cd4a0 2023-11-28T13:02:27.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.436+0000 7fdb88a6f700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdb680682f0 0x7fdb6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.437+0000 7fdb88a6f700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdb680682f0 0x7fdb6806a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdb840d1ed0 tx=0x7fdb78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:27.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.441+0000 7fdb817fa700 1 -- 172.21.15.6:0/4056348882 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdb840c8000 con 0x7fdb840cd4a0 2023-11-28T13:02:27.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.710+0000 7fdb8acd3700 1 -- 172.21.15.6:0/4056348882 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdb840c8000 con 0x7fdb840cd4a0 2023-11-28T13:02:27.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.710+0000 7fdb817fa700 1 -- 172.21.15.6:0/4056348882 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fdb840c8000 con 0x7fdb840cd4a0 2023-11-28T13:02:27.712 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:27.712 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:02:27.712 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_dump": [ 2023-11-28T13:02:27.712 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.712 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.a", 2023-11-28T13:02:27.712 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVl8nvUCxAA8XF3IdK1G4diiM0tQ5npVA==", 2023-11-28T13:02:27.712 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.712 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:02:27.712 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.b", 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlN9+EGBAA8WROC+gNBWJbQmf9ch69iA==", 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:02:27.713 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.c", 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVl/bQ5ExAAerGXuTwAj5uC6cL7ScyCZA==", 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.714 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.d", 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlo58FIBAAMZ4QjC/4ODXdO7T8j7SEYA==", 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.715 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.0", 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA15GVlma1kARAA50lI2mIZlVoGT0ks8qcvzw==", 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.1", 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA45GVl4YRiARAAZ7Qktx0lxuvP0H4ceAeVig==", 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:27.716 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.2", 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA75GVlqfAkARAAFyiIXHxyw6s3bPBB2+KINA==", 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.717 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.3", 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA+5GVlkjURARAANkhMOAGKdlDfEF9L29hxgA==", 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.718 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.4", 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBE5GVlNMQ9BxAA/gFfGDxTZ9TNAMQcCV2k9w==", 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.5", 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBH5GVl91Q8BxAAlVlRbZqylgmq9LkQp/O6jA==", 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:27.719 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.6", 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBK5GVlDoTjBhAAOXrgM8Q5MfN3I3aMZgjVvw==", 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.720 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.7", 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBN5GVlFzINBxAAN6E1A938lHTLYA1GamwdwQ==", 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.0", 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlRYR3JBAADaTghk6Ix3IQVes6gTE56A==", 2023-11-28T13:02:27.721 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow r", 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.1", 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVloippKxAAsZK5iYerb15VlfiZ4svbdg==", 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow r", 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow rw", 2023-11-28T13:02:27.722 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rwx" 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.2", 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlDlnhJxAAQG5yCtKJUjh6KGJ3Ennxjg==", 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow r", 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow rw", 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rwx" 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.723 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.3", 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlbJU6MRAAd4WTJLkmaaL+ATXGlIH6ig==", 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow r", 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow rw", 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rwx" 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.admin", 2023-11-28T13:02:27.724 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAw5GVlXWKtKRAAbAVVrygFCntIL9ljdMCXbg==", 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow *", 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mgr.x", 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlIRViCBAADpzuU8y4HYepMLT1WkF08g==", 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow *", 2023-11-28T13:02:27.725 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile mgr", 2023-11-28T13:02:27.726 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.726 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.726 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:27.726 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:27.726 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mgr.y", 2023-11-28T13:02:27.726 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVl4uVeBBAACyO9UoUP+RyJxFxE8rsFoQ==", 2023-11-28T13:02:27.726 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:02:27.726 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow *", 2023-11-28T13:02:27.726 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile mgr", 2023-11-28T13:02:27.727 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:02:27.727 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.727 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:27.727 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-28T13:02:27.727 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:02:27.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.712+0000 7fdb8acd3700 1 -- 172.21.15.6:0/4056348882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdb680682f0 msgr2=0x7fdb6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.712+0000 7fdb8acd3700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdb680682f0 0x7fdb6806a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdb840d1ed0 tx=0x7fdb78008040 comp rx=0 tx=0).stop 2023-11-28T13:02:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.712+0000 7fdb8acd3700 1 -- 172.21.15.6:0/4056348882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840cd4a0 msgr2=0x7fdb841578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.712+0000 7fdb8acd3700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840cd4a0 0x7fdb841578f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdb700090b0 tx=0x7fdb70004750 comp rx=0 tx=0).stop 2023-11-28T13:02:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.712+0000 7fdb8acd3700 1 -- 172.21.15.6:0/4056348882 shutdown_connections 2023-11-28T13:02:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.712+0000 7fdb8acd3700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdb8415a230 0x7fdb840b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.712+0000 7fdb8acd3700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdb680682f0 0x7fdb6806a7a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.712+0000 7fdb8acd3700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdb840b4fe0 0x7fdb840c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.712+0000 7fdb8acd3700 1 --2- 172.21.15.6:0/4056348882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb840cd4a0 0x7fdb841578f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.712+0000 7fdb8acd3700 1 -- 172.21.15.6:0/4056348882 >> 172.21.15.6:0/4056348882 conn(0x7fdb840cf8d0 msgr2=0x7fdb8415de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.713+0000 7fdb8acd3700 1 -- 172.21.15.6:0/4056348882 shutdown_connections 2023-11-28T13:02:27.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.713+0000 7fdb8acd3700 1 -- 172.21.15.6:0/4056348882 wait complete. 2023-11-28T13:02:27.730 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:27.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.853+0000 7f8ed88a4700 1 Processor -- start 2023-11-28T13:02:27.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.853+0000 7f8ed88a4700 1 -- start start 2023-11-28T13:02:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed88a4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ed415c990 0x7f8ed40cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed88a4700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8ed40cc310 0x7f8ed40cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed88a4700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed40ce340 con 0x7f8ed4161090 2023-11-28T13:02:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed88a4700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed40cebe0 con 0x7f8ed40cc310 2023-11-28T13:02:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed88a4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40cccf0 con 0x7f8ed415c990 2023-11-28T13:02:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed2d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8ed40cc310 0x7f8ed40cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed1d9b700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f8ed4161090 0x7f8ed40ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48612/0 (socket says 172.21.15.6:48612) 2023-11-28T13:02:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed1d9b700 1 -- 172.21.15.6:0/2065435394 learned_addr learned my addr 172.21.15.6:0/2065435394 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed259c700 1 --2- 172.21.15.6:0/2065435394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ed415c990 0x7f8ed40cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed2d9d700 1 -- 172.21.15.6:0/2065435394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ed415c990 msgr2=0x7f8ed40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed2d9d700 1 --2- 172.21.15.6:0/2065435394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ed415c990 0x7f8ed40cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed2d9d700 1 -- 172.21.15.6:0/2065435394 >> v1:172.21.15.6:6789/0 conn(0x7f8ed4161090 legacy=0x7f8ed40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed2d9d700 1 -- 172.21.15.6:0/2065435394 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed40cb3f0 con 0x7f8ed40cc310 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.854+0000 7f8ed159a700 1 -- 172.21.15.6:0/2065435394 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886553861 0 0) 0x7f8ed40ce340 con 0x7f8ed4161090 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed2d9d700 1 --2- 172.21.15.6:0/2065435394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed40cc310 0x7f8ed40cc7b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009d90 tx=0x7f8ecc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9a9425487e73c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed159a700 1 -- 172.21.15.6:0/2065435394 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc00e070 con 0x7f8ed40cc310 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed159a700 1 -- 172.21.15.6:0/2065435394 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc00bb20 con 0x7f8ed40cc310 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed159a700 1 -- 172.21.15.6:0/2065435394 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc012430 con 0x7f8ed40cc310 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed88a4700 1 -- 172.21.15.6:0/2065435394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed40cc310 msgr2=0x7f8ed40cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed88a4700 1 --2- 172.21.15.6:0/2065435394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed40cc310 0x7f8ed40cc7b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009d90 tx=0x7f8ecc00b330 comp rx=0 tx=0).stop 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed88a4700 1 -- 172.21.15.6:0/2065435394 shutdown_connections 2023-11-28T13:02:27.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed88a4700 1 --2- 172.21.15.6:0/2065435394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed40cc310 0x7f8ed40cc7b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed88a4700 1 --2- 172.21.15.6:0/2065435394 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ed415c990 0x7f8ed40cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed88a4700 1 -- 172.21.15.6:0/2065435394 >> 172.21.15.6:0/2065435394 conn(0x7f8ed40b1650 msgr2=0x7f8ed40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed88a4700 1 -- 172.21.15.6:0/2065435394 shutdown_connections 2023-11-28T13:02:27.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.855+0000 7f8ed88a4700 1 -- 172.21.15.6:0/2065435394 wait complete. 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.857+0000 7f8ed88a4700 1 Processor -- start 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.857+0000 7f8ed88a4700 1 -- start start 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed88a4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ed40cc310 0x7f8ed4150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed88a4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed415c990 0x7f8ed414abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed88a4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ed4161090 0x7f8ed414b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed88a4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40b4e60 con 0x7f8ed415c990 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed88a4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40b52f0 con 0x7f8ed40cc310 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed88a4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40b4530 con 0x7f8ed4161090 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed415c990 0x7f8ed414abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed415c990 0x7f8ed414abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33066/0 (socket says 172.21.15.6:33066) 2023-11-28T13:02:27.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed1d9b700 1 -- 172.21.15.6:0/144221028 learned_addr learned my addr 172.21.15.6:0/144221028 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed2d9d700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ed4161090 0x7f8ed414b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed259c700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ed40cc310 0x7f8ed4150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed1d9b700 1 -- 172.21.15.6:0/144221028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ed4161090 msgr2=0x7f8ed414b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed1d9b700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ed4161090 0x7f8ed414b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed1d9b700 1 -- 172.21.15.6:0/144221028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ed40cc310 msgr2=0x7f8ed4150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed1d9b700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ed40cc310 0x7f8ed4150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed1d9b700 1 -- 172.21.15.6:0/144221028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc009a70 con 0x7f8ed415c990 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed1d9b700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed415c990 0x7f8ed414abe0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400c540 tx=0x7f8ec400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ec2ffd700 1 -- 172.21.15.6:0/144221028 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4004c80 con 0x7f8ed415c990 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ec2ffd700 1 -- 172.21.15.6:0/144221028 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec4004de0 con 0x7f8ed415c990 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed88a4700 1 -- 172.21.15.6:0/144221028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed40ce870 con 0x7f8ed415c990 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ec2ffd700 1 -- 172.21.15.6:0/144221028 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4007310 con 0x7f8ed415c990 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.858+0000 7f8ed88a4700 1 -- 172.21.15.6:0/144221028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed40ce590 con 0x7f8ed415c990 2023-11-28T13:02:27.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.859+0000 7f8ed88a4700 1 -- 172.21.15.6:0/144221028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed40cb080 con 0x7f8ed415c990 2023-11-28T13:02:27.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.860+0000 7f8ec2ffd700 1 -- 172.21.15.6:0/144221028 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ec401a020 con 0x7f8ed415c990 2023-11-28T13:02:27.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.860+0000 7f8ec2ffd700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8eb40682f0 0x7f8eb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:27.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.860+0000 7f8ec2ffd700 1 -- 172.21.15.6:0/144221028 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f8ec4085700 con 0x7f8ed415c990 2023-11-28T13:02:27.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.860+0000 7f8ed259c700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8eb40682f0 0x7f8eb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:27.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.861+0000 7f8ed259c700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8eb40682f0 0x7f8eb406a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009b70 tx=0x7f8ec8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:27.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:27.865+0000 7f8ec2ffd700 1 -- 172.21.15.6:0/144221028 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ec4052170 con 0x7f8ed415c990 2023-11-28T13:02:28.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.060+0000 7f8ed88a4700 1 -- 172.21.15.6:0/144221028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ed40675a0 con 0x7f8ed415c990 2023-11-28T13:02:28.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.061+0000 7f8ec2ffd700 1 -- 172.21.15.6:0/144221028 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+14318 (secure 0 0 0) 0x7f8ec4055790 con 0x7f8ed415c990 2023-11-28T13:02:28.063 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:28.063 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":50,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:27.269054+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 -- 172.21.15.6:0/144221028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8eb40682f0 msgr2=0x7f8eb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8eb40682f0 0x7f8eb406a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009b70 tx=0x7f8ec8008040 comp rx=0 tx=0).stop 2023-11-28T13:02:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 -- 172.21.15.6:0/144221028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed415c990 msgr2=0x7f8ed414abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed415c990 0x7f8ed414abe0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400c540 tx=0x7f8ec400c900 comp rx=0 tx=0).stop 2023-11-28T13:02:28.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 -- 172.21.15.6:0/144221028 shutdown_connections 2023-11-28T13:02:28.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8eb40682f0 0x7f8eb406a7a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8ed4161090 0x7f8ed414b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ed40cc310 0x7f8ed4150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 --2- 172.21.15.6:0/144221028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ed415c990 0x7f8ed414abe0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 -- 172.21.15.6:0/144221028 >> 172.21.15.6:0/144221028 conn(0x7f8ed40b1650 msgr2=0x7f8ed406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:28.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 -- 172.21.15.6:0/144221028 shutdown_connections 2023-11-28T13:02:28.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.063+0000 7f8ed88a4700 1 -- 172.21.15.6:0/144221028 wait complete. 2023-11-28T13:02:28.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:02:28.077 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:02:28.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.206+0000 7f951603b700 1 Processor -- start 2023-11-28T13:02:28.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.206+0000 7f951603b700 1 -- start start 2023-11-28T13:02:28.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f951603b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9510165670 0x7f9510167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f951603b700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95100ce550 con 0x7f95100b1630 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f951603b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95100ce830 con 0x7f95100cf5f0 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f951603b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95100ceb10 con 0x7f9510165670 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950f7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f95100b1630 0x7f95100b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48624/0 (socket says 172.21.15.6:48624) 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950f7fe700 1 -- 172.21.15.6:0/619311836 learned_addr learned my addr 172.21.15.6:0/619311836 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950ffff700 1 --2- 172.21.15.6:0/619311836 >> v2:172.21.15.61:3300/0 conn(0x7f9510165670 0x7f9510167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60020998 0 0) 0x7f95100ce550 con 0x7f95100b1630 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94f80035b0 con 0x7f95100b1630 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3513916519 0 0) 0x7f94f80035b0 con 0x7f95100b1630 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 >> v2:172.21.15.61:3300/0 conn(0x7f9510165670 msgr2=0x7f9510167a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 --2- 172.21.15.6:0/619311836 >> v2:172.21.15.61:3300/0 conn(0x7f9510165670 0x7f9510167a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 >> v1:172.21.15.61:6789/0 conn(0x7f95100cf5f0 legacy=0x7f95100caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95100cedf0 con 0x7f95100b1630 2023-11-28T13:02:28.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f9500001670 con 0x7f95100b1630 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 >> v1:172.21.15.6:6789/0 conn(0x7f95100b1630 legacy=0x7f95100b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f94f8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f94f8004850 0x7f94f8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f94f8005200 0x7f94f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95100ceb10 con 0x7f94f8003a00 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f800c080 con 0x7f94f8005200 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f800c240 con 0x7f94f8004850 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.207+0000 7f950f7fe700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f94f8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950effd700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f94f8004850 0x7f94f8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950ffff700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f94f8005200 0x7f94f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950f7fe700 1 -- 172.21.15.6:0/619311836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f94f8004850 msgr2=0x7f94f8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:28.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950f7fe700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f94f8004850 0x7f94f8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950f7fe700 1 -- 172.21.15.6:0/619311836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f94f8005200 msgr2=0x7f94f800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950f7fe700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f94f8005200 0x7f94f800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950f7fe700 1 -- 172.21.15.6:0/619311836 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f80035b0 con 0x7f94f8003a00 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950ffff700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f94f8005200 0x7f94f800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950ffff700 1 -- 172.21.15.6:0/619311836 reap_dead start 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950f7fe700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f94f8003e10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f950000c240 tx=0x7f94f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842d1046445cd95b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500007c10 con 0x7f94f8003a00 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950000ce00 con 0x7f94f8003a00 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.208+0000 7f950e7fc700 1 -- 172.21.15.6:0/619311836 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500005ba0 con 0x7f94f8003a00 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.209+0000 7f951603b700 1 -- 172.21.15.6:0/619311836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 msgr2=0x7f94f8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.209+0000 7f951603b700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f94f8003e10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f950000c240 tx=0x7f94f8005880 comp rx=0 tx=0).stop 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.209+0000 7f951603b700 1 -- 172.21.15.6:0/619311836 shutdown_connections 2023-11-28T13:02:28.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.209+0000 7f951603b700 1 --2- 172.21.15.6:0/619311836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f94f8003e10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.209+0000 7f951603b700 1 -- 172.21.15.6:0/619311836 >> 172.21.15.6:0/619311836 conn(0x7f95100cc630 msgr2=0x7f95100cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:28.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.209+0000 7f951603b700 1 -- 172.21.15.6:0/619311836 shutdown_connections 2023-11-28T13:02:28.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.209+0000 7f951603b700 1 -- 172.21.15.6:0/619311836 wait complete. 2023-11-28T13:02:28.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.210+0000 7f951603b700 1 Processor -- start 2023-11-28T13:02:28.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.210+0000 7f951603b700 1 -- start start 2023-11-28T13:02:28.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f951603b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f951014ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f951603b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95100c4f10 0x7f95100c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f951603b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95101584e0 0x7f9510158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f951603b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9510158e90 con 0x7f94f8003a00 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f951603b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95100b29e0 con 0x7f95100c4f10 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f951603b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95100c5990 con 0x7f95101584e0 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f951014ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f951014ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33082/0 (socket says 172.21.15.6:33082) 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950f7fe700 1 -- 172.21.15.6:0/218911906 learned_addr learned my addr 172.21.15.6:0/218911906 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950effd700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95100c4f10 0x7f95100c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950ffff700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95101584e0 0x7f9510158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950f7fe700 1 -- 172.21.15.6:0/218911906 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95101584e0 msgr2=0x7f9510158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950f7fe700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95101584e0 0x7f9510158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950f7fe700 1 -- 172.21.15.6:0/218911906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95100c4f10 msgr2=0x7f95100c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950f7fe700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95100c4f10 0x7f95100c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950f7fe700 1 -- 172.21.15.6:0/218911906 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95100ce140 con 0x7f94f8003a00 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950effd700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95100c4f10 0x7f95100c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f950f7fe700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f951014ff20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f950000c830 tx=0x7f9500001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f94f7fff700 1 -- 172.21.15.6:0/218911906 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500001630 con 0x7f94f8003a00 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.211+0000 7f94f7fff700 1 -- 172.21.15.6:0/218911906 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95000070d0 con 0x7f94f8003a00 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.212+0000 7f951603b700 1 -- 172.21.15.6:0/218911906 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95100cf0d0 con 0x7f94f8003a00 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.212+0000 7f94f7fff700 1 -- 172.21.15.6:0/218911906 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95000074a0 con 0x7f94f8003a00 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.212+0000 7f951603b700 1 -- 172.21.15.6:0/218911906 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95100ce2d0 con 0x7f94f8003a00 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.213+0000 7f951603b700 1 -- 172.21.15.6:0/218911906 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95100739f0 con 0x7f94f8003a00 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.213+0000 7f94f7fff700 1 -- 172.21.15.6:0/218911906 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9500024070 con 0x7f94f8003a00 2023-11-28T13:02:28.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.214+0000 7f94f7fff700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f94fc0682f0 0x7f94fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.214+0000 7f950effd700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f94fc0682f0 0x7f94fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.214+0000 7f94f7fff700 1 -- 172.21.15.6:0/218911906 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f95000869b0 con 0x7f94f8003a00 2023-11-28T13:02:28.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.214+0000 7f950effd700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f94fc0682f0 0x7f94fc06a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f95100aea30 tx=0x7f9504009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:28.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.218+0000 7f94f7fff700 1 -- 172.21.15.6:0/218911906 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f95000533a0 con 0x7f94f8003a00 2023-11-28T13:02:28.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.409+0000 7f951603b700 1 -- 172.21.15.6:0/218911906 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f95100cac80 con 0x7f94f8003a00 2023-11-28T13:02:28.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.409+0000 7f94f7fff700 1 -- 172.21.15.6:0/218911906 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7f95000569c0 con 0x7f94f8003a00 2023-11-28T13:02:28.411 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:02:28.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 -- 172.21.15.6:0/218911906 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f94fc0682f0 msgr2=0x7f94fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f94fc0682f0 0x7f94fc06a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f95100aea30 tx=0x7f9504009040 comp rx=0 tx=0).stop 2023-11-28T13:02:28.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 -- 172.21.15.6:0/218911906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 msgr2=0x7f951014ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f951014ff20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f950000c830 tx=0x7f9500001a00 comp rx=0 tx=0).stop 2023-11-28T13:02:28.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 -- 172.21.15.6:0/218911906 shutdown_connections 2023-11-28T13:02:28.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f94fc0682f0 0x7f94fc06a7a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95101584e0 0x7f9510158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94f8003a00 0x7f951014ff20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 --2- 172.21.15.6:0/218911906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f95100c4f10 0x7f95100c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 -- 172.21.15.6:0/218911906 >> 172.21.15.6:0/218911906 conn(0x7f95100cc630 msgr2=0x7f95100cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:28.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 -- 172.21.15.6:0/218911906 shutdown_connections 2023-11-28T13:02:28.414 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.411+0000 7f951603b700 1 -- 172.21.15.6:0/218911906 wait complete. 2023-11-28T13:02:28.425 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:02:28.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.553+0000 7f879890f700 1 Processor -- start 2023-11-28T13:02:28.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.553+0000 7f879890f700 1 -- start start 2023-11-28T13:02:28.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.554+0000 7f879890f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8794159500 0x7f87940cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879890f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87940b1d40 0x7f87940ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879890f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879415e040 con 0x7f879415dbf0 2023-11-28T13:02:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879890f700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87940cede0 con 0x7f8794159500 2023-11-28T13:02:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879890f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87940ceff0 con 0x7f87940b1d40 2023-11-28T13:02:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879259c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8794159500 0x7f87940cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879259c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8794159500 0x7f87940cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33090/0 (socket says 172.21.15.6:33090) 2023-11-28T13:02:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879259c700 1 -- 172.21.15.6:0/1310984486 learned_addr learned my addr 172.21.15.6:0/1310984486 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879259c700 1 -- 172.21.15.6:0/1310984486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87940b1d40 msgr2=0x7f87940ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:28.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f8792d9d700 1 --2- 172.21.15.6:0/1310984486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87940b1d40 0x7f87940ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879259c700 1 --2- 172.21.15.6:0/1310984486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87940b1d40 0x7f87940ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879259c700 1 -- 172.21.15.6:0/1310984486 >> v1:172.21.15.61:6789/0 conn(0x7f879415dbf0 legacy=0x7f87940b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879259c700 1 -- 172.21.15.6:0/1310984486 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87940cf2d0 con 0x7f8794159500 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879259c700 1 --2- 172.21.15.6:0/1310984486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 0x7f87940cd140 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f878800bb90 tx=0x7f878800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594ed1f2a6396b73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879159a700 1 -- 172.21.15.6:0/1310984486 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878800dd10 con 0x7f8794159500 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879159a700 1 -- 172.21.15.6:0/1310984486 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878800de70 con 0x7f8794159500 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.555+0000 7f879159a700 1 -- 172.21.15.6:0/1310984486 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788004230 con 0x7f8794159500 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.556+0000 7f879890f700 1 -- 172.21.15.6:0/1310984486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 msgr2=0x7f87940cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.556+0000 7f879890f700 1 --2- 172.21.15.6:0/1310984486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 0x7f87940cd140 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f878800bb90 tx=0x7f878800bea0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.556+0000 7f879890f700 1 -- 172.21.15.6:0/1310984486 shutdown_connections 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.556+0000 7f879890f700 1 --2- 172.21.15.6:0/1310984486 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87940b1d40 0x7f87940ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.556+0000 7f879890f700 1 --2- 172.21.15.6:0/1310984486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 0x7f87940cd140 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.556+0000 7f879890f700 1 -- 172.21.15.6:0/1310984486 >> 172.21.15.6:0/1310984486 conn(0x7f87940cc630 msgr2=0x7f87940cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.556+0000 7f879890f700 1 -- 172.21.15.6:0/1310984486 shutdown_connections 2023-11-28T13:02:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.556+0000 7f879890f700 1 -- 172.21.15.6:0/1310984486 wait complete. 2023-11-28T13:02:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.557+0000 7f879890f700 1 Processor -- start 2023-11-28T13:02:28.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.557+0000 7f879890f700 1 -- start start 2023-11-28T13:02:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f879890f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87940b1d40 0x7f87941516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f879890f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 0x7f87941598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f879890f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f879415dbf0 0x7f879415bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f879890f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87940c4610 con 0x7f8794159500 2023-11-28T13:02:28.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f879890f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879415a320 con 0x7f879415dbf0 2023-11-28T13:02:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f879890f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87940c3fc0 con 0x7f87940b1d40 2023-11-28T13:02:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f8791d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 0x7f87941598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f8791d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 0x7f87941598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33098/0 (socket says 172.21.15.6:33098) 2023-11-28T13:02:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f8791d9b700 1 -- 172.21.15.6:0/1390458670 learned_addr learned my addr 172.21.15.6:0/1390458670 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f879259c700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87940b1d40 0x7f87941516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.558+0000 7f8792d9d700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f879415dbf0 0x7f879415bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f8791d9b700 1 -- 172.21.15.6:0/1390458670 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87940b1d40 msgr2=0x7f87941516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f8791d9b700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87940b1d40 0x7f87941516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f8791d9b700 1 -- 172.21.15.6:0/1390458670 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f879415dbf0 msgr2=0x7f879415bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f8791d9b700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f879415dbf0 0x7f879415bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f8791d9b700 1 -- 172.21.15.6:0/1390458670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878800b7f0 con 0x7f8794159500 2023-11-28T13:02:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f8791d9b700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 0x7f87941598d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f878400b450 tx=0x7f878400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:28.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f8782ffd700 1 -- 172.21.15.6:0/1390458670 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784011830 con 0x7f8794159500 2023-11-28T13:02:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f8782ffd700 1 -- 172.21.15.6:0/1390458670 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8784011990 con 0x7f8794159500 2023-11-28T13:02:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f879890f700 1 -- 172.21.15.6:0/1390458670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879415d3f0 con 0x7f8794159500 2023-11-28T13:02:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f8782ffd700 1 -- 172.21.15.6:0/1390458670 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784011d40 con 0x7f8794159500 2023-11-28T13:02:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.559+0000 7f879890f700 1 -- 172.21.15.6:0/1390458670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879415d110 con 0x7f8794159500 2023-11-28T13:02:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.560+0000 7f879890f700 1 -- 172.21.15.6:0/1390458670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879415d6d0 con 0x7f8794159500 2023-11-28T13:02:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.560+0000 7f8782ffd700 1 -- 172.21.15.6:0/1390458670 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f878400f050 con 0x7f8794159500 2023-11-28T13:02:28.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.561+0000 7f8782ffd700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87740682f0 0x7f877406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.561+0000 7f8782ffd700 1 -- 172.21.15.6:0/1390458670 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f8784085450 con 0x7f8794159500 2023-11-28T13:02:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.561+0000 7f879259c700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87740682f0 0x7f877406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.561+0000 7f879259c700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87740682f0 0x7f877406a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f878800b340 tx=0x7f878801d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:28.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.565+0000 7f8782ffd700 1 -- 172.21.15.6:0/1390458670 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8784051ec0 con 0x7f8794159500 2023-11-28T13:02:28.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.759+0000 7f879890f700 1 -- 172.21.15.6:0/1390458670 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8794151d10 con 0x7f8794159500 2023-11-28T13:02:28.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.759+0000 7f8782ffd700 1 -- 172.21.15.6:0/1390458670 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8794151d10 con 0x7f8794159500 2023-11-28T13:02:28.761 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:02:28.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 -- 172.21.15.6:0/1390458670 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87740682f0 msgr2=0x7f877406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87740682f0 0x7f877406a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f878800b340 tx=0x7f878801d040 comp rx=0 tx=0).stop 2023-11-28T13:02:28.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 -- 172.21.15.6:0/1390458670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 msgr2=0x7f87941598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 0x7f87941598d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f878400b450 tx=0x7f878400b760 comp rx=0 tx=0).stop 2023-11-28T13:02:28.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 -- 172.21.15.6:0/1390458670 shutdown_connections 2023-11-28T13:02:28.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f879415dbf0 0x7f879415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f87740682f0 0x7f877406a7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f87940b1d40 0x7f87941516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 --2- 172.21.15.6:0/1390458670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8794159500 0x7f87941598d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 -- 172.21.15.6:0/1390458670 >> 172.21.15.6:0/1390458670 conn(0x7f87940cc630 msgr2=0x7f87940cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:28.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 -- 172.21.15.6:0/1390458670 shutdown_connections 2023-11-28T13:02:28.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.761+0000 7f879890f700 1 -- 172.21.15.6:0/1390458670 wait complete. 2023-11-28T13:02:28.775 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:02:28.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.899+0000 7faf06fee700 1 Processor -- start 2023-11-28T13:02:28.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.899+0000 7faf06fee700 1 -- start start 2023-11-28T13:02:28.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.899+0000 7faf06fee700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faf0015c990 0x7faf000cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.899+0000 7faf06fee700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf000ce340 con 0x7faf000cc310 2023-11-28T13:02:28.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.899+0000 7faf06fee700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf000cebe0 con 0x7faf00161090 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.899+0000 7faf06fee700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf000cb3f0 con 0x7faf0015c990 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faf04d8a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faf0015c990 0x7faf000cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faf0558b700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7faf000cc310 0x7faf000cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48660/0 (socket says 172.21.15.6:48660) 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faf04d8a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faf0015c990 0x7faf000cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33104/0 (socket says 172.21.15.6:33104) 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faf0558b700 1 -- 172.21.15.6:0/1827319895 learned_addr learned my addr 172.21.15.6:0/1827319895 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faf04d8a700 1 -- 172.21.15.6:0/1827319895 >> v1:172.21.15.61:6789/0 conn(0x7faf00161090 legacy=0x7faf000ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faf04d8a700 1 -- 172.21.15.6:0/1827319895 >> v1:172.21.15.6:6789/0 conn(0x7faf000cc310 legacy=0x7faf000cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faf04d8a700 1 -- 172.21.15.6:0/1827319895 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000ce590 con 0x7faf0015c990 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faeff7fe700 1 -- 172.21.15.6:0/1827319895 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364969528 0 0) 0x7faf000ce340 con 0x7faf000cc310 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faf04d8a700 1 --2- 172.21.15.6:0/1827319895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0015c990 0x7faf000cf070 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7faef4009840 tx=0x7faef4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b227c19dda3475 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faeff7fe700 1 -- 172.21.15.6:0/1827319895 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4004230 con 0x7faf0015c990 2023-11-28T13:02:28.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faeff7fe700 1 -- 172.21.15.6:0/1827319895 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef4004390 con 0x7faf0015c990 2023-11-28T13:02:28.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.900+0000 7faeff7fe700 1 -- 172.21.15.6:0/1827319895 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef40036a0 con 0x7faf0015c990 2023-11-28T13:02:28.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.901+0000 7faf06fee700 1 -- 172.21.15.6:0/1827319895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0015c990 msgr2=0x7faf000cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.901+0000 7faf06fee700 1 --2- 172.21.15.6:0/1827319895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0015c990 0x7faf000cf070 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7faef4009840 tx=0x7faef4009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:28.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.901+0000 7faf06fee700 1 -- 172.21.15.6:0/1827319895 shutdown_connections 2023-11-28T13:02:28.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.901+0000 7faf06fee700 1 --2- 172.21.15.6:0/1827319895 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0015c990 0x7faf000cf070 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.901+0000 7faf06fee700 1 -- 172.21.15.6:0/1827319895 >> 172.21.15.6:0/1827319895 conn(0x7faf000b1650 msgr2=0x7faf000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:28.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.901+0000 7faf06fee700 1 -- 172.21.15.6:0/1827319895 shutdown_connections 2023-11-28T13:02:28.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.901+0000 7faf06fee700 1 -- 172.21.15.6:0/1827319895 wait complete. 2023-11-28T13:02:28.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.902+0000 7faf06fee700 1 Processor -- start 2023-11-28T13:02:28.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.902+0000 7faf06fee700 1 -- start start 2023-11-28T13:02:28.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf06fee700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf000cc310 0x7faf0015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf06fee700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf0015c990 0x7faf0006c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf06fee700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf00161090 0x7faf0006ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf06fee700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf000ae6d0 con 0x7faf000cc310 2023-11-28T13:02:28.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf06fee700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf000ac7d0 con 0x7faf0015c990 2023-11-28T13:02:28.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf06fee700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf000acab0 con 0x7faf00161090 2023-11-28T13:02:28.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf04d8a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf000cc310 0x7faf0015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf04d8a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf000cc310 0x7faf0015bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33110/0 (socket says 172.21.15.6:33110) 2023-11-28T13:02:28.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf04d8a700 1 -- 172.21.15.6:0/43456992 learned_addr learned my addr 172.21.15.6:0/43456992 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:28.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faefffff700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf0015c990 0x7faf0006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf0558b700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf00161090 0x7faf0006ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf04d8a700 1 -- 172.21.15.6:0/43456992 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf00161090 msgr2=0x7faf0006ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf04d8a700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf00161090 0x7faf0006ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf04d8a700 1 -- 172.21.15.6:0/43456992 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf0015c990 msgr2=0x7faf0006c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:28.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf04d8a700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf0015c990 0x7faf0006c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:28.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.903+0000 7faf04d8a700 1 -- 172.21.15.6:0/43456992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef40094a0 con 0x7faf000cc310 2023-11-28T13:02:28.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.904+0000 7faf04d8a700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf000cc310 0x7faf0015bbb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faef40090b0 tx=0x7faef40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:28.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.904+0000 7faefd7fa700 1 -- 172.21.15.6:0/43456992 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4003ea0 con 0x7faf000cc310 2023-11-28T13:02:28.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.904+0000 7faefd7fa700 1 -- 172.21.15.6:0/43456992 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef4011630 con 0x7faf000cc310 2023-11-28T13:02:28.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.904+0000 7faf06fee700 1 -- 172.21.15.6:0/43456992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf000cb080 con 0x7faf000cc310 2023-11-28T13:02:28.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.904+0000 7faefd7fa700 1 -- 172.21.15.6:0/43456992 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef40119e0 con 0x7faf000cc310 2023-11-28T13:02:28.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.904+0000 7faf06fee700 1 -- 172.21.15.6:0/43456992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf000ce870 con 0x7faf000cc310 2023-11-28T13:02:28.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.905+0000 7faf06fee700 1 -- 172.21.15.6:0/43456992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf000cf570 con 0x7faf000cc310 2023-11-28T13:02:28.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.905+0000 7faefd7fa700 1 -- 172.21.15.6:0/43456992 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faef4017020 con 0x7faf000cc310 2023-11-28T13:02:28.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.905+0000 7faefd7fa700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faee40682f0 0x7faee406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:28.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.906+0000 7faefd7fa700 1 -- 172.21.15.6:0/43456992 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7faef40871b0 con 0x7faf000cc310 2023-11-28T13:02:28.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.906+0000 7faefffff700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faee40682f0 0x7faee406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:28.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.906+0000 7faefffff700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faee40682f0 0x7faee406a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faf001605c0 tx=0x7faeec007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:28.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:28.910+0000 7faefd7fa700 1 -- 172.21.15.6:0/43456992 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faef4053b30 con 0x7faf000cc310 2023-11-28T13:02:29.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.107+0000 7faf06fee700 1 -- 172.21.15.6:0/43456992 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faf000675a0 con 0x7faf000cc310 2023-11-28T13:02:29.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.108+0000 7faefd7fa700 1 -- 172.21.15.6:0/43456992 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+126 (secure 0 0 0) 0x7faef4057150 con 0x7faf000cc310 2023-11-28T13:02:29.112 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:29.112 INFO:teuthology.orchestra.run.smithi006.stdout:[ 2023-11-28T13:02:29.112 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:29.112 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 1, 2023-11-28T13:02:29.112 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": "rbd" 2023-11-28T13:02:29.112 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:29.113 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:29.113 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 2, 2023-11-28T13:02:29.113 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": ".mgr" 2023-11-28T13:02:29.113 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:29.113 INFO:teuthology.orchestra.run.smithi006.stdout:] 2023-11-28T13:02:29.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 -- 172.21.15.6:0/43456992 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faee40682f0 msgr2=0x7faee406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:29.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faee40682f0 0x7faee406a7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faf001605c0 tx=0x7faeec007ec0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 -- 172.21.15.6:0/43456992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf000cc310 msgr2=0x7faf0015bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:29.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf000cc310 0x7faf0015bbb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7faef40090b0 tx=0x7faef40047c0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 -- 172.21.15.6:0/43456992 shutdown_connections 2023-11-28T13:02:29.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faee40682f0 0x7faee406a7a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf00161090 0x7faf0006ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf000cc310 0x7faf0015bbb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 --2- 172.21.15.6:0/43456992 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf0015c990 0x7faf0006c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 -- 172.21.15.6:0/43456992 >> 172.21.15.6:0/43456992 conn(0x7faf000b1650 msgr2=0x7faf000cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:29.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 -- 172.21.15.6:0/43456992 shutdown_connections 2023-11-28T13:02:29.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.110+0000 7faf06fee700 1 -- 172.21.15.6:0/43456992 wait complete. 2023-11-28T13:02:29.124 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:02:29.124 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:02:29.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.252+0000 7f4260c81700 1 Processor -- start 2023-11-28T13:02:29.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.252+0000 7f4260c81700 1 -- start start 2023-11-28T13:02:29.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.253+0000 7f4260c81700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f425c0b2660 0x7f425c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:29.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.253+0000 7f4260c81700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c0cfb90 con 0x7f425c15a6c0 2023-11-28T13:02:29.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.253+0000 7f4260c81700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c0cb680 con 0x7f425c0cb230 2023-11-28T13:02:29.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.253+0000 7f4260c81700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425c0b3020 con 0x7f425c0b2660 2023-11-28T13:02:29.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.253+0000 7f425a59c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f425c15a6c0 0x7f425c0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48670/0 (socket says 172.21.15.6:48670) 2023-11-28T13:02:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.253+0000 7f425a59c700 1 -- 172.21.15.6:0/3057114081 learned_addr learned my addr 172.21.15.6:0/3057114081 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.253+0000 7f425ad9d700 1 --2- 172.21.15.6:0/3057114081 >> v2:172.21.15.61:3300/0 conn(0x7f425c0b2660 0x7f425c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.253+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456281123 0 0) 0x7f425c0cfb90 con 0x7f425c15a6c0 2023-11-28T13:02:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.253+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42440035b0 con 0x7f425c15a6c0 2023-11-28T13:02:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3586503203 0 0) 0x7f42440035b0 con 0x7f425c15a6c0 2023-11-28T13:02:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 >> v2:172.21.15.61:3300/0 conn(0x7f425c0b2660 msgr2=0x7f425c0b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 --2- 172.21.15.6:0/3057114081 >> v2:172.21.15.61:3300/0 conn(0x7f425c0b2660 0x7f425c0b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 >> v1:172.21.15.61:6789/0 conn(0x7f425c0cb230 legacy=0x7f425c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0cb930 con 0x7f425c15a6c0 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f424c001670 con 0x7f425c15a6c0 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 >> v1:172.21.15.6:6789/0 conn(0x7f425c15a6c0 legacy=0x7f425c0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244003a00 0x7f4244003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4244004850 0x7f4244004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4244005200 0x7f424400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0b3020 con 0x7f4244003a00 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424400c060 con 0x7f4244004850 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424400c220 con 0x7f4244005200 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425a59c700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244003a00 0x7f4244003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f4259d9b700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4244004850 0x7f4244004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425ad9d700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4244005200 0x7f424400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425a59c700 1 -- 172.21.15.6:0/3057114081 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4244005200 msgr2=0x7f424400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425a59c700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4244005200 0x7f424400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425a59c700 1 -- 172.21.15.6:0/3057114081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4244004850 msgr2=0x7f4244004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425a59c700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4244004850 0x7f4244004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425a59c700 1 -- 172.21.15.6:0/3057114081 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440035b0 con 0x7f4244003a00 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425ad9d700 1 -- 172.21.15.6:0/3057114081 reap_dead start 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f4259d9b700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4244004850 0x7f4244004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425a59c700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244003a00 0x7f4244003e10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f424c00c240 tx=0x7f4244005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b948bce5ca8019 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c007c10 con 0x7f4244003a00 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c00ce00 con 0x7f4244003a00 2023-11-28T13:02:29.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.254+0000 7f425959a700 1 -- 172.21.15.6:0/3057114081 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c005ba0 con 0x7f4244003a00 2023-11-28T13:02:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.255+0000 7f4260c81700 1 -- 172.21.15.6:0/3057114081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244003a00 msgr2=0x7f4244003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.255+0000 7f4260c81700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244003a00 0x7f4244003e10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f424c00c240 tx=0x7f4244005880 comp rx=0 tx=0).stop 2023-11-28T13:02:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.255+0000 7f4260c81700 1 -- 172.21.15.6:0/3057114081 shutdown_connections 2023-11-28T13:02:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.255+0000 7f4260c81700 1 --2- 172.21.15.6:0/3057114081 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4244003a00 0x7f4244003e10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.255+0000 7f4260c81700 1 -- 172.21.15.6:0/3057114081 >> 172.21.15.6:0/3057114081 conn(0x7f425c0cd660 msgr2=0x7f425c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.255+0000 7f4260c81700 1 -- 172.21.15.6:0/3057114081 shutdown_connections 2023-11-28T13:02:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.255+0000 7f4260c81700 1 -- 172.21.15.6:0/3057114081 wait complete. 2023-11-28T13:02:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.256+0000 7f4260c81700 1 Processor -- start 2023-11-28T13:02:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.256+0000 7f4260c81700 1 -- start start 2023-11-28T13:02:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.257+0000 7f4260c81700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4244003a00 0x7f425c0bd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.257+0000 7f4260c81700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f425c154110 0x7f425c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.257+0000 7f4260c81700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f425c154580 0x7f425c15c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.257+0000 7f4260c81700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0c8af0 con 0x7f425c154110 2023-11-28T13:02:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.257+0000 7f4260c81700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c1573e0 con 0x7f4244003a00 2023-11-28T13:02:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.257+0000 7f4260c81700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c156e20 con 0x7f425c154580 2023-11-28T13:02:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.257+0000 7f4259d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f425c154110 0x7f425c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.257+0000 7f4259d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f425c154110 0x7f425c15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33132/0 (socket says 172.21.15.6:33132) 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.257+0000 7f4259d9b700 1 -- 172.21.15.6:0/3971383094 learned_addr learned my addr 172.21.15.6:0/3971383094 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f425a59c700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4244003a00 0x7f425c0bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f425ad9d700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f425c154580 0x7f425c15c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f4259d9b700 1 -- 172.21.15.6:0/3971383094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f425c154580 msgr2=0x7f425c15c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f4259d9b700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f425c154580 0x7f425c15c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f4259d9b700 1 -- 172.21.15.6:0/3971383094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4244003a00 msgr2=0x7f425c0bd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f4259d9b700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4244003a00 0x7f425c0bd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f4259d9b700 1 -- 172.21.15.6:0/3971383094 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0cf540 con 0x7f425c154110 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f4259d9b700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f425c154110 0x7f425c15bff0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f425000e460 tx=0x7f425000e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f424affd700 1 -- 172.21.15.6:0/3971383094 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42500052e0 con 0x7f425c154110 2023-11-28T13:02:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f424affd700 1 -- 172.21.15.6:0/3971383094 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4250005440 con 0x7f425c154110 2023-11-28T13:02:29.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f4260c81700 1 -- 172.21.15.6:0/3971383094 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c0cf260 con 0x7f425c154110 2023-11-28T13:02:29.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f424affd700 1 -- 172.21.15.6:0/3971383094 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42500038c0 con 0x7f425c154110 2023-11-28T13:02:29.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.258+0000 7f4260c81700 1 -- 172.21.15.6:0/3971383094 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c0cf820 con 0x7f425c154110 2023-11-28T13:02:29.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.259+0000 7f4260c81700 1 -- 172.21.15.6:0/3971383094 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c0739f0 con 0x7f425c154110 2023-11-28T13:02:29.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.259+0000 7f424affd700 1 -- 172.21.15.6:0/3971383094 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f425001b030 con 0x7f425c154110 2023-11-28T13:02:29.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.260+0000 7f424affd700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f423c0682f0 0x7f423c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:29.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.260+0000 7f424affd700 1 -- 172.21.15.6:0/3971383094 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f4250084fa0 con 0x7f425c154110 2023-11-28T13:02:29.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.260+0000 7f425a59c700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f423c0682f0 0x7f423c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:29.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.260+0000 7f425a59c700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f423c0682f0 0x7f423c06a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f424c000c00 tx=0x7f424c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:29.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.264+0000 7f424affd700 1 -- 172.21.15.6:0/3971383094 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f425004d5e0 con 0x7f425c154110 2023-11-28T13:02:29.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:29.459+0000 7f4260c81700 1 -- 172.21.15.6:0/3971383094 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f425c0b12e0 con 0x7f425c154110 2023-11-28T13:02:30.136 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.135+0000 7f424affd700 1 -- 172.21.15.6:0/3971383094 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4250051ac0 con 0x7f425c154110 2023-11-28T13:02:30.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.140+0000 7f4260c81700 1 -- 172.21.15.6:0/3971383094 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f423c0682f0 msgr2=0x7f423c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:30.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.140+0000 7f4260c81700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f423c0682f0 0x7f423c06a7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f424c000c00 tx=0x7f424c007000 comp rx=0 tx=0).stop 2023-11-28T13:02:30.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.140+0000 7f4260c81700 1 -- 172.21.15.6:0/3971383094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f425c154110 msgr2=0x7f425c15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:30.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.140+0000 7f4260c81700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f425c154110 0x7f425c15bff0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f425000e460 tx=0x7f425000e820 comp rx=0 tx=0).stop 2023-11-28T13:02:30.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.140+0000 7f4260c81700 1 -- 172.21.15.6:0/3971383094 shutdown_connections 2023-11-28T13:02:30.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.140+0000 7f4260c81700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f423c0682f0 0x7f423c06a7a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:30.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.140+0000 7f4260c81700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f425c154580 0x7f425c15c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:30.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.140+0000 7f4260c81700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4244003a00 0x7f425c0bd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:30.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.140+0000 7f4260c81700 1 --2- 172.21.15.6:0/3971383094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f425c154110 0x7f425c15bff0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:30.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.140+0000 7f4260c81700 1 -- 172.21.15.6:0/3971383094 >> 172.21.15.6:0/3971383094 conn(0x7f425c0cd660 msgr2=0x7f425c0b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:30.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.141+0000 7f4260c81700 1 -- 172.21.15.6:0/3971383094 shutdown_connections 2023-11-28T13:02:30.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.142+0000 7f4260c81700 1 -- 172.21.15.6:0/3971383094 wait complete. 2023-11-28T13:02:30.145 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' created 2023-11-28T13:02:30.159 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:02:30.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.296+0000 7f7157ee4700 1 Processor -- start 2023-11-28T13:02:30.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.297+0000 7f7157ee4700 1 -- start start 2023-11-28T13:02:30.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.297+0000 7f7157ee4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f71500b1630 0x7f71500cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:30.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.297+0000 7f7157ee4700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71500ce550 con 0x7f71500b1a00 2023-11-28T13:02:30.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.297+0000 7f7157ee4700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71500ce830 con 0x7f715006afc0 2023-11-28T13:02:30.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.297+0000 7f7157ee4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71500ceb10 con 0x7f71500b1630 2023-11-28T13:02:30.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.297+0000 7f715547f700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f71500b1a00 0x7f71500caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48692/0 (socket says 172.21.15.6:48692) 2023-11-28T13:02:30.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f715547f700 1 -- 172.21.15.6:0/1178676737 learned_addr learned my addr 172.21.15.6:0/1178676737 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:30.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7155c80700 1 --2- 172.21.15.6:0/1178676737 >> v2:172.21.15.61:3300/0 conn(0x7f71500b1630 0x7f71500cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:30.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760769070 0 0) 0x7f71500ce550 con 0x7f71500b1a00 2023-11-28T13:02:30.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f713c0035b0 con 0x7f71500b1a00 2023-11-28T13:02:30.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3977173775 0 0) 0x7f713c0035b0 con 0x7f71500b1a00 2023-11-28T13:02:30.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 >> v2:172.21.15.61:3300/0 conn(0x7f71500b1630 msgr2=0x7f71500cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:30.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 --2- 172.21.15.6:0/1178676737 >> v2:172.21.15.61:3300/0 conn(0x7f71500b1630 0x7f71500cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:30.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 >> v1:172.21.15.61:6789/0 conn(0x7f715006afc0 legacy=0x7f715006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:30.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71500cedf0 con 0x7f71500b1a00 2023-11-28T13:02:30.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f714c001670 con 0x7f71500b1a00 2023-11-28T13:02:30.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 >> v1:172.21.15.6:6789/0 conn(0x7f71500b1a00 legacy=0x7f71500caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:30.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c003a00 0x7f713c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:30.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f713c004850 0x7f713c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:30.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f713c005200 0x7f713c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:30.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71500ceb10 con 0x7f713c003a00 2023-11-28T13:02:30.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c00c080 con 0x7f713c005200 2023-11-28T13:02:30.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c00c240 con 0x7f713c004850 2023-11-28T13:02:30.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7155c80700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c003a00 0x7f713c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:30.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f715547f700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f713c004850 0x7f713c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:30.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.298+0000 7f7156481700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f713c005200 0x7f713c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:30.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7155c80700 1 -- 172.21.15.6:0/1178676737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f713c004850 msgr2=0x7f713c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:30.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7155c80700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f713c004850 0x7f713c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:30.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7155c80700 1 -- 172.21.15.6:0/1178676737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f713c005200 msgr2=0x7f713c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:30.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7155c80700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f713c005200 0x7f713c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7156481700 1 -- 172.21.15.6:0/1178676737 reap_dead start 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7155c80700 1 -- 172.21.15.6:0/1178676737 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c0035b0 con 0x7f713c003a00 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7155c80700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c003a00 0x7f713c003e10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7140009b60 tx=0x7f713c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518e093a41c8f4aa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714000e6c0 con 0x7f713c003a00 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714000e820 con 0x7f713c003a00 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7154c7e700 1 -- 172.21.15.6:0/1178676737 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714000eb90 con 0x7f713c003a00 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7157ee4700 1 -- 172.21.15.6:0/1178676737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c003a00 msgr2=0x7f713c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7157ee4700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c003a00 0x7f713c003e10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7140009b60 tx=0x7f713c005880 comp rx=0 tx=0).stop 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7157ee4700 1 -- 172.21.15.6:0/1178676737 shutdown_connections 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7157ee4700 1 --2- 172.21.15.6:0/1178676737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f713c003a00 0x7f713c003e10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.299+0000 7f7157ee4700 1 -- 172.21.15.6:0/1178676737 >> 172.21.15.6:0/1178676737 conn(0x7f71500cc630 msgr2=0x7f71500cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:30.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.300+0000 7f7157ee4700 1 -- 172.21.15.6:0/1178676737 shutdown_connections 2023-11-28T13:02:30.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.300+0000 7f7157ee4700 1 -- 172.21.15.6:0/1178676737 wait complete. 2023-11-28T13:02:30.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.301+0000 7f7157ee4700 1 Processor -- start 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.301+0000 7f7157ee4700 1 -- start start 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.301+0000 7f7157ee4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f713c003a00 0x7f71500c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.301+0000 7f7157ee4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71501498c0 0x7f715014caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.301+0000 7f7157ee4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f715015aa10 0x7f715015ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7157ee4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715015b3c0 con 0x7f715015aa10 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7157ee4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715014a030 con 0x7f71501498c0 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7157ee4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7150149d30 con 0x7f713c003a00 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7156481700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f715015aa10 0x7f715015ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7156481700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f715015aa10 0x7f715015ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33154/0 (socket says 172.21.15.6:33154) 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7156481700 1 -- 172.21.15.6:0/4013797524 learned_addr learned my addr 172.21.15.6:0/4013797524 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:30.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7155c80700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f713c003a00 0x7f71500c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f715547f700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71501498c0 0x7f715014caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7156481700 1 -- 172.21.15.6:0/4013797524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f713c003a00 msgr2=0x7f71500c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7156481700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f713c003a00 0x7f71500c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7156481700 1 -- 172.21.15.6:0/4013797524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71501498c0 msgr2=0x7f715014caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7156481700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71501498c0 0x7f715014caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7156481700 1 -- 172.21.15.6:0/4013797524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71500b1ea0 con 0x7f715015aa10 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7156481700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f715015aa10 0x7f715015ae80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f71500b2780 tx=0x7f714800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f71467fc700 1 -- 172.21.15.6:0/4013797524 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148003c10 con 0x7f715015aa10 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f71467fc700 1 -- 172.21.15.6:0/4013797524 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7148003d90 con 0x7f715015aa10 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7157ee4700 1 -- 172.21.15.6:0/4013797524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71500cf0d0 con 0x7f715015aa10 2023-11-28T13:02:30.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f71467fc700 1 -- 172.21.15.6:0/4013797524 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148005310 con 0x7f715015aa10 2023-11-28T13:02:30.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.302+0000 7f7157ee4700 1 -- 172.21.15.6:0/4013797524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71500ce140 con 0x7f715015aa10 2023-11-28T13:02:30.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.303+0000 7f7157ee4700 1 -- 172.21.15.6:0/4013797524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71500739f0 con 0x7f715015aa10 2023-11-28T13:02:30.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.304+0000 7f71467fc700 1 -- 172.21.15.6:0/4013797524 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7148005470 con 0x7f715015aa10 2023-11-28T13:02:30.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.304+0000 7f71467fc700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71340682f0 0x7f713406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:30.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.304+0000 7f71467fc700 1 -- 172.21.15.6:0/4013797524 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f7148085d00 con 0x7f715015aa10 2023-11-28T13:02:30.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.304+0000 7f7155c80700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71340682f0 0x7f713406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:30.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.305+0000 7f7155c80700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71340682f0 0x7f713406a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f71400090b0 tx=0x7f7140008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:30.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.309+0000 7f71467fc700 1 -- 172.21.15.6:0/4013797524 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7148089030 con 0x7f715015aa10 2023-11-28T13:02:30.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:30.503+0000 7f7157ee4700 1 -- 172.21.15.6:0/4013797524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f71500cac80 con 0x7f715015aa10 2023-11-28T13:02:31.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.141+0000 7f71467fc700 1 -- 172.21.15.6:0/4013797524 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7148052600 con 0x7f715015aa10 2023-11-28T13:02:31.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.146+0000 7f7157ee4700 1 -- 172.21.15.6:0/4013797524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71340682f0 msgr2=0x7f713406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.146+0000 7f7157ee4700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71340682f0 0x7f713406a7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f71400090b0 tx=0x7f7140008030 comp rx=0 tx=0).stop 2023-11-28T13:02:31.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.146+0000 7f7157ee4700 1 -- 172.21.15.6:0/4013797524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f715015aa10 msgr2=0x7f715015ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.146+0000 7f7157ee4700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f715015aa10 0x7f715015ae80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f71500b2780 tx=0x7f714800ee00 comp rx=0 tx=0).stop 2023-11-28T13:02:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.148+0000 7f7157ee4700 1 -- 172.21.15.6:0/4013797524 shutdown_connections 2023-11-28T13:02:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.148+0000 7f7157ee4700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f715015aa10 0x7f715015ae80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.148+0000 7f7157ee4700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f71340682f0 0x7f713406a7a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.148+0000 7f7157ee4700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f713c003a00 0x7f71500c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.148+0000 7f7157ee4700 1 --2- 172.21.15.6:0/4013797524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71501498c0 0x7f715014caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.148+0000 7f7157ee4700 1 -- 172.21.15.6:0/4013797524 >> 172.21.15.6:0/4013797524 conn(0x7f71500cc630 msgr2=0x7f71500cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:31.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.149+0000 7f7157ee4700 1 -- 172.21.15.6:0/4013797524 shutdown_connections 2023-11-28T13:02:31.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.149+0000 7f7157ee4700 1 -- 172.21.15.6:0/4013797524 wait complete. 2023-11-28T13:02:31.154 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' created 2023-11-28T13:02:31.174 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:02:31.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.305+0000 7f35ef94a700 1 Processor -- start 2023-11-28T13:02:31.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.306+0000 7f35ef94a700 1 -- start start 2023-11-28T13:02:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.306+0000 7f35ef94a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f35e8159500 0x7f35e80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.306+0000 7f35ef94a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f35e80b1d40 0x7f35e80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.306+0000 7f35ef94a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e815e040 con 0x7f35e815dbf0 2023-11-28T13:02:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.306+0000 7f35ef94a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e80cede0 con 0x7f35e8159500 2023-11-28T13:02:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.306+0000 7f35ef94a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e80ceff0 con 0x7f35e80b1d40 2023-11-28T13:02:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35ed6e6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f35e8159500 0x7f35e80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35ed6e6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f35e8159500 0x7f35e80cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33158/0 (socket says 172.21.15.6:33158) 2023-11-28T13:02:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35ecee5700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f35e815dbf0 0x7f35e80b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48710/0 (socket says 172.21.15.6:48710) 2023-11-28T13:02:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35ed6e6700 1 -- 172.21.15.6:0/3497727265 learned_addr learned my addr 172.21.15.6:0/3497727265 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:31.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35edee7700 1 --2- 172.21.15.6:0/3497727265 >> v2:172.21.15.61:3300/0 conn(0x7f35e80b1d40 0x7f35e80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35ed6e6700 1 -- 172.21.15.6:0/3497727265 >> v2:172.21.15.61:3300/0 conn(0x7f35e80b1d40 msgr2=0x7f35e80ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35ed6e6700 1 --2- 172.21.15.6:0/3497727265 >> v2:172.21.15.61:3300/0 conn(0x7f35e80b1d40 0x7f35e80ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35ed6e6700 1 -- 172.21.15.6:0/3497727265 >> v1:172.21.15.6:6789/0 conn(0x7f35e815dbf0 legacy=0x7f35e80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35dbfff700 1 -- 172.21.15.6:0/3497727265 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260254973 0 0) 0x7f35e815e040 con 0x7f35e815dbf0 2023-11-28T13:02:31.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35ed6e6700 1 -- 172.21.15.6:0/3497727265 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e80cf2d0 con 0x7f35e8159500 2023-11-28T13:02:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35edee7700 1 --2- 172.21.15.6:0/3497727265 >> v2:172.21.15.61:3300/0 conn(0x7f35e80b1d40 0x7f35e80ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35ed6e6700 1 --2- 172.21.15.6:0/3497727265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e8159500 0x7f35e80cd140 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f35e80cfb50 tx=0x7f35e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ca181fd4c623fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35dbfff700 1 -- 172.21.15.6:0/3497727265 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400b870 con 0x7f35e8159500 2023-11-28T13:02:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35dbfff700 1 -- 172.21.15.6:0/3497727265 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e400b9d0 con 0x7f35e8159500 2023-11-28T13:02:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.307+0000 7f35dbfff700 1 -- 172.21.15.6:0/3497727265 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e400bd60 con 0x7f35e8159500 2023-11-28T13:02:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.308+0000 7f35ef94a700 1 -- 172.21.15.6:0/3497727265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e8159500 msgr2=0x7f35e80cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.308+0000 7f35ef94a700 1 --2- 172.21.15.6:0/3497727265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e8159500 0x7f35e80cd140 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f35e80cfb50 tx=0x7f35e4009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.308+0000 7f35ef94a700 1 -- 172.21.15.6:0/3497727265 shutdown_connections 2023-11-28T13:02:31.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.308+0000 7f35ef94a700 1 --2- 172.21.15.6:0/3497727265 >> v2:172.21.15.61:3300/0 conn(0x7f35e80b1d40 0x7f35e80ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.308+0000 7f35ef94a700 1 --2- 172.21.15.6:0/3497727265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e8159500 0x7f35e80cd140 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.308+0000 7f35ef94a700 1 -- 172.21.15.6:0/3497727265 >> 172.21.15.6:0/3497727265 conn(0x7f35e80cc630 msgr2=0x7f35e80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:31.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.308+0000 7f35ef94a700 1 -- 172.21.15.6:0/3497727265 shutdown_connections 2023-11-28T13:02:31.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.308+0000 7f35ef94a700 1 -- 172.21.15.6:0/3497727265 wait complete. 2023-11-28T13:02:31.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.309+0000 7f35ef94a700 1 Processor -- start 2023-11-28T13:02:31.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.309+0000 7f35ef94a700 1 -- start start 2023-11-28T13:02:31.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ef94a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e80b1d40 0x7f35e80b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ef94a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f35e8159500 0x7f35e8164440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ef94a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f35e815dbf0 0x7f35e80c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ef94a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e8164980 con 0x7f35e80b1d40 2023-11-28T13:02:31.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ef94a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e8164be0 con 0x7f35e815dbf0 2023-11-28T13:02:31.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ef94a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e8164ec0 con 0x7f35e8159500 2023-11-28T13:02:31.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ed6e6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e80b1d40 0x7f35e80b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ed6e6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e80b1d40 0x7f35e80b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33164/0 (socket says 172.21.15.6:33164) 2023-11-28T13:02:31.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ed6e6700 1 -- 172.21.15.6:0/1618905034 learned_addr learned my addr 172.21.15.6:0/1618905034 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:31.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35edee7700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f35e815dbf0 0x7f35e80c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ecee5700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f35e8159500 0x7f35e8164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ed6e6700 1 -- 172.21.15.6:0/1618905034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f35e8159500 msgr2=0x7f35e8164440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ed6e6700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f35e8159500 0x7f35e8164440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ed6e6700 1 -- 172.21.15.6:0/1618905034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f35e815dbf0 msgr2=0x7f35e80c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ed6e6700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f35e815dbf0 0x7f35e80c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35ed6e6700 1 -- 172.21.15.6:0/1618905034 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4009550 con 0x7f35e80b1d40 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.310+0000 7f35edee7700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f35e815dbf0 0x7f35e80c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.311+0000 7f35ed6e6700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e80b1d40 0x7f35e80b3580 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009c10 tx=0x7f35e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.311+0000 7f35d9ffb700 1 -- 172.21.15.6:0/1618905034 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4004310 con 0x7f35e80b1d40 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.311+0000 7f35d9ffb700 1 -- 172.21.15.6:0/1618905034 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e401f070 con 0x7f35e80b1d40 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.311+0000 7f35ef94a700 1 -- 172.21.15.6:0/1618905034 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e815d3f0 con 0x7f35e80b1d40 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.311+0000 7f35d9ffb700 1 -- 172.21.15.6:0/1618905034 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e40036a0 con 0x7f35e80b1d40 2023-11-28T13:02:31.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.311+0000 7f35ef94a700 1 -- 172.21.15.6:0/1618905034 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e815d110 con 0x7f35e80b1d40 2023-11-28T13:02:31.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.312+0000 7f35ef94a700 1 -- 172.21.15.6:0/1618905034 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e815d6d0 con 0x7f35e80b1d40 2023-11-28T13:02:31.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.313+0000 7f35d9ffb700 1 -- 172.21.15.6:0/1618905034 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f35e402a020 con 0x7f35e80b1d40 2023-11-28T13:02:31.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.313+0000 7f35d9ffb700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35cc068320 0x7f35cc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.313+0000 7f35d9ffb700 1 -- 172.21.15.6:0/1618905034 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f35e40877e0 con 0x7f35e80b1d40 2023-11-28T13:02:31.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.313+0000 7f35ecee5700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35cc068320 0x7f35cc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.313+0000 7f35ecee5700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35cc068320 0x7f35cc06a7d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f35dc000f60 tx=0x7f35dc007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:31.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.317+0000 7f35d9ffb700 1 -- 172.21.15.6:0/1618905034 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f35e4054000 con 0x7f35e80b1d40 2023-11-28T13:02:31.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.560+0000 7f35ef94a700 1 -- 172.21.15.6:0/1618905034 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f35e80739f0 con 0x7f35e80b1d40 2023-11-28T13:02:31.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.568+0000 7f35d9ffb700 1 -- 172.21.15.6:0/1618905034 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f35e4057620 con 0x7f35e80b1d40 2023-11-28T13:02:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.578+0000 7f35ef94a700 1 -- 172.21.15.6:0/1618905034 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35cc068320 msgr2=0x7f35cc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.578+0000 7f35ef94a700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35cc068320 0x7f35cc06a7d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f35dc000f60 tx=0x7f35dc007c90 comp rx=0 tx=0).stop 2023-11-28T13:02:31.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.578+0000 7f35ef94a700 1 -- 172.21.15.6:0/1618905034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e80b1d40 msgr2=0x7f35e80b3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.578+0000 7f35ef94a700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e80b1d40 0x7f35e80b3580 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009c10 tx=0x7f35e4004750 comp rx=0 tx=0).stop 2023-11-28T13:02:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.578+0000 7f35ef94a700 1 -- 172.21.15.6:0/1618905034 shutdown_connections 2023-11-28T13:02:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.578+0000 7f35ef94a700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f35e815dbf0 0x7f35e80c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.578+0000 7f35ef94a700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35e80b1d40 0x7f35e80b3580 secure :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009c10 tx=0x7f35e4004750 comp rx=0 tx=0).stop 2023-11-28T13:02:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.578+0000 7f35ef94a700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35cc068320 0x7f35cc06a7d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.578+0000 7f35ef94a700 1 --2- 172.21.15.6:0/1618905034 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f35e8159500 0x7f35e8164440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.578+0000 7f35ef94a700 1 -- 172.21.15.6:0/1618905034 >> 172.21.15.6:0/1618905034 conn(0x7f35e80cc630 msgr2=0x7f35e806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.583+0000 7f35ef94a700 1 -- 172.21.15.6:0/1618905034 shutdown_connections 2023-11-28T13:02:31.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.583+0000 7f35ef94a700 1 -- 172.21.15.6:0/1618905034 wait complete. 2023-11-28T13:02:31.593 INFO:teuthology.orchestra.run.smithi006.stderr:new fs with metadata pool 8 and data pool 9 2023-11-28T13:02:31.606 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:31.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.729+0000 7f04c8a0a700 1 Processor -- start 2023-11-28T13:02:31.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.729+0000 7f04c8a0a700 1 -- start start 2023-11-28T13:02:31.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.730+0000 7f04c8a0a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f04c415c560 0x7f04c406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.730+0000 7f04c8a0a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c40b30b0 con 0x7f04c40b2660 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.730+0000 7f04c8a0a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c406c020 con 0x7f04c40ceec0 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.730+0000 7f04c8a0a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c40d09c0 con 0x7f04c415c560 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.730+0000 7f04c259c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f04c415c560 0x7f04c406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.731+0000 7f04c1d9b700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f04c40b2660 0x7f04c40ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48736/0 (socket says 172.21.15.6:48736) 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.731+0000 7f04c1d9b700 1 -- 172.21.15.6:0/1817772299 learned_addr learned my addr 172.21.15.6:0/1817772299 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.731+0000 7f04c259c700 1 -- 172.21.15.6:0/1817772299 >> v1:172.21.15.61:6789/0 conn(0x7f04c40ceec0 legacy=0x7f04c406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.731+0000 7f04c259c700 1 -- 172.21.15.6:0/1817772299 >> v1:172.21.15.6:6789/0 conn(0x7f04c40b2660 legacy=0x7f04c40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.731+0000 7f04c159a700 1 -- 172.21.15.6:0/1817772299 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948470940 0 0) 0x7f04c40b30b0 con 0x7f04c40b2660 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.731+0000 7f04c259c700 1 -- 172.21.15.6:0/1817772299 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c406fdf0 con 0x7f04c415c560 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.731+0000 7f04c259c700 1 --2- 172.21.15.6:0/1817772299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c415c560 0x7f04c406f8b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009840 tx=0x7f04b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901c8bfcc3e54d80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.731+0000 7f04c159a700 1 -- 172.21.15.6:0/1817772299 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4011830 con 0x7f04c415c560 2023-11-28T13:02:31.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.731+0000 7f04c159a700 1 -- 172.21.15.6:0/1817772299 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b4011990 con 0x7f04c415c560 2023-11-28T13:02:31.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.731+0000 7f04c159a700 1 -- 172.21.15.6:0/1817772299 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4011d20 con 0x7f04c415c560 2023-11-28T13:02:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.732+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1817772299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c415c560 msgr2=0x7f04c406f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.732+0000 7f04c8a0a700 1 --2- 172.21.15.6:0/1817772299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c415c560 0x7f04c406f8b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009840 tx=0x7f04b4009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.732+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1817772299 shutdown_connections 2023-11-28T13:02:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.732+0000 7f04c8a0a700 1 --2- 172.21.15.6:0/1817772299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c415c560 0x7f04c406f8b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.732+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1817772299 >> 172.21.15.6:0/1817772299 conn(0x7f04c40cd660 msgr2=0x7f04c40cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.732+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1817772299 shutdown_connections 2023-11-28T13:02:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.732+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1817772299 wait complete. 2023-11-28T13:02:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.733+0000 7f04c8a0a700 1 Processor -- start 2023-11-28T13:02:31.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.733+0000 7f04c8a0a700 1 -- start start 2023-11-28T13:02:31.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c8a0a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04c40b2660 0x7f04c414d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c8a0a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c40ceec0 0x7f04c40ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c8a0a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04c415c560 0x7f04c415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c8a0a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c414d6a0 con 0x7f04c40ceec0 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c8a0a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40ace20 con 0x7f04c40b2660 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c8a0a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c40ac210 con 0x7f04c415c560 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c40ceec0 0x7f04c40ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c40ceec0 0x7f04c40ab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33190/0 (socket says 172.21.15.6:33190) 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c1d9b700 1 -- 172.21.15.6:0/1687909758 learned_addr learned my addr 172.21.15.6:0/1687909758 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c259c700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04c40b2660 0x7f04c414d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c2d9d700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04c415c560 0x7f04c415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c1d9b700 1 -- 172.21.15.6:0/1687909758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04c415c560 msgr2=0x7f04c415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c1d9b700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04c415c560 0x7f04c415d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c1d9b700 1 -- 172.21.15.6:0/1687909758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04c40b2660 msgr2=0x7f04c414d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c1d9b700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04c40b2660 0x7f04c414d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c1d9b700 1 -- 172.21.15.6:0/1687909758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b40094a0 con 0x7f04c40ceec0 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.734+0000 7f04c1d9b700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c40ceec0 0x7f04c40ab9c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c520 tx=0x7f04b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.735+0000 7f04b2ffd700 1 -- 172.21.15.6:0/1687909758 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8004c80 con 0x7f04c40ceec0 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.735+0000 7f04b2ffd700 1 -- 172.21.15.6:0/1687909758 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b8004e00 con 0x7f04c40ceec0 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.735+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1687909758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c40b2dd0 con 0x7f04c40ceec0 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.735+0000 7f04b2ffd700 1 -- 172.21.15.6:0/1687909758 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8007310 con 0x7f04c40ceec0 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.735+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1687909758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c40b2af0 con 0x7f04c40ceec0 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.736+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1687909758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c406bcb0 con 0x7f04c40ceec0 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.736+0000 7f04b2ffd700 1 -- 172.21.15.6:0/1687909758 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f04b801a020 con 0x7f04c40ceec0 2023-11-28T13:02:31.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.736+0000 7f04b2ffd700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04a40682f0 0x7f04a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:31.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.737+0000 7f04b2ffd700 1 -- 172.21.15.6:0/1687909758 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f04b8085b40 con 0x7f04c40ceec0 2023-11-28T13:02:31.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.737+0000 7f04c259c700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04a40682f0 0x7f04a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:31.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.737+0000 7f04c259c700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04a40682f0 0x7f04a406a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009b90 tx=0x7f04b401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:31.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.741+0000 7f04b2ffd700 1 -- 172.21.15.6:0/1687909758 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f04b8052170 con 0x7f04c40ceec0 2023-11-28T13:02:31.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.935+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1687909758 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04c40675a0 con 0x7f04c40ceec0 2023-11-28T13:02:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.936+0000 7f04b2ffd700 1 -- 172.21.15.6:0/1687909758 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18039 (secure 0 0 0) 0x7f04b8055790 con 0x7f04c40ceec0 2023-11-28T13:02:31.937 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:31.937 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":53,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:31.562011+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:29.490442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:30.534532+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1687909758 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04a40682f0 msgr2=0x7f04a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04a40682f0 0x7f04a406a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009b90 tx=0x7f04b401c040 comp rx=0 tx=0).stop 2023-11-28T13:02:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1687909758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c40ceec0 msgr2=0x7f04c40ab9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c40ceec0 0x7f04c40ab9c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c520 tx=0x7f04b800c8e0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1687909758 shutdown_connections 2023-11-28T13:02:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04a40682f0 0x7f04a406a7a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04c415c560 0x7f04c415d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04c40b2660 0x7f04c414d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 --2- 172.21.15.6:0/1687909758 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04c40ceec0 0x7f04c40ab9c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1687909758 >> 172.21.15.6:0/1687909758 conn(0x7f04c40cd660 msgr2=0x7f04c40b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1687909758 shutdown_connections 2023-11-28T13:02:31.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:31.938+0000 7f04c8a0a700 1 -- 172.21.15.6:0/1687909758 wait complete. 2023-11-28T13:02:31.952 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:32.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.075+0000 7f3d826f0700 1 Processor -- start 2023-11-28T13:02:32.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.075+0000 7f3d826f0700 1 -- start start 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d826f0700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3d7c15c990 0x7f3d7c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d826f0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3d7c161090 0x7f3d7c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d826f0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c0caeb0 0x7f3d7c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d826f0700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7c0ce340 con 0x7f3d7c15c990 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d826f0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7c0cebe0 con 0x7f3d7c161090 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d826f0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0cb3f0 con 0x7f3d7c0caeb0 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7bfff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3d7c15c990 0x7f3d7c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7bfff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3d7c15c990 0x7f3d7c0cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33198/0 (socket says 172.21.15.6:33198) 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7bfff700 1 -- 172.21.15.6:0/2215956960 learned_addr learned my addr 172.21.15.6:0/2215956960 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7b7fe700 1 --2- 172.21.15.6:0/2215956960 >> v2:172.21.15.61:3300/0 conn(0x7f3d7c161090 0x7f3d7c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d80c8d700 1 --2- 172.21.15.6:0/2215956960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c0caeb0 0x7f3d7c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7bfff700 1 -- 172.21.15.6:0/2215956960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c0caeb0 msgr2=0x7f3d7c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7bfff700 1 --2- 172.21.15.6:0/2215956960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c0caeb0 0x7f3d7c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7bfff700 1 -- 172.21.15.6:0/2215956960 >> v2:172.21.15.61:3300/0 conn(0x7f3d7c161090 msgr2=0x7f3d7c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7bfff700 1 --2- 172.21.15.6:0/2215956960 >> v2:172.21.15.61:3300/0 conn(0x7f3d7c161090 0x7f3d7c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7bfff700 1 -- 172.21.15.6:0/2215956960 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0ce590 con 0x7f3d7c15c990 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d80c8d700 1 --2- 172.21.15.6:0/2215956960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c0caeb0 0x7f3d7c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7b7fe700 1 --2- 172.21.15.6:0/2215956960 >> v2:172.21.15.61:3300/0 conn(0x7f3d7c161090 0x7f3d7c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.076+0000 7f3d7bfff700 1 --2- 172.21.15.6:0/2215956960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c15c990 0x7f3d7c0cf070 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c06ab20 tx=0x7f3d64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0157343c03210c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d7affd700 1 -- 172.21.15.6:0/2215956960 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400b870 con 0x7f3d7c15c990 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d7affd700 1 -- 172.21.15.6:0/2215956960 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6400b9d0 con 0x7f3d7c15c990 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d7affd700 1 -- 172.21.15.6:0/2215956960 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400bd60 con 0x7f3d7c15c990 2023-11-28T13:02:32.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d826f0700 1 -- 172.21.15.6:0/2215956960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c15c990 msgr2=0x7f3d7c0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:32.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d826f0700 1 --2- 172.21.15.6:0/2215956960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c15c990 0x7f3d7c0cf070 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c06ab20 tx=0x7f3d64009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d826f0700 1 -- 172.21.15.6:0/2215956960 shutdown_connections 2023-11-28T13:02:32.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d826f0700 1 --2- 172.21.15.6:0/2215956960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c0caeb0 0x7f3d7c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d826f0700 1 --2- 172.21.15.6:0/2215956960 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c15c990 0x7f3d7c0cf070 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d826f0700 1 --2- 172.21.15.6:0/2215956960 >> v2:172.21.15.61:3300/0 conn(0x7f3d7c161090 0x7f3d7c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d826f0700 1 -- 172.21.15.6:0/2215956960 >> 172.21.15.6:0/2215956960 conn(0x7f3d7c0b1650 msgr2=0x7f3d7c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:32.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d826f0700 1 -- 172.21.15.6:0/2215956960 shutdown_connections 2023-11-28T13:02:32.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.077+0000 7f3d826f0700 1 -- 172.21.15.6:0/2215956960 wait complete. 2023-11-28T13:02:32.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.079+0000 7f3d826f0700 1 Processor -- start 2023-11-28T13:02:32.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.079+0000 7f3d826f0700 1 -- start start 2023-11-28T13:02:32.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d826f0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d7c0caeb0 0x7f3d7c0c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d826f0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c15c990 0x7f3d7c0c3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d826f0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c161090 0x7f3d7c1485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d826f0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c3690 con 0x7f3d7c161090 2023-11-28T13:02:32.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d826f0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c15f190 con 0x7f3d7c0caeb0 2023-11-28T13:02:32.084 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d826f0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c15e860 con 0x7f3d7c15c990 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d80c8d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c161090 0x7f3d7c1485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d80c8d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c161090 0x7f3d7c1485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33204/0 (socket says 172.21.15.6:33204) 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d80c8d700 1 -- 172.21.15.6:0/2619749051 learned_addr learned my addr 172.21.15.6:0/2619749051 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d7bfff700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d7c0caeb0 0x7f3d7c0c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d7b7fe700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c15c990 0x7f3d7c0c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d80c8d700 1 -- 172.21.15.6:0/2619749051 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c15c990 msgr2=0x7f3d7c0c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d80c8d700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c15c990 0x7f3d7c0c3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d80c8d700 1 -- 172.21.15.6:0/2619749051 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d7c0caeb0 msgr2=0x7f3d7c0c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d80c8d700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d7c0caeb0 0x7f3d7c0c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d80c8d700 1 -- 172.21.15.6:0/2619749051 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d64009550 con 0x7f3d7c161090 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d7b7fe700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c15c990 0x7f3d7c0c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.080+0000 7f3d80c8d700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c161090 0x7f3d7c1485e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3d74007db0 tx=0x7f3d7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:32.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.081+0000 7f3d78ff9700 1 -- 172.21.15.6:0/2619749051 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400ddd0 con 0x7f3d7c161090 2023-11-28T13:02:32.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.081+0000 7f3d78ff9700 1 -- 172.21.15.6:0/2619749051 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7400f040 con 0x7f3d7c161090 2023-11-28T13:02:32.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.081+0000 7f3d826f0700 1 -- 172.21.15.6:0/2619749051 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c0cf5b0 con 0x7f3d7c161090 2023-11-28T13:02:32.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.081+0000 7f3d78ff9700 1 -- 172.21.15.6:0/2619749051 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74003c60 con 0x7f3d7c161090 2023-11-28T13:02:32.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.081+0000 7f3d826f0700 1 -- 172.21.15.6:0/2619749051 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c0ce870 con 0x7f3d7c161090 2023-11-28T13:02:32.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.082+0000 7f3d826f0700 1 -- 172.21.15.6:0/2619749051 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c0cf740 con 0x7f3d7c161090 2023-11-28T13:02:32.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.082+0000 7f3d78ff9700 1 -- 172.21.15.6:0/2619749051 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3d7401c030 con 0x7f3d7c161090 2023-11-28T13:02:32.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.082+0000 7f3d78ff9700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3d5c068320 0x7f3d5c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.083+0000 7f3d78ff9700 1 -- 172.21.15.6:0/2619749051 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3d74085c70 con 0x7f3d7c161090 2023-11-28T13:02:32.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.083+0000 7f3d7bfff700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3d5c068320 0x7f3d5c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.083+0000 7f3d7bfff700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3d5c068320 0x7f3d5c06a7d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3d64000c00 tx=0x7f3d64008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:32.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.087+0000 7f3d78ff9700 1 -- 172.21.15.6:0/2619749051 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3d74052390 con 0x7f3d7c161090 2023-11-28T13:02:32.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.283+0000 7f3d826f0700 1 -- 172.21.15.6:0/2619749051 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d7c0675a0 con 0x7f3d7c161090 2023-11-28T13:02:32.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.284+0000 7f3d78ff9700 1 -- 172.21.15.6:0/2619749051 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18039 (secure 0 0 0) 0x7f3d740559b0 con 0x7f3d7c161090 2023-11-28T13:02:32.286 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:32.286 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":53,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:31.562011+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:29.490442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:30.534532+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:32.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.286+0000 7f3d826f0700 1 -- 172.21.15.6:0/2619749051 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3d5c068320 msgr2=0x7f3d5c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:32.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.286+0000 7f3d826f0700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3d5c068320 0x7f3d5c06a7d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3d64000c00 tx=0x7f3d64008030 comp rx=0 tx=0).stop 2023-11-28T13:02:32.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.286+0000 7f3d826f0700 1 -- 172.21.15.6:0/2619749051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c161090 msgr2=0x7f3d7c1485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:32.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.286+0000 7f3d826f0700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c161090 0x7f3d7c1485e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3d74007db0 tx=0x7f3d7400d3b0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.286+0000 7f3d826f0700 1 -- 172.21.15.6:0/2619749051 shutdown_connections 2023-11-28T13:02:32.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.286+0000 7f3d826f0700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d7c161090 0x7f3d7c1485e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.286+0000 7f3d826f0700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d7c0caeb0 0x7f3d7c0c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.286+0000 7f3d826f0700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3d5c068320 0x7f3d5c06a7d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.286+0000 7f3d826f0700 1 --2- 172.21.15.6:0/2619749051 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3d7c15c990 0x7f3d7c0c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.286+0000 7f3d826f0700 1 -- 172.21.15.6:0/2619749051 >> 172.21.15.6:0/2619749051 conn(0x7f3d7c0b1650 msgr2=0x7f3d7c0cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:32.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.287+0000 7f3d826f0700 1 -- 172.21.15.6:0/2619749051 shutdown_connections 2023-11-28T13:02:32.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.287+0000 7f3d826f0700 1 -- 172.21.15.6:0/2619749051 wait complete. 2023-11-28T13:02:32.301 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:02:32.428 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.427+0000 7fb516782700 1 Processor -- start 2023-11-28T13:02:32.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.427+0000 7fb516782700 1 -- start start 2023-11-28T13:02:32.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb516782700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015a640 0x7fb5100c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb516782700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb51006b900 0x7fb5100cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb516782700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100ae730 con 0x7fb5100cd140 2023-11-28T13:02:32.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb516782700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5100cb210 con 0x7fb51006b900 2023-11-28T13:02:32.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb516782700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51006e9e0 con 0x7fb51015a640 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb50f7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb5100cd140 0x7fb51006b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48762/0 (socket says 172.21.15.6:48762) 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb50f7fe700 1 -- 172.21.15.6:0/3067506954 learned_addr learned my addr 172.21.15.6:0/3067506954 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb50ffff700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015a640 0x7fb5100c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb514d1f700 1 --2- 172.21.15.6:0/3067506954 >> v2:172.21.15.61:3300/0 conn(0x7fb51006b900 0x7fb5100cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397842235 0 0) 0x7fb5100ae730 con 0x7fb5100cd140 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.428+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4fc0035b0 con 0x7fb5100cd140 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2951592581 0 0) 0x7fb4fc0035b0 con 0x7fb5100cd140 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015a640 msgr2=0x7fb5100c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015a640 0x7fb5100c3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 >> v2:172.21.15.61:3300/0 conn(0x7fb51006b900 msgr2=0x7fb5100cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 --2- 172.21.15.6:0/3067506954 >> v2:172.21.15.61:3300/0 conn(0x7fb51006b900 0x7fb5100cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100ae980 con 0x7fb5100cd140 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fb500001670 con 0x7fb5100cd140 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 >> v1:172.21.15.6:6789/0 conn(0x7fb5100cd140 legacy=0x7fb51006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4fc003a00 0x7fb4fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4fc004430 0x7fb4fc0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015a640 0x7fb4fc004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100cb210 con 0x7fb4fc004430 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51006e9e0 con 0x7fb4fc003a00 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc00ba10 con 0x7fb51015a640 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50f7fe700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4fc004430 0x7fb4fc0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50ffff700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4fc003a00 0x7fb4fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb514d1f700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015a640 0x7fb4fc004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50f7fe700 1 -- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015a640 msgr2=0x7fb4fc004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50f7fe700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015a640 0x7fb4fc004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50f7fe700 1 -- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4fc003a00 msgr2=0x7fb4fc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50f7fe700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4fc003a00 0x7fb4fc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50f7fe700 1 -- 172.21.15.6:0/3067506954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0035b0 con 0x7fb4fc004430 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.429+0000 7fb50f7fe700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4fc004430 0x7fb4fc0048f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c370 tx=0x7fb4fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ec6dd9e68ea935 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.430+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500007c60 con 0x7fb4fc004430 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.430+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb500005960 con 0x7fb4fc004430 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.430+0000 7fb50effd700 1 -- 172.21.15.6:0/3067506954 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500005d10 con 0x7fb4fc004430 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.430+0000 7fb516782700 1 -- 172.21.15.6:0/3067506954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4fc004430 msgr2=0x7fb4fc0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.430+0000 7fb516782700 1 --2- 172.21.15.6:0/3067506954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4fc004430 0x7fb4fc0048f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c370 tx=0x7fb4fc005880 comp rx=0 tx=0).stop 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.430+0000 7fb514d1f700 1 -- 172.21.15.6:0/3067506954 reap_dead start 2023-11-28T13:02:32.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.430+0000 7fb516782700 1 -- 172.21.15.6:0/3067506954 shutdown_connections 2023-11-28T13:02:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.430+0000 7fb516782700 1 -- 172.21.15.6:0/3067506954 >> 172.21.15.6:0/3067506954 conn(0x7fb5100cc630 msgr2=0x7fb5100cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.430+0000 7fb516782700 1 -- 172.21.15.6:0/3067506954 shutdown_connections 2023-11-28T13:02:32.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.430+0000 7fb516782700 1 -- 172.21.15.6:0/3067506954 wait complete. 2023-11-28T13:02:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.431+0000 7fb516782700 1 Processor -- start 2023-11-28T13:02:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.432+0000 7fb516782700 1 -- start start 2023-11-28T13:02:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb516782700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4fc004430 0x7fb510145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb516782700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015c810 0x7fb510144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb516782700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb51015cc80 0x7fb5100c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb516782700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51015d120 con 0x7fb51015cc80 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb516782700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51015d400 con 0x7fb4fc004430 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb516782700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51015d6e0 con 0x7fb51015c810 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb514d1f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb51015cc80 0x7fb5100c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb514d1f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb51015cc80 0x7fb5100c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33224/0 (socket says 172.21.15.6:33224) 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb514d1f700 1 -- 172.21.15.6:0/2245965119 learned_addr learned my addr 172.21.15.6:0/2245965119 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb50ffff700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4fc004430 0x7fb510145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb50f7fe700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015c810 0x7fb510144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb514d1f700 1 -- 172.21.15.6:0/2245965119 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015c810 msgr2=0x7fb510144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb514d1f700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015c810 0x7fb510144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb514d1f700 1 -- 172.21.15.6:0/2245965119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4fc004430 msgr2=0x7fb510145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:32.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb514d1f700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4fc004430 0x7fb510145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb514d1f700 1 -- 172.21.15.6:0/2245965119 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100aef40 con 0x7fb51015cc80 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb514d1f700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb51015cc80 0x7fb5100c5cd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb5100c6980 tx=0x7fb50800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb50cff9700 1 -- 172.21.15.6:0/2245965119 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800e040 con 0x7fb51015cc80 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb50cff9700 1 -- 172.21.15.6:0/2245965119 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50800f040 con 0x7fb51015cc80 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb516782700 1 -- 172.21.15.6:0/2245965119 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5100aec60 con 0x7fb51015cc80 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb50cff9700 1 -- 172.21.15.6:0/2245965119 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508015430 con 0x7fb51015cc80 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.433+0000 7fb516782700 1 -- 172.21.15.6:0/2245965119 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5100af220 con 0x7fb51015cc80 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.434+0000 7fb516782700 1 -- 172.21.15.6:0/2245965119 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51015dd00 con 0x7fb51015cc80 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.435+0000 7fb50cff9700 1 -- 172.21.15.6:0/2245965119 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb508015590 con 0x7fb51015cc80 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.435+0000 7fb50cff9700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4f00682f0 0x7fb4f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.436+0000 7fb50ffff700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4f00682f0 0x7fb4f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:32.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.436+0000 7fb50cff9700 1 -- 172.21.15.6:0/2245965119 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb508086540 con 0x7fb51015cc80 2023-11-28T13:02:32.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.436+0000 7fb50ffff700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4f00682f0 0x7fb4f006a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009a60 tx=0x7fb4f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:32.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.440+0000 7fb50cff9700 1 -- 172.21.15.6:0/2245965119 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb508052c60 con 0x7fb51015cc80 2023-11-28T13:02:32.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:32.680+0000 7fb516782700 1 -- 172.21.15.6:0/2245965119 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb5100739f0 con 0x7fb51015cc80 2023-11-28T13:02:33.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.570+0000 7fb50cff9700 1 -- 172.21.15.6:0/2245965119 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb508056280 con 0x7fb51015cc80 2023-11-28T13:02:33.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 -- 172.21.15.6:0/2245965119 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4f00682f0 msgr2=0x7fb4f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:33.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4f00682f0 0x7fb4f006a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009a60 tx=0x7fb4f8008040 comp rx=0 tx=0).stop 2023-11-28T13:02:33.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 -- 172.21.15.6:0/2245965119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb51015cc80 msgr2=0x7fb5100c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:33.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb51015cc80 0x7fb5100c5cd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb5100c6980 tx=0x7fb50800f490 comp rx=0 tx=0).stop 2023-11-28T13:02:33.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 -- 172.21.15.6:0/2245965119 shutdown_connections 2023-11-28T13:02:33.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb51015cc80 0x7fb5100c5cd0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:33.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4fc004430 0x7fb510145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:33.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4f00682f0 0x7fb4f006a7a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:33.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 --2- 172.21.15.6:0/2245965119 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb51015c810 0x7fb510144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:33.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 -- 172.21.15.6:0/2245965119 >> 172.21.15.6:0/2245965119 conn(0x7fb5100cc630 msgr2=0x7fb5101e4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:33.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 -- 172.21.15.6:0/2245965119 shutdown_connections 2023-11-28T13:02:33.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.573+0000 7fb516782700 1 -- 172.21.15.6:0/2245965119 wait complete. 2023-11-28T13:02:33.589 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:02:33.590 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:02:33.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.723+0000 7f4fc487c700 1 Processor -- start 2023-11-28T13:02:33.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.723+0000 7f4fc487c700 1 -- start start 2023-11-28T13:02:33.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fc487c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4fc015c4b0 0x7f4fc015d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:33.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fc487c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4fc00c9950 0x7f4fc00cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:33.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fc487c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc00cae80 con 0x7f4fc015c880 2023-11-28T13:02:33.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fc487c700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc015dba0 con 0x7f4fc00c9950 2023-11-28T13:02:33.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fc487c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc00c9de0 con 0x7f4fc015c4b0 2023-11-28T13:02:33.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fbed9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4fc00c9950 0x7f4fc00cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:33.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fbed9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4fc00c9950 0x7f4fc00cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33232/0 (socket says 172.21.15.6:33232) 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fbed9d700 1 -- 172.21.15.6:0/1193558031 learned_addr learned my addr 172.21.15.6:0/1193558031 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fbe59c700 1 --2- 172.21.15.6:0/1193558031 >> v2:172.21.15.61:3300/0 conn(0x7f4fc015c4b0 0x7f4fc015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fbed9d700 1 -- 172.21.15.6:0/1193558031 >> v2:172.21.15.61:3300/0 conn(0x7f4fc015c4b0 msgr2=0x7f4fc015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fbed9d700 1 --2- 172.21.15.6:0/1193558031 >> v2:172.21.15.61:3300/0 conn(0x7f4fc015c4b0 0x7f4fc015d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fbed9d700 1 -- 172.21.15.6:0/1193558031 >> v1:172.21.15.61:6789/0 conn(0x7f4fc015c880 legacy=0x7f4fc006a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fbed9d700 1 -- 172.21.15.6:0/1193558031 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00ca0c0 con 0x7f4fc00c9950 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.724+0000 7f4fbe59c700 1 --2- 172.21.15.6:0/1193558031 >> v2:172.21.15.61:3300/0 conn(0x7f4fc015c4b0 0x7f4fc015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fbed9d700 1 --2- 172.21.15.6:0/1193558031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 0x7f4fc00cced0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0070df0 tx=0x7f4fb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d996b29b02f8617c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fbd59a700 1 -- 172.21.15.6:0/1193558031 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800e070 con 0x7f4fc00c9950 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fbd59a700 1 -- 172.21.15.6:0/1193558031 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb800b9c0 con 0x7f4fc00c9950 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fbd59a700 1 -- 172.21.15.6:0/1193558031 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800bd70 con 0x7f4fc00c9950 2023-11-28T13:02:33.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fc487c700 1 -- 172.21.15.6:0/1193558031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 msgr2=0x7f4fc00cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:33.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fc487c700 1 --2- 172.21.15.6:0/1193558031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 0x7f4fc00cced0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0070df0 tx=0x7f4fb800b330 comp rx=0 tx=0).stop 2023-11-28T13:02:33.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fc487c700 1 -- 172.21.15.6:0/1193558031 shutdown_connections 2023-11-28T13:02:33.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fc487c700 1 --2- 172.21.15.6:0/1193558031 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 0x7f4fc00cced0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:33.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fc487c700 1 --2- 172.21.15.6:0/1193558031 >> v2:172.21.15.61:3300/0 conn(0x7f4fc015c4b0 0x7f4fc015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:33.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fc487c700 1 -- 172.21.15.6:0/1193558031 >> 172.21.15.6:0/1193558031 conn(0x7f4fc00b0df0 msgr2=0x7f4fc00b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:33.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fc487c700 1 -- 172.21.15.6:0/1193558031 shutdown_connections 2023-11-28T13:02:33.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.725+0000 7f4fc487c700 1 -- 172.21.15.6:0/1193558031 wait complete. 2023-11-28T13:02:33.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.727+0000 7f4fc487c700 1 Processor -- start 2023-11-28T13:02:33.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.727+0000 7f4fc487c700 1 -- start start 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fc487c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 0x7f4fc01598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fc487c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fc015c4b0 0x7f4fc00b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fc487c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fc015c880 0x7f4fc00ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fc487c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00c3270 con 0x7f4fc00c9950 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fc487c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc0159df0 con 0x7f4fc015c4b0 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fc487c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc00c2ed0 con 0x7f4fc015c880 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbe59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 0x7f4fc01598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbe59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 0x7f4fc01598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33234/0 (socket says 172.21.15.6:33234) 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbe59c700 1 -- 172.21.15.6:0/2230360470 learned_addr learned my addr 172.21.15.6:0/2230360470 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbdd9b700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fc015c4b0 0x7f4fc00b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:33.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbed9d700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fc015c880 0x7f4fc00ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbe59c700 1 -- 172.21.15.6:0/2230360470 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fc015c880 msgr2=0x7f4fc00ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbe59c700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fc015c880 0x7f4fc00ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbe59c700 1 -- 172.21.15.6:0/2230360470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fc015c4b0 msgr2=0x7f4fc00b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbe59c700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fc015c4b0 0x7f4fc00b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbe59c700 1 -- 172.21.15.6:0/2230360470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8009b30 con 0x7f4fc00c9950 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbdd9b700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fc015c4b0 0x7f4fc00b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbed9d700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fc015c880 0x7f4fc00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fbe59c700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 0x7f4fc01598b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400e810 tx=0x7f4fb400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4faeffd700 1 -- 172.21.15.6:0/2230360470 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4004230 con 0x7f4fc00c9950 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4faeffd700 1 -- 172.21.15.6:0/2230360470 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb4004390 con 0x7f4fc00c9950 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fc487c700 1 -- 172.21.15.6:0/2230360470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc00ca680 con 0x7f4fc00c9950 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4faeffd700 1 -- 172.21.15.6:0/2230360470 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40036a0 con 0x7f4fc00c9950 2023-11-28T13:02:33.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.728+0000 7f4fc487c700 1 -- 172.21.15.6:0/2230360470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc00ca3a0 con 0x7f4fc00c9950 2023-11-28T13:02:33.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.729+0000 7f4fc487c700 1 -- 172.21.15.6:0/2230360470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc00ca960 con 0x7f4fc00c9950 2023-11-28T13:02:33.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.730+0000 7f4faeffd700 1 -- 172.21.15.6:0/2230360470 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4fb4022070 con 0x7f4fc00c9950 2023-11-28T13:02:33.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.730+0000 7f4faeffd700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4fa00682f0 0x7f4fa006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:33.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.730+0000 7f4faeffd700 1 -- 172.21.15.6:0/2230360470 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4fb4085910 con 0x7f4fc00c9950 2023-11-28T13:02:33.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.730+0000 7f4fbdd9b700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4fa00682f0 0x7f4fa006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:33.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.731+0000 7f4fbdd9b700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4fa00682f0 0x7f4fa006a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00c5e30 tx=0x7f4fb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:33.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.735+0000 7f4faeffd700 1 -- 172.21.15.6:0/2230360470 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4fb4051f50 con 0x7f4fc00c9950 2023-11-28T13:02:33.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:33.977+0000 7f4fc487c700 1 -- 172.21.15.6:0/2230360470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4fc00c34e0 con 0x7f4fc00c9950 2023-11-28T13:02:34.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.941+0000 7f4faeffd700 1 -- 172.21.15.6:0/2230360470 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4fb4055570 con 0x7f4fc00c9950 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 -- 172.21.15.6:0/2230360470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4fa00682f0 msgr2=0x7f4fa006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4fa00682f0 0x7f4fa006a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00c5e30 tx=0x7f4fb0009040 comp rx=0 tx=0).stop 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 -- 172.21.15.6:0/2230360470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 msgr2=0x7f4fc01598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 0x7f4fc01598b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400e810 tx=0x7f4fb400ebd0 comp rx=0 tx=0).stop 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 -- 172.21.15.6:0/2230360470 shutdown_connections 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4fa00682f0 0x7f4fa006a7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4fc015c880 0x7f4fc00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4fc00c9950 0x7f4fc01598b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 --2- 172.21.15.6:0/2230360470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4fc015c4b0 0x7f4fc00b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 -- 172.21.15.6:0/2230360470 >> 172.21.15.6:0/2230360470 conn(0x7f4fc00b0df0 msgr2=0x7f4fc00cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:34.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 -- 172.21.15.6:0/2230360470 shutdown_connections 2023-11-28T13:02:34.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:34.944+0000 7f4fc487c700 1 -- 172.21.15.6:0/2230360470 wait complete. 2023-11-28T13:02:34.961 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:02:34.961 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:35.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.092+0000 7f5f52680700 1 Processor -- start 2023-11-28T13:02:35.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.092+0000 7f5f52680700 1 -- start start 2023-11-28T13:02:35.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.093+0000 7f5f52680700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5f4c15c990 0x7f5f4c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.093+0000 7f5f52680700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c0ce340 con 0x7f5f4c161090 2023-11-28T13:02:35.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.093+0000 7f5f52680700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c0cebe0 con 0x7f5f4c0cc310 2023-11-28T13:02:35.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.093+0000 7f5f52680700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c0cb3f0 con 0x7f5f4c15c990 2023-11-28T13:02:35.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.093+0000 7f5f4bfff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5f4c15c990 0x7f5f4c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.093+0000 7f5f4b7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f5f4c161090 0x7f5f4c0ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48794/0 (socket says 172.21.15.6:48794) 2023-11-28T13:02:35.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.093+0000 7f5f4bfff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5f4c15c990 0x7f5f4c0cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33242/0 (socket says 172.21.15.6:33242) 2023-11-28T13:02:35.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.093+0000 7f5f4b7fe700 1 -- 172.21.15.6:0/953741900 learned_addr learned my addr 172.21.15.6:0/953741900 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:35.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.094+0000 7f5f4bfff700 1 -- 172.21.15.6:0/953741900 >> v1:172.21.15.61:6789/0 conn(0x7f5f4c0cc310 legacy=0x7f5f4c0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.094+0000 7f5f4bfff700 1 -- 172.21.15.6:0/953741900 >> v1:172.21.15.6:6789/0 conn(0x7f5f4c161090 legacy=0x7f5f4c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.094+0000 7f5f4affd700 1 -- 172.21.15.6:0/953741900 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312438195 0 0) 0x7f5f4c0ce340 con 0x7f5f4c161090 2023-11-28T13:02:35.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.094+0000 7f5f4bfff700 1 -- 172.21.15.6:0/953741900 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0ce590 con 0x7f5f4c15c990 2023-11-28T13:02:35.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.094+0000 7f5f4bfff700 1 --2- 172.21.15.6:0/953741900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 0x7f5f4c0cf070 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009840 tx=0x7f5f3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81582c35821e782f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.094+0000 7f5f4affd700 1 -- 172.21.15.6:0/953741900 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c004230 con 0x7f5f4c15c990 2023-11-28T13:02:35.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.094+0000 7f5f4affd700 1 -- 172.21.15.6:0/953741900 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c004390 con 0x7f5f4c15c990 2023-11-28T13:02:35.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.094+0000 7f5f4affd700 1 -- 172.21.15.6:0/953741900 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0036a0 con 0x7f5f4c15c990 2023-11-28T13:02:35.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.095+0000 7f5f52680700 1 -- 172.21.15.6:0/953741900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 msgr2=0x7f5f4c0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.095+0000 7f5f52680700 1 --2- 172.21.15.6:0/953741900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 0x7f5f4c0cf070 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009840 tx=0x7f5f3c009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:35.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.095+0000 7f5f52680700 1 -- 172.21.15.6:0/953741900 shutdown_connections 2023-11-28T13:02:35.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.095+0000 7f5f52680700 1 --2- 172.21.15.6:0/953741900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 0x7f5f4c0cf070 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.095+0000 7f5f52680700 1 -- 172.21.15.6:0/953741900 >> 172.21.15.6:0/953741900 conn(0x7f5f4c0b1650 msgr2=0x7f5f4c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.095+0000 7f5f52680700 1 -- 172.21.15.6:0/953741900 shutdown_connections 2023-11-28T13:02:35.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.095+0000 7f5f52680700 1 -- 172.21.15.6:0/953741900 wait complete. 2023-11-28T13:02:35.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.096+0000 7f5f52680700 1 Processor -- start 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.097+0000 7f5f52680700 1 -- start start 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.097+0000 7f5f52680700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f4c0cc310 0x7f5f4c148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.097+0000 7f5f52680700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 0x7f5f4c15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.097+0000 7f5f52680700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f4c161090 0x7f5f4c15f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.097+0000 7f5f52680700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c15f830 con 0x7f5f4c15c990 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.097+0000 7f5f52680700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c15fb10 con 0x7f5f4c161090 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.097+0000 7f5f52680700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c15fdf0 con 0x7f5f4c0cc310 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 0x7f5f4c15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f4b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 0x7f5f4c15ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33250/0 (socket says 172.21.15.6:33250) 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f4b7fe700 1 -- 172.21.15.6:0/2299160279 learned_addr learned my addr 172.21.15.6:0/2299160279 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:35.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f50c1d700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f4c161090 0x7f5f4c15f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f4bfff700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f4c0cc310 0x7f5f4c148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f4b7fe700 1 -- 172.21.15.6:0/2299160279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f4c0cc310 msgr2=0x7f5f4c148960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f4b7fe700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f4c0cc310 0x7f5f4c148960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f4b7fe700 1 -- 172.21.15.6:0/2299160279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f4c161090 msgr2=0x7f5f4c15f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f4b7fe700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f4c161090 0x7f5f4c15f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f4b7fe700 1 -- 172.21.15.6:0/2299160279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0094a0 con 0x7f5f4c15c990 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f4b7fe700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 0x7f5f4c15ec70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5f340011e0 tx=0x7f5f3400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f48ff9700 1 -- 172.21.15.6:0/2299160279 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34004c80 con 0x7f5f4c15c990 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f48ff9700 1 -- 172.21.15.6:0/2299160279 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34004e00 con 0x7f5f4c15c990 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f52680700 1 -- 172.21.15.6:0/2299160279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c0cb080 con 0x7f5f4c15c990 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f48ff9700 1 -- 172.21.15.6:0/2299160279 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34007310 con 0x7f5f4c15c990 2023-11-28T13:02:35.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.098+0000 7f5f52680700 1 -- 172.21.15.6:0/2299160279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c0ce870 con 0x7f5f4c15c990 2023-11-28T13:02:35.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.099+0000 7f5f52680700 1 -- 172.21.15.6:0/2299160279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c0cf5b0 con 0x7f5f4c15c990 2023-11-28T13:02:35.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.100+0000 7f5f48ff9700 1 -- 172.21.15.6:0/2299160279 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5f3401a020 con 0x7f5f4c15c990 2023-11-28T13:02:35.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.100+0000 7f5f48ff9700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5f2c0682f0 0x7f5f2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.100+0000 7f5f4bfff700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5f2c0682f0 0x7f5f2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.100+0000 7f5f48ff9700 1 -- 172.21.15.6:0/2299160279 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5f34085900 con 0x7f5f4c15c990 2023-11-28T13:02:35.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.101+0000 7f5f4bfff700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5f2c0682f0 0x7f5f2c06a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0090b0 tx=0x7f5f3c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.105+0000 7f5f48ff9700 1 -- 172.21.15.6:0/2299160279 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5f34052020 con 0x7f5f4c15c990 2023-11-28T13:02:35.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.342+0000 7f5f52680700 1 -- 172.21.15.6:0/2299160279 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f4c160440 con 0x7f5f4c15c990 2023-11-28T13:02:35.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.342+0000 7f5f48ff9700 1 -- 172.21.15.6:0/2299160279 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4636 (secure 0 0 0) 0x7f5f4c160440 con 0x7f5f4c15c990 2023-11-28T13:02:35.344 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:35.344 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4004512137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4004512137},{"type":"v1","addr":"172.21.15.61:6833","nonce":4004512137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/1313775424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":1313775424},{"type":"v1","addr":"172.21.15.61:6835","nonce":1313775424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/233781200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":233781200},{"type":"v1","addr":"172.21.15.6:6836","nonce":233781200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:31.562404+0000","modified":"2023-11-28T13:02:34.939442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/451011578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":451011578},{"type":"v1","addr":"172.21.15.6:6837","nonce":451011578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:35.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.344+0000 7f5f52680700 1 -- 172.21.15.6:0/2299160279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5f2c0682f0 msgr2=0x7f5f2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.344+0000 7f5f52680700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5f2c0682f0 0x7f5f2c06a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0090b0 tx=0x7f5f3c008030 comp rx=0 tx=0).stop 2023-11-28T13:02:35.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.344+0000 7f5f52680700 1 -- 172.21.15.6:0/2299160279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 msgr2=0x7f5f4c15ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.344+0000 7f5f52680700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 0x7f5f4c15ec70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5f340011e0 tx=0x7f5f3400ca30 comp rx=0 tx=0).stop 2023-11-28T13:02:35.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.345+0000 7f5f52680700 1 -- 172.21.15.6:0/2299160279 shutdown_connections 2023-11-28T13:02:35.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.345+0000 7f5f52680700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f4c161090 0x7f5f4c15f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.345+0000 7f5f52680700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5f2c0682f0 0x7f5f2c06a7a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.345+0000 7f5f52680700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5f4c0cc310 0x7f5f4c148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.345+0000 7f5f52680700 1 --2- 172.21.15.6:0/2299160279 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f4c15c990 0x7f5f4c15ec70 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.345+0000 7f5f52680700 1 -- 172.21.15.6:0/2299160279 >> 172.21.15.6:0/2299160279 conn(0x7f5f4c0b1650 msgr2=0x7f5f4c065ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.345+0000 7f5f52680700 1 -- 172.21.15.6:0/2299160279 shutdown_connections 2023-11-28T13:02:35.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.345+0000 7f5f52680700 1 -- 172.21.15.6:0/2299160279 wait complete. 2023-11-28T13:02:35.348 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2023-11-28T13:02:35.358 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:35.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.481+0000 7fe46c443700 1 Processor -- start 2023-11-28T13:02:35.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.481+0000 7fe46c443700 1 -- start start 2023-11-28T13:02:35.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.482+0000 7fe46c443700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe46406bc30 0x7fe46415b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.482+0000 7fe46c443700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe46406dda0 0x7fe46415e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.482+0000 7fe46c443700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46406c3a0 con 0x7fe46406d9d0 2023-11-28T13:02:35.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.482+0000 7fe46c443700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46415ee90 con 0x7fe46406bc30 2023-11-28T13:02:35.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.482+0000 7fe46c443700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe46406c0c0 con 0x7fe46406dda0 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4699de700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe46406bc30 0x7fe46415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4699de700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe46406bc30 0x7fe46415b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33258/0 (socket says 172.21.15.6:33258) 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4699de700 1 -- 172.21.15.6:0/2304171751 learned_addr learned my addr 172.21.15.6:0/2304171751 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe46a1df700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe46406d9d0 0x7fe464150eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48808/0 (socket says 172.21.15.6:48808) 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe46a9e0700 1 --2- 172.21.15.6:0/2304171751 >> v2:172.21.15.61:3300/0 conn(0x7fe46406dda0 0x7fe46415e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4699de700 1 -- 172.21.15.6:0/2304171751 >> v2:172.21.15.61:3300/0 conn(0x7fe46406dda0 msgr2=0x7fe46415e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4699de700 1 --2- 172.21.15.6:0/2304171751 >> v2:172.21.15.61:3300/0 conn(0x7fe46406dda0 0x7fe46415e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4699de700 1 -- 172.21.15.6:0/2304171751 >> v1:172.21.15.6:6789/0 conn(0x7fe46406d9d0 legacy=0x7fe464150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4699de700 1 -- 172.21.15.6:0/2304171751 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46415ab60 con 0x7fe46406bc30 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4699de700 1 --2- 172.21.15.6:0/2304171751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406bc30 0x7fe46415b640 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe46406e430 tx=0x7fe460009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6bdb0932438d14 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.486 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4691dd700 1 -- 172.21.15.6:0/2304171751 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460010a90 con 0x7fe46406bc30 2023-11-28T13:02:35.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4691dd700 1 -- 172.21.15.6:0/2304171751 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe460010bf0 con 0x7fe46406bc30 2023-11-28T13:02:35.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.483+0000 7fe4691dd700 1 -- 172.21.15.6:0/2304171751 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460018440 con 0x7fe46406bc30 2023-11-28T13:02:35.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.484+0000 7fe46c443700 1 -- 172.21.15.6:0/2304171751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406bc30 msgr2=0x7fe46415b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.484+0000 7fe46c443700 1 --2- 172.21.15.6:0/2304171751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406bc30 0x7fe46415b640 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe46406e430 tx=0x7fe460009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.484+0000 7fe46c443700 1 -- 172.21.15.6:0/2304171751 shutdown_connections 2023-11-28T13:02:35.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.484+0000 7fe46c443700 1 --2- 172.21.15.6:0/2304171751 >> v2:172.21.15.61:3300/0 conn(0x7fe46406dda0 0x7fe46415e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.484+0000 7fe46c443700 1 --2- 172.21.15.6:0/2304171751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406bc30 0x7fe46415b640 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.484+0000 7fe46c443700 1 -- 172.21.15.6:0/2304171751 >> 172.21.15.6:0/2304171751 conn(0x7fe4640b0dd0 msgr2=0x7fe4640b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.484+0000 7fe46c443700 1 -- 172.21.15.6:0/2304171751 shutdown_connections 2023-11-28T13:02:35.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.484+0000 7fe46c443700 1 -- 172.21.15.6:0/2304171751 wait complete. 2023-11-28T13:02:35.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.485+0000 7fe46c443700 1 Processor -- start 2023-11-28T13:02:35.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.485+0000 7fe46c443700 1 -- start start 2023-11-28T13:02:35.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe46c443700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe46406bc30 0x7fe464147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe46c443700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406d9d0 0x7fe4640ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe46c443700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe46406dda0 0x7fe4640ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe46c443700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640b2110 con 0x7fe46406d9d0 2023-11-28T13:02:35.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe46c443700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe464147ca0 con 0x7fe46406bc30 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe46c443700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe464157a80 con 0x7fe46406dda0 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe4699de700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406d9d0 0x7fe4640ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe4699de700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406d9d0 0x7fe4640ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33264/0 (socket says 172.21.15.6:33264) 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe4699de700 1 -- 172.21.15.6:0/2233883166 learned_addr learned my addr 172.21.15.6:0/2233883166 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe46a1df700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe46406bc30 0x7fe464147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe46a9e0700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe46406dda0 0x7fe4640ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe4699de700 1 -- 172.21.15.6:0/2233883166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe46406dda0 msgr2=0x7fe4640ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe4699de700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe46406dda0 0x7fe4640ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe4699de700 1 -- 172.21.15.6:0/2233883166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe46406bc30 msgr2=0x7fe464147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe4699de700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe46406bc30 0x7fe464147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.486+0000 7fe4699de700 1 -- 172.21.15.6:0/2233883166 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe460009550 con 0x7fe46406d9d0 2023-11-28T13:02:35.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.487+0000 7fe4699de700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406d9d0 0x7fe4640ce1b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe460000c00 tx=0x7fe460004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.487+0000 7fe456ffd700 1 -- 172.21.15.6:0/2233883166 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460018440 con 0x7fe46406d9d0 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.487+0000 7fe456ffd700 1 -- 172.21.15.6:0/2233883166 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe460003ab0 con 0x7fe46406d9d0 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.487+0000 7fe46c443700 1 -- 172.21.15.6:0/2233883166 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46415b120 con 0x7fe46406d9d0 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.487+0000 7fe456ffd700 1 -- 172.21.15.6:0/2233883166 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460003e60 con 0x7fe46406d9d0 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.487+0000 7fe46c443700 1 -- 172.21.15.6:0/2233883166 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46415ae40 con 0x7fe46406d9d0 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.488+0000 7fe46c443700 1 -- 172.21.15.6:0/2233883166 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46406fac0 con 0x7fe46406d9d0 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.488+0000 7fe456ffd700 1 -- 172.21.15.6:0/2233883166 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe4600185f0 con 0x7fe46406d9d0 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.488+0000 7fe456ffd700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4480682f0 0x7fe44806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.489+0000 7fe456ffd700 1 -- 172.21.15.6:0/2233883166 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe46008b9b0 con 0x7fe46406d9d0 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.489+0000 7fe46a1df700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4480682f0 0x7fe44806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.489+0000 7fe46a1df700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4480682f0 0x7fe44806a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe464155b60 tx=0x7fe458007f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.493+0000 7fe456ffd700 1 -- 172.21.15.6:0/2233883166 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe460057ff0 con 0x7fe46406d9d0 2023-11-28T13:02:35.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.689+0000 7fe46c443700 1 -- 172.21.15.6:0/2233883166 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4640675a0 con 0x7fe46406d9d0 2023-11-28T13:02:35.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.690+0000 7fe456ffd700 1 -- 172.21.15.6:0/2233883166 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18026 (secure 0 0 0) 0x7fe46005b610 con 0x7fe46406d9d0 2023-11-28T13:02:35.692 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:35.692 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":54,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:32.564104+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:29.490442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:30.534532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:35.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 -- 172.21.15.6:0/2233883166 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4480682f0 msgr2=0x7fe44806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4480682f0 0x7fe44806a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe464155b60 tx=0x7fe458007f10 comp rx=0 tx=0).stop 2023-11-28T13:02:35.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 -- 172.21.15.6:0/2233883166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406d9d0 msgr2=0x7fe4640ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406d9d0 0x7fe4640ce1b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe460000c00 tx=0x7fe460004750 comp rx=0 tx=0).stop 2023-11-28T13:02:35.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 -- 172.21.15.6:0/2233883166 shutdown_connections 2023-11-28T13:02:35.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4480682f0 0x7fe44806a7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe46406dda0 0x7fe4640ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe46406bc30 0x7fe464147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 --2- 172.21.15.6:0/2233883166 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe46406d9d0 0x7fe4640ce1b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 -- 172.21.15.6:0/2233883166 >> 172.21.15.6:0/2233883166 conn(0x7fe4640b0dd0 msgr2=0x7fe46415c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 -- 172.21.15.6:0/2233883166 shutdown_connections 2023-11-28T13:02:35.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.692+0000 7fe46c443700 1 -- 172.21.15.6:0/2233883166 wait complete. 2023-11-28T13:02:35.706 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:02:35.706 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:02:35.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.829+0000 7f70be3d2700 1 Processor -- start 2023-11-28T13:02:35.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.829+0000 7f70be3d2700 1 -- start start 2023-11-28T13:02:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70be3d2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 0x7f70b80cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70be3d2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f70b8161090 0x7f70b80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70be3d2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f70b80caeb0 0x7f70b80cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70be3d2700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b80ce340 con 0x7f70b80caeb0 2023-11-28T13:02:35.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70be3d2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b80cebe0 con 0x7f70b8161090 2023-11-28T13:02:35.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70be3d2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80cb3f0 con 0x7f70b815c990 2023-11-28T13:02:35.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70bc96f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f70b80caeb0 0x7f70b80cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70bc96f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f70b80caeb0 0x7f70b80cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33274/0 (socket says 172.21.15.6:33274) 2023-11-28T13:02:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70bc96f700 1 -- 172.21.15.6:0/1123448716 learned_addr learned my addr 172.21.15.6:0/1123448716 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70b77fe700 1 --2- 172.21.15.6:0/1123448716 >> v2:172.21.15.61:3300/0 conn(0x7f70b8161090 0x7f70b80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70b7fff700 1 --2- 172.21.15.6:0/1123448716 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 0x7f70b80cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70bc96f700 1 -- 172.21.15.6:0/1123448716 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 msgr2=0x7f70b80cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70bc96f700 1 --2- 172.21.15.6:0/1123448716 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 0x7f70b80cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70bc96f700 1 -- 172.21.15.6:0/1123448716 >> v2:172.21.15.61:3300/0 conn(0x7f70b8161090 msgr2=0x7f70b80ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70bc96f700 1 --2- 172.21.15.6:0/1123448716 >> v2:172.21.15.61:3300/0 conn(0x7f70b8161090 0x7f70b80ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70bc96f700 1 -- 172.21.15.6:0/1123448716 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b80ce590 con 0x7f70b80caeb0 2023-11-28T13:02:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70b77fe700 1 --2- 172.21.15.6:0/1123448716 >> v2:172.21.15.61:3300/0 conn(0x7f70b8161090 0x7f70b80ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.830+0000 7f70bc96f700 1 --2- 172.21.15.6:0/1123448716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 0x7f70b80cc310 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f70b806f850 tx=0x7f70b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff69b794edb836e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70b6ffd700 1 -- 172.21.15.6:0/1123448716 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000e070 con 0x7f70b80caeb0 2023-11-28T13:02:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70b6ffd700 1 -- 172.21.15.6:0/1123448716 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b000b9c0 con 0x7f70b80caeb0 2023-11-28T13:02:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70b6ffd700 1 -- 172.21.15.6:0/1123448716 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000bd70 con 0x7f70b80caeb0 2023-11-28T13:02:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70be3d2700 1 -- 172.21.15.6:0/1123448716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 msgr2=0x7f70b80cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70be3d2700 1 --2- 172.21.15.6:0/1123448716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 0x7f70b80cc310 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f70b806f850 tx=0x7f70b000b330 comp rx=0 tx=0).stop 2023-11-28T13:02:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70be3d2700 1 -- 172.21.15.6:0/1123448716 shutdown_connections 2023-11-28T13:02:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70be3d2700 1 --2- 172.21.15.6:0/1123448716 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 0x7f70b80cc310 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70be3d2700 1 --2- 172.21.15.6:0/1123448716 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 0x7f70b80cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70be3d2700 1 --2- 172.21.15.6:0/1123448716 >> v2:172.21.15.61:3300/0 conn(0x7f70b8161090 0x7f70b80ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70be3d2700 1 -- 172.21.15.6:0/1123448716 >> 172.21.15.6:0/1123448716 conn(0x7f70b80b1650 msgr2=0x7f70b80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:35.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70be3d2700 1 -- 172.21.15.6:0/1123448716 shutdown_connections 2023-11-28T13:02:35.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.831+0000 7f70be3d2700 1 -- 172.21.15.6:0/1123448716 wait complete. 2023-11-28T13:02:35.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.832+0000 7f70be3d2700 1 Processor -- start 2023-11-28T13:02:35.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.833+0000 7f70be3d2700 1 -- start start 2023-11-28T13:02:35.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.833+0000 7f70be3d2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 0x7f70b81e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.833+0000 7f70be3d2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 0x7f70b815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.833+0000 7f70be3d2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70b8161090 0x7f70b80c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.833+0000 7f70be3d2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80b3f30 con 0x7f70b80caeb0 2023-11-28T13:02:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.833+0000 7f70be3d2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80b3320 con 0x7f70b8161090 2023-11-28T13:02:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.833+0000 7f70be3d2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80b3600 con 0x7f70b815c990 2023-11-28T13:02:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.833+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 0x7f70b81e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.833+0000 7f70b7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 0x7f70b81e8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33276/0 (socket says 172.21.15.6:33276) 2023-11-28T13:02:35.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.833+0000 7f70b7fff700 1 -- 172.21.15.6:0/3626448236 learned_addr learned my addr 172.21.15.6:0/3626448236 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70bc96f700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70b8161090 0x7f70b80c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b77fe700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 0x7f70b815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b7fff700 1 -- 172.21.15.6:0/3626448236 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 msgr2=0x7f70b815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b7fff700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 0x7f70b815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b7fff700 1 -- 172.21.15.6:0/3626448236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70b8161090 msgr2=0x7f70b80c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b7fff700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70b8161090 0x7f70b80c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b7fff700 1 -- 172.21.15.6:0/3626448236 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b0009b30 con 0x7f70b80caeb0 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b77fe700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 0x7f70b815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b7fff700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 0x7f70b81e8630 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f70a000e810 tx=0x7f70a000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b4ff9700 1 -- 172.21.15.6:0/3626448236 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0013610 con 0x7f70b80caeb0 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b4ff9700 1 -- 172.21.15.6:0/3626448236 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a0013790 con 0x7f70b80caeb0 2023-11-28T13:02:35.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70b4ff9700 1 -- 172.21.15.6:0/3626448236 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0013b40 con 0x7f70b80caeb0 2023-11-28T13:02:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70be3d2700 1 -- 172.21.15.6:0/3626448236 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b80cf5b0 con 0x7f70b80caeb0 2023-11-28T13:02:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.834+0000 7f70be3d2700 1 -- 172.21.15.6:0/3626448236 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b80ce870 con 0x7f70b80caeb0 2023-11-28T13:02:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.835+0000 7f70be3d2700 1 -- 172.21.15.6:0/3626448236 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b80cf740 con 0x7f70b80caeb0 2023-11-28T13:02:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.836+0000 7f70b4ff9700 1 -- 172.21.15.6:0/3626448236 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f70a0023070 con 0x7f70b80caeb0 2023-11-28T13:02:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.836+0000 7f70b4ff9700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f70980682f0 0x7f709806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.836+0000 7f70b4ff9700 1 -- 172.21.15.6:0/3626448236 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f70a0085960 con 0x7f70b80caeb0 2023-11-28T13:02:35.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.836+0000 7f70b77fe700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f70980682f0 0x7f709806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.837+0000 7f70b77fe700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f70980682f0 0x7f709806a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f70b80b2eb0 tx=0x7f70a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:35.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:35.840+0000 7f70b4ff9700 1 -- 172.21.15.6:0/3626448236 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f70a0052080 con 0x7f70b80caeb0 2023-11-28T13:02:36.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.029+0000 7f70be3d2700 1 -- 172.21.15.6:0/3626448236 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70b80b4070 con 0x7f70980682f0 2023-11-28T13:02:36.038 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.037+0000 7f70b4ff9700 1 -- 172.21.15.6:0/3626448236 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311899 (secure 0 0 0) 0x7f70a800ce00 con 0x7f70980682f0 2023-11-28T13:02:36.042 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:36.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.040+0000 7f70be3d2700 1 -- 172.21.15.6:0/3626448236 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f70980682f0 msgr2=0x7f709806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.040+0000 7f70be3d2700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f70980682f0 0x7f709806a7a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f70b80b2eb0 tx=0x7f70a8008040 comp rx=0 tx=0).stop 2023-11-28T13:02:36.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.040+0000 7f70be3d2700 1 -- 172.21.15.6:0/3626448236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 msgr2=0x7f70b81e8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.040+0000 7f70be3d2700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 0x7f70b81e8630 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f70a000e810 tx=0x7f70a000eb20 comp rx=0 tx=0).stop 2023-11-28T13:02:36.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.040+0000 7f70be3d2700 1 -- 172.21.15.6:0/3626448236 shutdown_connections 2023-11-28T13:02:36.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.040+0000 7f70be3d2700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f70b8161090 0x7f70b80c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.040+0000 7f70be3d2700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f70b80caeb0 0x7f70b81e8630 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.040+0000 7f70be3d2700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f70980682f0 0x7f709806a7a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.040+0000 7f70be3d2700 1 --2- 172.21.15.6:0/3626448236 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f70b815c990 0x7f70b815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.040+0000 7f70be3d2700 1 -- 172.21.15.6:0/3626448236 >> 172.21.15.6:0/3626448236 conn(0x7f70b80b1650 msgr2=0x7f70b806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:36.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.041+0000 7f70be3d2700 1 -- 172.21.15.6:0/3626448236 shutdown_connections 2023-11-28T13:02:36.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.041+0000 7f70be3d2700 1 -- 172.21.15.6:0/3626448236 wait complete. 2023-11-28T13:02:36.046 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:02:36.055 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2023-11-28T13:02:34.970866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":76064,"kb_used_data":6240,"kb_used_omap":0,"kb_used_meta":69760,"kb_avail":754898656,"statfs":{"total":773094113280,"available":773016223744,"internally_reserved":0,"allocated":6389760,"data_stored":3574589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71434240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.837796"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392564+0000","last_change":"2023-11-28T13:02:31.591399+0000","last_active":"2023-11-28T13:02:33.392564+0000","last_peered":"2023-11-28T13:02:33.392564+0000","last_clean":"2023-11-28T13:02:33.392564+0000","last_became_active":"2023-11-28T13:02:31.591062+0000","last_became_peered":"2023-11-28T13:02:31.591062+0000","last_unstale":"2023-11-28T13:02:33.392564+0000","last_undegraded":"2023-11-28T13:02:33.392564+0000","last_fullsized":"2023-11-28T13:02:33.392564+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:32.296512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390747+0000","last_change":"2023-11-28T13:02:31.583920+0000","last_active":"2023-11-28T13:02:33.390747+0000","last_peered":"2023-11-28T13:02:33.390747+0000","last_clean":"2023-11-28T13:02:33.390747+0000","last_became_active":"2023-11-28T13:02:31.583643+0000","last_became_peered":"2023-11-28T13:02:31.583643+0000","last_unstale":"2023-11-28T13:02:33.390747+0000","last_undegraded":"2023-11-28T13:02:33.390747+0000","last_fullsized":"2023-11-28T13:02:33.390747+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:41.354914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571938+0000","last_change":"2023-11-28T13:02:31.585521+0000","last_active":"2023-11-28T13:02:32.571938+0000","last_peered":"2023-11-28T13:02:32.571938+0000","last_clean":"2023-11-28T13:02:32.571938+0000","last_became_active":"2023-11-28T13:02:31.585222+0000","last_became_peered":"2023-11-28T13:02:31.585222+0000","last_unstale":"2023-11-28T13:02:32.571938+0000","last_undegraded":"2023-11-28T13:02:32.571938+0000","last_fullsized":"2023-11-28T13:02:32.571938+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:57.159327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571873+0000","last_change":"2023-11-28T13:02:31.586065+0000","last_active":"2023-11-28T13:02:32.571873+0000","last_peered":"2023-11-28T13:02:32.571873+0000","last_clean":"2023-11-28T13:02:32.571873+0000","last_became_active":"2023-11-28T13:02:31.585791+0000","last_became_peered":"2023-11-28T13:02:31.585791+0000","last_unstale":"2023-11-28T13:02:32.571873+0000","last_undegraded":"2023-11-28T13:02:32.571873+0000","last_fullsized":"2023-11-28T13:02:32.571873+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:45.032643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392445+0000","last_change":"2023-11-28T13:02:31.588039+0000","last_active":"2023-11-28T13:02:33.392445+0000","last_peered":"2023-11-28T13:02:33.392445+0000","last_clean":"2023-11-28T13:02:33.392445+0000","last_became_active":"2023-11-28T13:02:31.587794+0000","last_became_peered":"2023-11-28T13:02:31.587794+0000","last_unstale":"2023-11-28T13:02:33.392445+0000","last_undegraded":"2023-11-28T13:02:33.392445+0000","last_fullsized":"2023-11-28T13:02:33.392445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:43.148353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391918+0000","last_change":"2023-11-28T13:02:31.590953+0000","last_active":"2023-11-28T13:02:33.391918+0000","last_peered":"2023-11-28T13:02:33.391918+0000","last_clean":"2023-11-28T13:02:33.391918+0000","last_became_active":"2023-11-28T13:02:31.586161+0000","last_became_peered":"2023-11-28T13:02:31.586161+0000","last_unstale":"2023-11-28T13:02:33.391918+0000","last_undegraded":"2023-11-28T13:02:33.391918+0000","last_fullsized":"2023-11-28T13:02:33.391918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:41.354914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641741+0000","last_change":"2023-11-28T13:02:31.588103+0000","last_active":"2023-11-28T13:02:32.641741+0000","last_peered":"2023-11-28T13:02:32.641741+0000","last_clean":"2023-11-28T13:02:32.641741+0000","last_became_active":"2023-11-28T13:02:31.587650+0000","last_became_peered":"2023-11-28T13:02:31.587650+0000","last_unstale":"2023-11-28T13:02:32.641741+0000","last_undegraded":"2023-11-28T13:02:32.641741+0000","last_fullsized":"2023-11-28T13:02:32.641741+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:12:37.986813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390455+0000","last_change":"2023-11-28T13:02:31.594513+0000","last_active":"2023-11-28T13:02:33.390455+0000","last_peered":"2023-11-28T13:02:33.390455+0000","last_clean":"2023-11-28T13:02:33.390455+0000","last_became_active":"2023-11-28T13:02:31.594253+0000","last_became_peered":"2023-11-28T13:02:31.594253+0000","last_unstale":"2023-11-28T13:02:33.390455+0000","last_undegraded":"2023-11-28T13:02:33.390455+0000","last_fullsized":"2023-11-28T13:02:33.390455+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:57:17.900013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.642231+0000","last_change":"2023-11-28T13:02:31.592287+0000","last_active":"2023-11-28T13:02:32.642231+0000","last_peered":"2023-11-28T13:02:32.642231+0000","last_clean":"2023-11-28T13:02:32.642231+0000","last_became_active":"2023-11-28T13:02:31.591985+0000","last_became_peered":"2023-11-28T13:02:31.591985+0000","last_unstale":"2023-11-28T13:02:32.642231+0000","last_undegraded":"2023-11-28T13:02:32.642231+0000","last_fullsized":"2023-11-28T13:02:32.642231+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:53.469232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570672+0000","last_change":"2023-11-28T13:02:31.580992+0000","last_active":"2023-11-28T13:02:32.570672+0000","last_peered":"2023-11-28T13:02:32.570672+0000","last_clean":"2023-11-28T13:02:32.570672+0000","last_became_active":"2023-11-28T13:02:31.580780+0000","last_became_peered":"2023-11-28T13:02:31.580780+0000","last_unstale":"2023-11-28T13:02:32.570672+0000","last_undegraded":"2023-11-28T13:02:32.570672+0000","last_fullsized":"2023-11-28T13:02:32.570672+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:59.400367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570557+0000","last_change":"2023-11-28T13:02:31.593244+0000","last_active":"2023-11-28T13:02:32.570557+0000","last_peered":"2023-11-28T13:02:32.570557+0000","last_clean":"2023-11-28T13:02:32.570557+0000","last_became_active":"2023-11-28T13:02:31.592982+0000","last_became_peered":"2023-11-28T13:02:31.592982+0000","last_unstale":"2023-11-28T13:02:32.570557+0000","last_undegraded":"2023-11-28T13:02:32.570557+0000","last_fullsized":"2023-11-28T13:02:32.570557+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:21.600980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572257+0000","last_change":"2023-11-28T13:02:31.592028+0000","last_active":"2023-11-28T13:02:32.572257+0000","last_peered":"2023-11-28T13:02:32.572257+0000","last_clean":"2023-11-28T13:02:32.572257+0000","last_became_active":"2023-11-28T13:02:31.591490+0000","last_became_peered":"2023-11-28T13:02:31.591490+0000","last_unstale":"2023-11-28T13:02:32.572257+0000","last_undegraded":"2023-11-28T13:02:32.572257+0000","last_fullsized":"2023-11-28T13:02:32.572257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:02:25.266256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856393+0000","last_change":"2023-11-28T13:02:31.588171+0000","last_active":"2023-11-28T13:02:32.856393+0000","last_peered":"2023-11-28T13:02:32.856393+0000","last_clean":"2023-11-28T13:02:32.856393+0000","last_became_active":"2023-11-28T13:02:31.587869+0000","last_became_peered":"2023-11-28T13:02:31.587869+0000","last_unstale":"2023-11-28T13:02:32.856393+0000","last_undegraded":"2023-11-28T13:02:32.856393+0000","last_fullsized":"2023-11-28T13:02:32.856393+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:08.799483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641872+0000","last_change":"2023-11-28T13:02:31.584572+0000","last_active":"2023-11-28T13:02:32.641872+0000","last_peered":"2023-11-28T13:02:32.641872+0000","last_clean":"2023-11-28T13:02:32.641872+0000","last_became_active":"2023-11-28T13:02:31.584002+0000","last_became_peered":"2023-11-28T13:02:31.584002+0000","last_unstale":"2023-11-28T13:02:32.641872+0000","last_undegraded":"2023-11-28T13:02:32.641872+0000","last_fullsized":"2023-11-28T13:02:32.641872+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:55.094506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641772+0000","last_change":"2023-11-28T13:02:31.570902+0000","last_active":"2023-11-28T13:02:32.641772+0000","last_peered":"2023-11-28T13:02:32.641772+0000","last_clean":"2023-11-28T13:02:32.641772+0000","last_became_active":"2023-11-28T13:02:31.570577+0000","last_became_peered":"2023-11-28T13:02:31.570577+0000","last_unstale":"2023-11-28T13:02:32.641772+0000","last_undegraded":"2023-11-28T13:02:32.641772+0000","last_fullsized":"2023-11-28T13:02:32.641772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:15.215786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391593+0000","last_change":"2023-11-28T13:02:31.586105+0000","last_active":"2023-11-28T13:02:33.391593+0000","last_peered":"2023-11-28T13:02:33.391593+0000","last_clean":"2023-11-28T13:02:33.391593+0000","last_became_active":"2023-11-28T13:02:31.585808+0000","last_became_peered":"2023-11-28T13:02:31.585808+0000","last_unstale":"2023-11-28T13:02:33.391593+0000","last_undegraded":"2023-11-28T13:02:33.391593+0000","last_fullsized":"2023-11-28T13:02:33.391593+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:56.872669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390985+0000","last_change":"2023-11-28T13:02:31.588041+0000","last_active":"2023-11-28T13:02:33.390985+0000","last_peered":"2023-11-28T13:02:33.390985+0000","last_clean":"2023-11-28T13:02:33.390985+0000","last_became_active":"2023-11-28T13:02:31.587782+0000","last_became_peered":"2023-11-28T13:02:31.587782+0000","last_unstale":"2023-11-28T13:02:33.390985+0000","last_undegraded":"2023-11-28T13:02:33.390985+0000","last_fullsized":"2023-11-28T13:02:33.390985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:13.019187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571668+0000","last_change":"2023-11-28T13:02:31.587891+0000","last_active":"2023-11-28T13:02:32.571668+0000","last_peered":"2023-11-28T13:02:32.571668+0000","last_clean":"2023-11-28T13:02:32.571668+0000","last_became_active":"2023-11-28T13:02:31.587355+0000","last_became_peered":"2023-11-28T13:02:31.587355+0000","last_unstale":"2023-11-28T13:02:32.571668+0000","last_undegraded":"2023-11-28T13:02:32.571668+0000","last_fullsized":"2023-11-28T13:02:32.571668+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:41:40.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391824+0000","last_change":"2023-11-28T13:02:31.591666+0000","last_active":"2023-11-28T13:02:33.391824+0000","last_peered":"2023-11-28T13:02:33.391824+0000","last_clean":"2023-11-28T13:02:33.391824+0000","last_became_active":"2023-11-28T13:02:31.591350+0000","last_became_peered":"2023-11-28T13:02:31.591350+0000","last_unstale":"2023-11-28T13:02:33.391824+0000","last_undegraded":"2023-11-28T13:02:33.391824+0000","last_fullsized":"2023-11-28T13:02:33.391824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:20:11.378168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390765+0000","last_change":"2023-11-28T13:02:31.597001+0000","last_active":"2023-11-28T13:02:33.390765+0000","last_peered":"2023-11-28T13:02:33.390765+0000","last_clean":"2023-11-28T13:02:33.390765+0000","last_became_active":"2023-11-28T13:02:31.596636+0000","last_became_peered":"2023-11-28T13:02:31.596636+0000","last_unstale":"2023-11-28T13:02:33.390765+0000","last_undegraded":"2023-11-28T13:02:33.390765+0000","last_fullsized":"2023-11-28T13:02:33.390765+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:16.872040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572069+0000","last_change":"2023-11-28T13:02:31.588505+0000","last_active":"2023-11-28T13:02:32.572069+0000","last_peered":"2023-11-28T13:02:32.572069+0000","last_clean":"2023-11-28T13:02:32.572069+0000","last_became_active":"2023-11-28T13:02:31.588200+0000","last_became_peered":"2023-11-28T13:02:31.588200+0000","last_unstale":"2023-11-28T13:02:32.572069+0000","last_undegraded":"2023-11-28T13:02:32.572069+0000","last_fullsized":"2023-11-28T13:02:32.572069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:43:21.360546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391007+0000","last_change":"2023-11-28T13:02:31.588603+0000","last_active":"2023-11-28T13:02:33.391007+0000","last_peered":"2023-11-28T13:02:33.391007+0000","last_clean":"2023-11-28T13:02:33.391007+0000","last_became_active":"2023-11-28T13:02:31.588355+0000","last_became_peered":"2023-11-28T13:02:31.588355+0000","last_unstale":"2023-11-28T13:02:33.391007+0000","last_undegraded":"2023-11-28T13:02:33.391007+0000","last_fullsized":"2023-11-28T13:02:33.391007+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:17.335526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571170+0000","last_change":"2023-11-28T13:02:31.591969+0000","last_active":"2023-11-28T13:02:32.571170+0000","last_peered":"2023-11-28T13:02:32.571170+0000","last_clean":"2023-11-28T13:02:32.571170+0000","last_became_active":"2023-11-28T13:02:31.591563+0000","last_became_peered":"2023-11-28T13:02:31.591563+0000","last_unstale":"2023-11-28T13:02:32.571170+0000","last_undegraded":"2023-11-28T13:02:32.571170+0000","last_fullsized":"2023-11-28T13:02:32.571170+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:13.477548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856246+0000","last_change":"2023-11-28T13:02:31.596962+0000","last_active":"2023-11-28T13:02:32.856246+0000","last_peered":"2023-11-28T13:02:32.856246+0000","last_clean":"2023-11-28T13:02:32.856246+0000","last_became_active":"2023-11-28T13:02:31.596681+0000","last_became_peered":"2023-11-28T13:02:31.596681+0000","last_unstale":"2023-11-28T13:02:32.856246+0000","last_undegraded":"2023-11-28T13:02:32.856246+0000","last_fullsized":"2023-11-28T13:02:32.856246+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:29:52.301201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392290+0000","last_change":"2023-11-28T13:02:31.579462+0000","last_active":"2023-11-28T13:02:33.392290+0000","last_peered":"2023-11-28T13:02:33.392290+0000","last_clean":"2023-11-28T13:02:33.392290+0000","last_became_active":"2023-11-28T13:02:31.579189+0000","last_became_peered":"2023-11-28T13:02:31.579189+0000","last_unstale":"2023-11-28T13:02:33.392290+0000","last_undegraded":"2023-11-28T13:02:33.392290+0000","last_fullsized":"2023-11-28T13:02:33.392290+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:14.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571424+0000","last_change":"2023-11-28T13:02:31.586885+0000","last_active":"2023-11-28T13:02:32.571424+0000","last_peered":"2023-11-28T13:02:32.571424+0000","last_clean":"2023-11-28T13:02:32.571424+0000","last_became_active":"2023-11-28T13:02:31.586485+0000","last_became_peered":"2023-11-28T13:02:31.586485+0000","last_unstale":"2023-11-28T13:02:32.571424+0000","last_undegraded":"2023-11-28T13:02:32.571424+0000","last_fullsized":"2023-11-28T13:02:32.571424+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:55.094506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392211+0000","last_change":"2023-11-28T13:02:31.597906+0000","last_active":"2023-11-28T13:02:33.392211+0000","last_peered":"2023-11-28T13:02:33.392211+0000","last_clean":"2023-11-28T13:02:33.392211+0000","last_became_active":"2023-11-28T13:02:31.597087+0000","last_became_peered":"2023-11-28T13:02:31.597087+0000","last_unstale":"2023-11-28T13:02:33.392211+0000","last_undegraded":"2023-11-28T13:02:33.392211+0000","last_fullsized":"2023-11-28T13:02:33.392211+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:57.092839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392511+0000","last_change":"2023-11-28T13:02:31.586899+0000","last_active":"2023-11-28T13:02:33.392511+0000","last_peered":"2023-11-28T13:02:33.392511+0000","last_clean":"2023-11-28T13:02:33.392511+0000","last_became_active":"2023-11-28T13:02:31.586608+0000","last_became_peered":"2023-11-28T13:02:31.586608+0000","last_unstale":"2023-11-28T13:02:33.392511+0000","last_undegraded":"2023-11-28T13:02:33.392511+0000","last_fullsized":"2023-11-28T13:02:33.392511+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:04.210874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390596+0000","last_change":"2023-11-28T13:02:31.592049+0000","last_active":"2023-11-28T13:02:33.390596+0000","last_peered":"2023-11-28T13:02:33.390596+0000","last_clean":"2023-11-28T13:02:33.390596+0000","last_became_active":"2023-11-28T13:02:31.591751+0000","last_became_peered":"2023-11-28T13:02:31.591751+0000","last_unstale":"2023-11-28T13:02:33.390596+0000","last_undegraded":"2023-11-28T13:02:33.390596+0000","last_fullsized":"2023-11-28T13:02:33.390596+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:24.133607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571342+0000","last_change":"2023-11-28T13:02:31.580080+0000","last_active":"2023-11-28T13:02:32.571342+0000","last_peered":"2023-11-28T13:02:32.571342+0000","last_clean":"2023-11-28T13:02:32.571342+0000","last_became_active":"2023-11-28T13:02:31.579755+0000","last_became_peered":"2023-11-28T13:02:31.579755+0000","last_unstale":"2023-11-28T13:02:32.571342+0000","last_undegraded":"2023-11-28T13:02:32.571342+0000","last_fullsized":"2023-11-28T13:02:32.571342+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:14:48.513955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392013+0000","last_change":"2023-11-28T13:02:31.577331+0000","last_active":"2023-11-28T13:02:33.392013+0000","last_peered":"2023-11-28T13:02:33.392013+0000","last_clean":"2023-11-28T13:02:33.392013+0000","last_became_active":"2023-11-28T13:02:31.577070+0000","last_became_peered":"2023-11-28T13:02:31.577070+0000","last_unstale":"2023-11-28T13:02:33.392013+0000","last_undegraded":"2023-11-28T13:02:33.392013+0000","last_fullsized":"2023-11-28T13:02:33.392013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:46.233325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391624+0000","last_change":"2023-11-28T13:02:31.589485+0000","last_active":"2023-11-28T13:02:33.391624+0000","last_peered":"2023-11-28T13:02:33.391624+0000","last_clean":"2023-11-28T13:02:33.391624+0000","last_became_active":"2023-11-28T13:02:31.589233+0000","last_became_peered":"2023-11-28T13:02:31.589233+0000","last_unstale":"2023-11-28T13:02:33.391624+0000","last_undegraded":"2023-11-28T13:02:33.391624+0000","last_fullsized":"2023-11-28T13:02:33.391624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:30.759324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392252+0000","last_change":"2023-11-28T13:02:31.589787+0000","last_active":"2023-11-28T13:02:33.392252+0000","last_peered":"2023-11-28T13:02:33.392252+0000","last_clean":"2023-11-28T13:02:33.392252+0000","last_became_active":"2023-11-28T13:02:31.586183+0000","last_became_peered":"2023-11-28T13:02:31.586183+0000","last_unstale":"2023-11-28T13:02:33.392252+0000","last_undegraded":"2023-11-28T13:02:33.392252+0000","last_fullsized":"2023-11-28T13:02:33.392252+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:02:07.071203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392392+0000","last_change":"2023-11-28T13:02:31.168749+0000","last_active":"2023-11-28T13:02:33.392392+0000","last_peered":"2023-11-28T13:02:33.392392+0000","last_clean":"2023-11-28T13:02:33.392392+0000","last_became_active":"2023-11-28T13:02:31.167729+0000","last_became_peered":"2023-11-28T13:02:31.167729+0000","last_unstale":"2023-11-28T13:02:33.392392+0000","last_undegraded":"2023-11-28T13:02:33.392392+0000","last_fullsized":"2023-11-28T13:02:33.392392+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:50:00.079781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570753+0000","last_change":"2023-11-28T13:02:31.583640+0000","last_active":"2023-11-28T13:02:32.570753+0000","last_peered":"2023-11-28T13:02:32.570753+0000","last_clean":"2023-11-28T13:02:32.570753+0000","last_became_active":"2023-11-28T13:02:31.583195+0000","last_became_peered":"2023-11-28T13:02:31.583195+0000","last_unstale":"2023-11-28T13:02:32.570753+0000","last_undegraded":"2023-11-28T13:02:32.570753+0000","last_fullsized":"2023-11-28T13:02:32.570753+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:31.030504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390886+0000","last_change":"2023-11-28T13:02:31.160037+0000","last_active":"2023-11-28T13:02:33.390886+0000","last_peered":"2023-11-28T13:02:33.390886+0000","last_clean":"2023-11-28T13:02:33.390886+0000","last_became_active":"2023-11-28T13:02:31.159581+0000","last_became_peered":"2023-11-28T13:02:31.159581+0000","last_unstale":"2023-11-28T13:02:33.390886+0000","last_undegraded":"2023-11-28T13:02:33.390886+0000","last_fullsized":"2023-11-28T13:02:33.390886+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:14.909317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391996+0000","last_change":"2023-11-28T13:02:31.595649+0000","last_active":"2023-11-28T13:02:33.391996+0000","last_peered":"2023-11-28T13:02:33.391996+0000","last_clean":"2023-11-28T13:02:33.391996+0000","last_became_active":"2023-11-28T13:02:31.586303+0000","last_became_peered":"2023-11-28T13:02:31.586303+0000","last_unstale":"2023-11-28T13:02:33.391996+0000","last_undegraded":"2023-11-28T13:02:33.391996+0000","last_fullsized":"2023-11-28T13:02:33.391996+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:29:23.609219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392568+0000","last_change":"2023-11-28T13:02:31.163301+0000","last_active":"2023-11-28T13:02:33.392568+0000","last_peered":"2023-11-28T13:02:33.392568+0000","last_clean":"2023-11-28T13:02:33.392568+0000","last_became_active":"2023-11-28T13:02:31.163004+0000","last_became_peered":"2023-11-28T13:02:31.163004+0000","last_unstale":"2023-11-28T13:02:33.392568+0000","last_undegraded":"2023-11-28T13:02:33.392568+0000","last_fullsized":"2023-11-28T13:02:33.392568+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:09.784960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392330+0000","last_change":"2023-11-28T13:02:31.598141+0000","last_active":"2023-11-28T13:02:33.392330+0000","last_peered":"2023-11-28T13:02:33.392330+0000","last_clean":"2023-11-28T13:02:33.392330+0000","last_became_active":"2023-11-28T13:02:31.596640+0000","last_became_peered":"2023-11-28T13:02:31.596640+0000","last_unstale":"2023-11-28T13:02:33.392330+0000","last_undegraded":"2023-11-28T13:02:33.392330+0000","last_fullsized":"2023-11-28T13:02:33.392330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:16.872040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572133+0000","last_change":"2023-11-28T13:02:31.158200+0000","last_active":"2023-11-28T13:02:32.572133+0000","last_peered":"2023-11-28T13:02:32.572133+0000","last_clean":"2023-11-28T13:02:32.572133+0000","last_became_active":"2023-11-28T13:02:31.157926+0000","last_became_peered":"2023-11-28T13:02:31.157926+0000","last_unstale":"2023-11-28T13:02:32.572133+0000","last_undegraded":"2023-11-28T13:02:32.572133+0000","last_fullsized":"2023-11-28T13:02:32.572133+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:59.239818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390979+0000","last_change":"2023-11-28T13:02:31.592098+0000","last_active":"2023-11-28T13:02:33.390979+0000","last_peered":"2023-11-28T13:02:33.390979+0000","last_clean":"2023-11-28T13:02:33.390979+0000","last_became_active":"2023-11-28T13:02:31.591833+0000","last_became_peered":"2023-11-28T13:02:31.591833+0000","last_unstale":"2023-11-28T13:02:33.390979+0000","last_undegraded":"2023-11-28T13:02:33.390979+0000","last_fullsized":"2023-11-28T13:02:33.390979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:56.872669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641921+0000","last_change":"2023-11-28T13:02:31.153946+0000","last_active":"2023-11-28T13:02:32.641921+0000","last_peered":"2023-11-28T13:02:32.641921+0000","last_clean":"2023-11-28T13:02:32.641921+0000","last_became_active":"2023-11-28T13:02:31.153698+0000","last_became_peered":"2023-11-28T13:02:31.153698+0000","last_unstale":"2023-11-28T13:02:32.641921+0000","last_undegraded":"2023-11-28T13:02:32.641921+0000","last_fullsized":"2023-11-28T13:02:32.641921+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:43:20.358576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390793+0000","last_change":"2023-11-28T13:02:31.597441+0000","last_active":"2023-11-28T13:02:33.390793+0000","last_peered":"2023-11-28T13:02:33.390793+0000","last_clean":"2023-11-28T13:02:33.390793+0000","last_became_active":"2023-11-28T13:02:31.597167+0000","last_became_peered":"2023-11-28T13:02:31.597167+0000","last_unstale":"2023-11-28T13:02:33.390793+0000","last_undegraded":"2023-11-28T13:02:33.390793+0000","last_fullsized":"2023-11-28T13:02:33.390793+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:29:23.609219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392033+0000","last_change":"2023-11-28T13:02:31.163261+0000","last_active":"2023-11-28T13:02:33.392033+0000","last_peered":"2023-11-28T13:02:33.392033+0000","last_clean":"2023-11-28T13:02:33.392033+0000","last_became_active":"2023-11-28T13:02:31.162980+0000","last_became_peered":"2023-11-28T13:02:31.162980+0000","last_unstale":"2023-11-28T13:02:33.392033+0000","last_undegraded":"2023-11-28T13:02:33.392033+0000","last_fullsized":"2023-11-28T13:02:33.392033+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:33.575223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856229+0000","last_change":"2023-11-28T13:02:31.591930+0000","last_active":"2023-11-28T13:02:32.856229+0000","last_peered":"2023-11-28T13:02:32.856229+0000","last_clean":"2023-11-28T13:02:32.856229+0000","last_became_active":"2023-11-28T13:02:31.591658+0000","last_became_peered":"2023-11-28T13:02:31.591658+0000","last_unstale":"2023-11-28T13:02:32.856229+0000","last_undegraded":"2023-11-28T13:02:32.856229+0000","last_fullsized":"2023-11-28T13:02:32.856229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:56.096171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571999+0000","last_change":"2023-11-28T13:02:31.154256+0000","last_active":"2023-11-28T13:02:32.571999+0000","last_peered":"2023-11-28T13:02:32.571999+0000","last_clean":"2023-11-28T13:02:32.571999+0000","last_became_active":"2023-11-28T13:02:31.153880+0000","last_became_peered":"2023-11-28T13:02:31.153880+0000","last_unstale":"2023-11-28T13:02:32.571999+0000","last_undegraded":"2023-11-28T13:02:32.571999+0000","last_fullsized":"2023-11-28T13:02:32.571999+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:19.194642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390458+0000","last_change":"2023-11-28T13:02:31.589235+0000","last_active":"2023-11-28T13:02:33.390458+0000","last_peered":"2023-11-28T13:02:33.390458+0000","last_clean":"2023-11-28T13:02:33.390458+0000","last_became_active":"2023-11-28T13:02:31.588946+0000","last_became_peered":"2023-11-28T13:02:31.588946+0000","last_unstale":"2023-11-28T13:02:33.390458+0000","last_undegraded":"2023-11-28T13:02:33.390458+0000","last_fullsized":"2023-11-28T13:02:33.390458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:30.759324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641862+0000","last_change":"2023-11-28T13:02:31.161186+0000","last_active":"2023-11-28T13:02:32.641862+0000","last_peered":"2023-11-28T13:02:32.641862+0000","last_clean":"2023-11-28T13:02:32.641862+0000","last_became_active":"2023-11-28T13:02:31.160828+0000","last_became_peered":"2023-11-28T13:02:31.160828+0000","last_unstale":"2023-11-28T13:02:32.641862+0000","last_undegraded":"2023-11-28T13:02:32.641862+0000","last_fullsized":"2023-11-28T13:02:32.641862+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:59.848012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.642235+0000","last_change":"2023-11-28T13:02:31.573929+0000","last_active":"2023-11-28T13:02:32.642235+0000","last_peered":"2023-11-28T13:02:32.642235+0000","last_clean":"2023-11-28T13:02:32.642235+0000","last_became_active":"2023-11-28T13:02:31.573604+0000","last_became_peered":"2023-11-28T13:02:31.573604+0000","last_unstale":"2023-11-28T13:02:32.642235+0000","last_undegraded":"2023-11-28T13:02:32.642235+0000","last_fullsized":"2023-11-28T13:02:32.642235+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:07.955246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571501+0000","last_change":"2023-11-28T13:02:31.168769+0000","last_active":"2023-11-28T13:02:32.571501+0000","last_peered":"2023-11-28T13:02:32.571501+0000","last_clean":"2023-11-28T13:02:32.571501+0000","last_became_active":"2023-11-28T13:02:31.168462+0000","last_became_peered":"2023-11-28T13:02:31.168462+0000","last_unstale":"2023-11-28T13:02:32.571501+0000","last_undegraded":"2023-11-28T13:02:32.571501+0000","last_fullsized":"2023-11-28T13:02:32.571501+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:50.251928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571428+0000","last_change":"2023-11-28T13:02:31.581348+0000","last_active":"2023-11-28T13:02:32.571428+0000","last_peered":"2023-11-28T13:02:32.571428+0000","last_clean":"2023-11-28T13:02:32.571428+0000","last_became_active":"2023-11-28T13:02:31.581082+0000","last_became_peered":"2023-11-28T13:02:31.581082+0000","last_unstale":"2023-11-28T13:02:32.571428+0000","last_undegraded":"2023-11-28T13:02:32.571428+0000","last_fullsized":"2023-11-28T13:02:32.571428+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:07:22.317413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390996+0000","last_change":"2023-11-28T13:02:31.159847+0000","last_active":"2023-11-28T13:02:33.390996+0000","last_peered":"2023-11-28T13:02:33.390996+0000","last_clean":"2023-11-28T13:02:33.390996+0000","last_became_active":"2023-11-28T13:02:31.159227+0000","last_became_peered":"2023-11-28T13:02:31.159227+0000","last_unstale":"2023-11-28T13:02:33.390996+0000","last_undegraded":"2023-11-28T13:02:33.390996+0000","last_fullsized":"2023-11-28T13:02:33.390996+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:46.056976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570472+0000","last_change":"2023-11-28T13:02:31.570890+0000","last_active":"2023-11-28T13:02:32.570472+0000","last_peered":"2023-11-28T13:02:32.570472+0000","last_clean":"2023-11-28T13:02:32.570472+0000","last_became_active":"2023-11-28T13:02:31.570573+0000","last_became_peered":"2023-11-28T13:02:31.570573+0000","last_unstale":"2023-11-28T13:02:32.570472+0000","last_undegraded":"2023-11-28T13:02:32.570472+0000","last_fullsized":"2023-11-28T13:02:32.570472+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:39:38.704425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571870+0000","last_change":"2023-11-28T13:02:31.157987+0000","last_active":"2023-11-28T13:02:32.571870+0000","last_peered":"2023-11-28T13:02:32.571870+0000","last_clean":"2023-11-28T13:02:32.571870+0000","last_became_active":"2023-11-28T13:02:31.157605+0000","last_became_peered":"2023-11-28T13:02:31.157605+0000","last_unstale":"2023-11-28T13:02:32.571870+0000","last_undegraded":"2023-11-28T13:02:32.571870+0000","last_fullsized":"2023-11-28T13:02:32.571870+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:08.041508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572042+0000","last_change":"2023-11-28T13:02:31.581849+0000","last_active":"2023-11-28T13:02:32.572042+0000","last_peered":"2023-11-28T13:02:32.572042+0000","last_clean":"2023-11-28T13:02:32.572042+0000","last_became_active":"2023-11-28T13:02:31.581378+0000","last_became_peered":"2023-11-28T13:02:31.581378+0000","last_unstale":"2023-11-28T13:02:32.572042+0000","last_undegraded":"2023-11-28T13:02:32.572042+0000","last_fullsized":"2023-11-28T13:02:32.572042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:00.849982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.642111+0000","last_change":"2023-11-28T13:02:31.168730+0000","last_active":"2023-11-28T13:02:32.642111+0000","last_peered":"2023-11-28T13:02:32.642111+0000","last_clean":"2023-11-28T13:02:32.642111+0000","last_became_active":"2023-11-28T13:02:31.168366+0000","last_became_peered":"2023-11-28T13:02:31.168366+0000","last_unstale":"2023-11-28T13:02:32.642111+0000","last_undegraded":"2023-11-28T13:02:32.642111+0000","last_fullsized":"2023-11-28T13:02:32.642111+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:07:21.315443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572491+0000","last_change":"2023-11-28T13:02:31.586586+0000","last_active":"2023-11-28T13:02:32.572491+0000","last_peered":"2023-11-28T13:02:32.572491+0000","last_clean":"2023-11-28T13:02:32.572491+0000","last_became_active":"2023-11-28T13:02:31.586073+0000","last_became_peered":"2023-11-28T13:02:31.586073+0000","last_unstale":"2023-11-28T13:02:32.572491+0000","last_undegraded":"2023-11-28T13:02:32.572491+0000","last_fullsized":"2023-11-28T13:02:32.572491+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:07.955246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391032+0000","last_change":"2023-11-28T13:02:31.160978+0000","last_active":"2023-11-28T13:02:33.391032+0000","last_peered":"2023-11-28T13:02:33.391032+0000","last_clean":"2023-11-28T13:02:33.391032+0000","last_became_active":"2023-11-28T13:02:31.160401+0000","last_became_peered":"2023-11-28T13:02:31.160401+0000","last_unstale":"2023-11-28T13:02:33.391032+0000","last_undegraded":"2023-11-28T13:02:33.391032+0000","last_fullsized":"2023-11-28T13:02:33.391032+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:17.271316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571510+0000","last_change":"2023-11-28T13:02:31.587922+0000","last_active":"2023-11-28T13:02:32.571510+0000","last_peered":"2023-11-28T13:02:32.571510+0000","last_clean":"2023-11-28T13:02:32.571510+0000","last_became_active":"2023-11-28T13:02:31.587621+0000","last_became_peered":"2023-11-28T13:02:31.587621+0000","last_unstale":"2023-11-28T13:02:32.571510+0000","last_undegraded":"2023-11-28T13:02:32.571510+0000","last_fullsized":"2023-11-28T13:02:32.571510+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:23.864039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856254+0000","last_change":"2023-11-28T13:02:31.158209+0000","last_active":"2023-11-28T13:02:32.856254+0000","last_peered":"2023-11-28T13:02:32.856254+0000","last_clean":"2023-11-28T13:02:32.856254+0000","last_became_active":"2023-11-28T13:02:31.157931+0000","last_became_peered":"2023-11-28T13:02:31.157931+0000","last_unstale":"2023-11-28T13:02:32.856254+0000","last_undegraded":"2023-11-28T13:02:32.856254+0000","last_fullsized":"2023-11-28T13:02:32.856254+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:42.997557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641740+0000","last_change":"2023-11-28T13:02:31.584276+0000","last_active":"2023-11-28T13:02:32.641740+0000","last_peered":"2023-11-28T13:02:32.641740+0000","last_clean":"2023-11-28T13:02:32.641740+0000","last_became_active":"2023-11-28T13:02:31.583989+0000","last_became_peered":"2023-11-28T13:02:31.583989+0000","last_unstale":"2023-11-28T13:02:32.641740+0000","last_undegraded":"2023-11-28T13:02:32.641740+0000","last_fullsized":"2023-11-28T13:02:32.641740+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:00.075913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571740+0000","last_change":"2023-11-28T13:02:31.156773+0000","last_active":"2023-11-28T13:02:32.571740+0000","last_peered":"2023-11-28T13:02:32.571740+0000","last_clean":"2023-11-28T13:02:32.571740+0000","last_became_active":"2023-11-28T13:02:31.156454+0000","last_became_peered":"2023-11-28T13:02:31.156454+0000","last_unstale":"2023-11-28T13:02:32.571740+0000","last_undegraded":"2023-11-28T13:02:32.571740+0000","last_fullsized":"2023-11-28T13:02:32.571740+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:26.161562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571284+0000","last_change":"2023-11-28T13:02:31.585050+0000","last_active":"2023-11-28T13:02:32.571284+0000","last_peered":"2023-11-28T13:02:32.571284+0000","last_clean":"2023-11-28T13:02:32.571284+0000","last_became_active":"2023-11-28T13:02:31.584713+0000","last_became_peered":"2023-11-28T13:02:31.584713+0000","last_unstale":"2023-11-28T13:02:32.571284+0000","last_undegraded":"2023-11-28T13:02:32.571284+0000","last_fullsized":"2023-11-28T13:02:32.571284+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:15.802346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":19,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570971+0000","last_change":"2023-11-28T13:02:31.157623+0000","last_active":"2023-11-28T13:02:32.570971+0000","last_peered":"2023-11-28T13:02:32.570971+0000","last_clean":"2023-11-28T13:02:32.570971+0000","last_became_active":"2023-11-28T13:02:31.157281+0000","last_became_peered":"2023-11-28T13:02:31.157281+0000","last_unstale":"2023-11-28T13:02:32.570971+0000","last_undegraded":"2023-11-28T13:02:32.570971+0000","last_fullsized":"2023-11-28T13:02:32.570971+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:12.819597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571540+0000","last_change":"2023-11-28T13:02:31.587125+0000","last_active":"2023-11-28T13:02:32.571540+0000","last_peered":"2023-11-28T13:02:32.571540+0000","last_clean":"2023-11-28T13:02:32.571540+0000","last_became_active":"2023-11-28T13:02:31.586818+0000","last_became_peered":"2023-11-28T13:02:31.586818+0000","last_unstale":"2023-11-28T13:02:32.571540+0000","last_undegraded":"2023-11-28T13:02:32.571540+0000","last_fullsized":"2023-11-28T13:02:32.571540+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:13.477548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570812+0000","last_change":"2023-11-28T13:02:31.163041+0000","last_active":"2023-11-28T13:02:32.570812+0000","last_peered":"2023-11-28T13:02:32.570812+0000","last_clean":"2023-11-28T13:02:32.570812+0000","last_became_active":"2023-11-28T13:02:31.162740+0000","last_became_peered":"2023-11-28T13:02:31.162740+0000","last_unstale":"2023-11-28T13:02:32.570812+0000","last_undegraded":"2023-11-28T13:02:32.570812+0000","last_fullsized":"2023-11-28T13:02:32.570812+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:09:06.353454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571756+0000","last_change":"2023-11-28T13:02:31.583605+0000","last_active":"2023-11-28T13:02:32.571756+0000","last_peered":"2023-11-28T13:02:32.571756+0000","last_clean":"2023-11-28T13:02:32.571756+0000","last_became_active":"2023-11-28T13:02:31.583312+0000","last_became_peered":"2023-11-28T13:02:31.583312+0000","last_unstale":"2023-11-28T13:02:32.571756+0000","last_undegraded":"2023-11-28T13:02:32.571756+0000","last_fullsized":"2023-11-28T13:02:32.571756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:53.469232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392644+0000","last_change":"2023-11-28T13:02:31.168129+0000","last_active":"2023-11-28T13:02:33.392644+0000","last_peered":"2023-11-28T13:02:33.392644+0000","last_clean":"2023-11-28T13:02:33.392644+0000","last_became_active":"2023-11-28T13:02:31.167679+0000","last_became_peered":"2023-11-28T13:02:31.167679+0000","last_unstale":"2023-11-28T13:02:33.392644+0000","last_undegraded":"2023-11-28T13:02:33.392644+0000","last_fullsized":"2023-11-28T13:02:33.392644+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:02.510602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570511+0000","last_change":"2023-11-28T13:02:31.573089+0000","last_active":"2023-11-28T13:02:32.570511+0000","last_peered":"2023-11-28T13:02:32.570511+0000","last_clean":"2023-11-28T13:02:32.570511+0000","last_became_active":"2023-11-28T13:02:31.572854+0000","last_became_peered":"2023-11-28T13:02:31.572854+0000","last_unstale":"2023-11-28T13:02:32.570511+0000","last_undegraded":"2023-11-28T13:02:32.570511+0000","last_fullsized":"2023-11-28T13:02:32.570511+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:47.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856443+0000","last_change":"2023-11-28T13:02:31.155385+0000","last_active":"2023-11-28T13:02:32.856443+0000","last_peered":"2023-11-28T13:02:32.856443+0000","last_clean":"2023-11-28T13:02:32.856443+0000","last_became_active":"2023-11-28T13:02:31.155110+0000","last_became_peered":"2023-11-28T13:02:31.155110+0000","last_unstale":"2023-11-28T13:02:32.856443+0000","last_undegraded":"2023-11-28T13:02:32.856443+0000","last_fullsized":"2023-11-28T13:02:32.856443+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:31.087515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392471+0000","last_change":"2023-11-28T13:02:31.597730+0000","last_active":"2023-11-28T13:02:33.392471+0000","last_peered":"2023-11-28T13:02:33.392471+0000","last_clean":"2023-11-28T13:02:33.392471+0000","last_became_active":"2023-11-28T13:02:31.596819+0000","last_became_peered":"2023-11-28T13:02:31.596819+0000","last_unstale":"2023-11-28T13:02:33.392471+0000","last_undegraded":"2023-11-28T13:02:33.392471+0000","last_fullsized":"2023-11-28T13:02:33.392471+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:12:32.498368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572030+0000","last_change":"2023-11-28T13:02:31.153020+0000","last_active":"2023-11-28T13:02:32.572030+0000","last_peered":"2023-11-28T13:02:32.572030+0000","last_clean":"2023-11-28T13:02:32.572030+0000","last_became_active":"2023-11-28T13:02:31.152726+0000","last_became_peered":"2023-11-28T13:02:31.152726+0000","last_unstale":"2023-11-28T13:02:32.572030+0000","last_undegraded":"2023-11-28T13:02:32.572030+0000","last_fullsized":"2023-11-28T13:02:32.572030+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:46.981104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390779+0000","last_change":"2023-11-28T13:02:31.587261+0000","last_active":"2023-11-28T13:02:33.390779+0000","last_peered":"2023-11-28T13:02:33.390779+0000","last_clean":"2023-11-28T13:02:33.390779+0000","last_became_active":"2023-11-28T13:02:31.584178+0000","last_became_peered":"2023-11-28T13:02:31.584178+0000","last_unstale":"2023-11-28T13:02:33.390779+0000","last_undegraded":"2023-11-28T13:02:33.390779+0000","last_fullsized":"2023-11-28T13:02:33.390779+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:12:32.498368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570027+0000","last_change":"2023-11-28T13:02:31.156804+0000","last_active":"2023-11-28T13:02:32.570027+0000","last_peered":"2023-11-28T13:02:32.570027+0000","last_clean":"2023-11-28T13:02:32.570027+0000","last_became_active":"2023-11-28T13:02:31.156515+0000","last_became_peered":"2023-11-28T13:02:31.156515+0000","last_unstale":"2023-11-28T13:02:32.570027+0000","last_undegraded":"2023-11-28T13:02:32.570027+0000","last_fullsized":"2023-11-28T13:02:32.570027+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:56.259977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571626+0000","last_change":"2023-11-28T13:02:31.592265+0000","last_active":"2023-11-28T13:02:32.571626+0000","last_peered":"2023-11-28T13:02:32.571626+0000","last_clean":"2023-11-28T13:02:32.571626+0000","last_became_active":"2023-11-28T13:02:31.591625+0000","last_became_peered":"2023-11-28T13:02:31.591625+0000","last_unstale":"2023-11-28T13:02:32.571626+0000","last_undegraded":"2023-11-28T13:02:32.571626+0000","last_fullsized":"2023-11-28T13:02:32.571626+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:45.032643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641909+0000","last_change":"2023-11-28T13:02:31.169320+0000","last_active":"2023-11-28T13:02:32.641909+0000","last_peered":"2023-11-28T13:02:32.641909+0000","last_clean":"2023-11-28T13:02:32.641909+0000","last_became_active":"2023-11-28T13:02:31.168966+0000","last_became_peered":"2023-11-28T13:02:31.168966+0000","last_unstale":"2023-11-28T13:02:32.641909+0000","last_undegraded":"2023-11-28T13:02:32.641909+0000","last_fullsized":"2023-11-28T13:02:32.641909+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:56.157357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.569965+0000","last_change":"2023-11-28T13:02:31.582200+0000","last_active":"2023-11-28T13:02:32.569965+0000","last_peered":"2023-11-28T13:02:32.569965+0000","last_clean":"2023-11-28T13:02:32.569965+0000","last_became_active":"2023-11-28T13:02:31.581927+0000","last_became_peered":"2023-11-28T13:02:31.581927+0000","last_unstale":"2023-11-28T13:02:32.569965+0000","last_undegraded":"2023-11-28T13:02:32.569965+0000","last_fullsized":"2023-11-28T13:02:32.569965+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:58.821868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570129+0000","last_change":"2023-11-28T13:02:31.151164+0000","last_active":"2023-11-28T13:02:32.570129+0000","last_peered":"2023-11-28T13:02:32.570129+0000","last_clean":"2023-11-28T13:02:32.570129+0000","last_became_active":"2023-11-28T13:02:31.150880+0000","last_became_peered":"2023-11-28T13:02:31.150880+0000","last_unstale":"2023-11-28T13:02:32.570129+0000","last_undegraded":"2023-11-28T13:02:32.570129+0000","last_fullsized":"2023-11-28T13:02:32.570129+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:20:24.962468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390465+0000","last_change":"2023-11-28T13:02:31.588209+0000","last_active":"2023-11-28T13:02:33.390465+0000","last_peered":"2023-11-28T13:02:33.390465+0000","last_clean":"2023-11-28T13:02:33.390465+0000","last_became_active":"2023-11-28T13:02:31.587833+0000","last_became_peered":"2023-11-28T13:02:31.587833+0000","last_unstale":"2023-11-28T13:02:33.390465+0000","last_undegraded":"2023-11-28T13:02:33.390465+0000","last_fullsized":"2023-11-28T13:02:33.390465+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:02:07.071203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390575+0000","last_change":"2023-11-28T13:02:31.157931+0000","last_active":"2023-11-28T13:02:33.390575+0000","last_peered":"2023-11-28T13:02:33.390575+0000","last_clean":"2023-11-28T13:02:33.390575+0000","last_became_active":"2023-11-28T13:02:31.157478+0000","last_became_peered":"2023-11-28T13:02:31.157478+0000","last_unstale":"2023-11-28T13:02:33.390575+0000","last_undegraded":"2023-11-28T13:02:33.390575+0000","last_fullsized":"2023-11-28T13:02:33.390575+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:50:00.079781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571992+0000","last_change":"2023-11-28T13:02:31.581663+0000","last_active":"2023-11-28T13:02:32.571992+0000","last_peered":"2023-11-28T13:02:32.571992+0000","last_clean":"2023-11-28T13:02:32.571992+0000","last_became_active":"2023-11-28T13:02:31.581031+0000","last_became_peered":"2023-11-28T13:02:31.581031+0000","last_unstale":"2023-11-28T13:02:32.571992+0000","last_undegraded":"2023-11-28T13:02:32.571992+0000","last_fullsized":"2023-11-28T13:02:32.571992+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:53.469232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392437+0000","last_change":"2023-11-28T13:02:31.168319+0000","last_active":"2023-11-28T13:02:33.392437+0000","last_peered":"2023-11-28T13:02:33.392437+0000","last_clean":"2023-11-28T13:02:33.392437+0000","last_became_active":"2023-11-28T13:02:31.167709+0000","last_became_peered":"2023-11-28T13:02:31.167709+0000","last_unstale":"2023-11-28T13:02:33.392437+0000","last_undegraded":"2023-11-28T13:02:33.392437+0000","last_fullsized":"2023-11-28T13:02:33.392437+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:09.741137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.567424+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:02:31.567424+0000","last_peered":"2023-11-28T13:02:31.567424+0000","last_clean":"2023-11-28T13:02:31.567424+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:02:31.567424+0000","last_undegraded":"2023-11-28T13:02:31.567424+0000","last_fullsized":"2023-11-28T13:02:31.567424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390834+0000","last_change":"2023-11-28T13:02:31.591456+0000","last_active":"2023-11-28T13:02:33.390834+0000","last_peered":"2023-11-28T13:02:33.390834+0000","last_clean":"2023-11-28T13:02:33.390834+0000","last_became_active":"2023-11-28T13:02:31.584118+0000","last_became_peered":"2023-11-28T13:02:31.584118+0000","last_unstale":"2023-11-28T13:02:33.390834+0000","last_undegraded":"2023-11-28T13:02:33.390834+0000","last_fullsized":"2023-11-28T13:02:33.390834+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:04.890441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572533+0000","last_change":"2023-11-28T13:02:31.156754+0000","last_active":"2023-11-28T13:02:32.572533+0000","last_peered":"2023-11-28T13:02:32.572533+0000","last_clean":"2023-11-28T13:02:32.572533+0000","last_became_active":"2023-11-28T13:02:31.156415+0000","last_became_peered":"2023-11-28T13:02:31.156415+0000","last_unstale":"2023-11-28T13:02:32.572533+0000","last_undegraded":"2023-11-28T13:02:32.572533+0000","last_fullsized":"2023-11-28T13:02:32.572533+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:04.093914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":44,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.581970+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:02:31.581970+0000","last_peered":"2023-11-28T13:02:31.581970+0000","last_clean":"2023-11-28T13:02:31.581970+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:02:31.581970+0000","last_undegraded":"2023-11-28T13:02:31.581970+0000","last_fullsized":"2023-11-28T13:02:31.581970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392572+0000","last_change":"2023-11-28T13:02:31.590718+0000","last_active":"2023-11-28T13:02:33.392572+0000","last_peered":"2023-11-28T13:02:33.392572+0000","last_clean":"2023-11-28T13:02:33.392572+0000","last_became_active":"2023-11-28T13:02:31.590275+0000","last_became_peered":"2023-11-28T13:02:31.590275+0000","last_unstale":"2023-11-28T13:02:33.392572+0000","last_undegraded":"2023-11-28T13:02:33.392572+0000","last_fullsized":"2023-11-28T13:02:33.392572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:04.890441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570275+0000","last_change":"2023-11-28T13:02:31.153173+0000","last_active":"2023-11-28T13:02:32.570275+0000","last_peered":"2023-11-28T13:02:32.570275+0000","last_clean":"2023-11-28T13:02:32.570275+0000","last_became_active":"2023-11-28T13:02:31.152787+0000","last_became_peered":"2023-11-28T13:02:31.152787+0000","last_unstale":"2023-11-28T13:02:32.570275+0000","last_undegraded":"2023-11-28T13:02:32.570275+0000","last_fullsized":"2023-11-28T13:02:32.570275+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:39.835807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.567895+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:02:31.567895+0000","last_peered":"2023-11-28T13:02:31.567895+0000","last_clean":"2023-11-28T13:02:31.567895+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:02:31.567895+0000","last_undegraded":"2023-11-28T13:02:31.567895+0000","last_fullsized":"2023-11-28T13:02:31.567895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":198,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.572883+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:02:31.572883+0000","last_peered":"2023-11-28T13:02:31.572883+0000","last_clean":"2023-11-28T13:02:31.572883+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:02:31.572883+0000","last_undegraded":"2023-11-28T13:02:31.572883+0000","last_fullsized":"2023-11-28T13:02:31.572883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572573+0000","last_change":"2023-11-28T13:02:31.586394+0000","last_active":"2023-11-28T13:02:32.572573+0000","last_peered":"2023-11-28T13:02:32.572573+0000","last_clean":"2023-11-28T13:02:32.572573+0000","last_became_active":"2023-11-28T13:02:31.585743+0000","last_became_peered":"2023-11-28T13:02:31.585743+0000","last_unstale":"2023-11-28T13:02:32.572573+0000","last_undegraded":"2023-11-28T13:02:32.572573+0000","last_fullsized":"2023-11-28T13:02:32.572573+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:00.075913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391162+0000","last_change":"2023-11-28T13:02:31.160616+0000","last_active":"2023-11-28T13:02:33.391162+0000","last_peered":"2023-11-28T13:02:33.391162+0000","last_clean":"2023-11-28T13:02:33.391162+0000","last_became_active":"2023-11-28T13:02:31.159814+0000","last_became_peered":"2023-11-28T13:02:31.159814+0000","last_unstale":"2023-11-28T13:02:33.391162+0000","last_undegraded":"2023-11-28T13:02:33.391162+0000","last_fullsized":"2023-11-28T13:02:33.391162+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:43.400645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.572098+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:02:31.572098+0000","last_peered":"2023-11-28T13:02:31.572098+0000","last_clean":"2023-11-28T13:02:31.572098+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:02:31.572098+0000","last_undegraded":"2023-11-28T13:02:31.572098+0000","last_fullsized":"2023-11-28T13:02:31.572098+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390545+0000","last_change":"2023-11-28T13:02:31.596874+0000","last_active":"2023-11-28T13:02:33.390545+0000","last_peered":"2023-11-28T13:02:33.390545+0000","last_clean":"2023-11-28T13:02:33.390545+0000","last_became_active":"2023-11-28T13:02:31.596620+0000","last_became_peered":"2023-11-28T13:02:31.596620+0000","last_unstale":"2023-11-28T13:02:33.390545+0000","last_undegraded":"2023-11-28T13:02:33.390545+0000","last_fullsized":"2023-11-28T13:02:33.390545+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:57.092839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856253+0000","last_change":"2023-11-28T13:02:31.152084+0000","last_active":"2023-11-28T13:02:32.856253+0000","last_peered":"2023-11-28T13:02:32.856253+0000","last_clean":"2023-11-28T13:02:32.856253+0000","last_became_active":"2023-11-28T13:02:31.151769+0000","last_became_peered":"2023-11-28T13:02:31.151769+0000","last_unstale":"2023-11-28T13:02:32.856253+0000","last_undegraded":"2023-11-28T13:02:32.856253+0000","last_fullsized":"2023-11-28T13:02:32.856253+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:46.802595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.572778+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:02:31.572778+0000","last_peered":"2023-11-28T13:02:31.572778+0000","last_clean":"2023-11-28T13:02:31.572778+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:02:31.572778+0000","last_undegraded":"2023-11-28T13:02:31.572778+0000","last_fullsized":"2023-11-28T13:02:31.572778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391715+0000","last_change":"2023-11-28T13:02:31.586237+0000","last_active":"2023-11-28T13:02:33.391715+0000","last_peered":"2023-11-28T13:02:33.391715+0000","last_clean":"2023-11-28T13:02:33.391715+0000","last_became_active":"2023-11-28T13:02:31.585914+0000","last_became_peered":"2023-11-28T13:02:31.585914+0000","last_unstale":"2023-11-28T13:02:33.391715+0000","last_undegraded":"2023-11-28T13:02:33.391715+0000","last_fullsized":"2023-11-28T13:02:33.391715+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:40.289844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571405+0000","last_change":"2023-11-28T13:02:31.164931+0000","last_active":"2023-11-28T13:02:32.571405+0000","last_peered":"2023-11-28T13:02:32.571405+0000","last_clean":"2023-11-28T13:02:32.571405+0000","last_became_active":"2023-11-28T13:02:31.164606+0000","last_became_peered":"2023-11-28T13:02:31.164606+0000","last_unstale":"2023-11-28T13:02:32.571405+0000","last_undegraded":"2023-11-28T13:02:32.571405+0000","last_fullsized":"2023-11-28T13:02:32.571405+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:00:03.898632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.576376+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:02:31.576376+0000","last_peered":"2023-11-28T13:02:31.576376+0000","last_clean":"2023-11-28T13:02:31.576376+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:02:31.576376+0000","last_undegraded":"2023-11-28T13:02:31.576376+0000","last_fullsized":"2023-11-28T13:02:31.576376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571387+0000","last_change":"2023-11-28T13:02:31.585412+0000","last_active":"2023-11-28T13:02:32.571387+0000","last_peered":"2023-11-28T13:02:32.571387+0000","last_clean":"2023-11-28T13:02:32.571387+0000","last_became_active":"2023-11-28T13:02:31.585091+0000","last_became_peered":"2023-11-28T13:02:31.585091+0000","last_unstale":"2023-11-28T13:02:32.571387+0000","last_undegraded":"2023-11-28T13:02:32.571387+0000","last_fullsized":"2023-11-28T13:02:32.571387+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:09.625935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391133+0000","last_change":"2023-11-28T13:02:31.159531+0000","last_active":"2023-11-28T13:02:33.391133+0000","last_peered":"2023-11-28T13:02:33.391133+0000","last_clean":"2023-11-28T13:02:33.391133+0000","last_became_active":"2023-11-28T13:02:31.159249+0000","last_became_peered":"2023-11-28T13:02:31.159249+0000","last_unstale":"2023-11-28T13:02:33.391133+0000","last_undegraded":"2023-11-28T13:02:33.391133+0000","last_fullsized":"2023-11-28T13:02:33.391133+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:23.856955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.576373+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:02:31.576373+0000","last_peered":"2023-11-28T13:02:31.576373+0000","last_clean":"2023-11-28T13:02:31.576373+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:02:31.576373+0000","last_undegraded":"2023-11-28T13:02:31.576373+0000","last_fullsized":"2023-11-28T13:02:31.576373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571667+0000","last_change":"2023-11-28T13:02:31.586198+0000","last_active":"2023-11-28T13:02:32.571667+0000","last_peered":"2023-11-28T13:02:32.571667+0000","last_clean":"2023-11-28T13:02:32.571667+0000","last_became_active":"2023-11-28T13:02:31.585863+0000","last_became_peered":"2023-11-28T13:02:31.585863+0000","last_unstale":"2023-11-28T13:02:32.571667+0000","last_undegraded":"2023-11-28T13:02:32.571667+0000","last_fullsized":"2023-11-28T13:02:32.571667+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:15.215786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392321+0000","last_change":"2023-11-28T13:02:31.168641+0000","last_active":"2023-11-28T13:02:33.392321+0000","last_peered":"2023-11-28T13:02:33.392321+0000","last_clean":"2023-11-28T13:02:33.392321+0000","last_became_active":"2023-11-28T13:02:31.168363+0000","last_became_peered":"2023-11-28T13:02:31.168363+0000","last_unstale":"2023-11-28T13:02:33.392321+0000","last_undegraded":"2023-11-28T13:02:33.392321+0000","last_fullsized":"2023-11-28T13:02:33.392321+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:43.400645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.576014+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:02:31.576014+0000","last_peered":"2023-11-28T13:02:31.576014+0000","last_clean":"2023-11-28T13:02:31.576014+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:02:31.576014+0000","last_undegraded":"2023-11-28T13:02:31.576014+0000","last_fullsized":"2023-11-28T13:02:31.576014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10376,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361464,"statfs":{"total":96636764160,"available":96626139136,"internally_reserved":0,"allocated":1245184,"data_stored":889426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.45700000000000002,"15min":0.442},"min":{"1min":0.32200000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69899999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.495}]},{"osd":1,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.36499999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.432}]},{"osd":2,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.40799999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64800000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51800000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77100000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56799999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65200000000000002,"15min":0.65500000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63200000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.46500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.79000000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85299999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9148,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362692,"statfs":{"total":96636764160,"available":96627396608,"internally_reserved":0,"allocated":643072,"data_stored":298898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.33900000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.49399999999999999,"15min":0.48199999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58299999999999996}]},{"osd":1,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.629},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72799999999999998,"5min":0.87,"15min":0.87},"last":0.60399999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.71399999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.54100000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":1.2829999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.38900000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.74299999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.626},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.77100000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.755}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10240,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361600,"statfs":{"total":96636764160,"available":96626278400,"internally_reserved":0,"allocated":1236992,"data_stored":888351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.313,"15min":0.309},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56799999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.373}]},{"osd":2,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.307},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38200000000000001,"15min":0.38},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.29099999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32700000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.82899999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.71999999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.47299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":1.157,"15min":1.157},"last":0.51100000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86799999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.495},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9352,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362488,"statfs":{"total":96636764160,"available":96627187712,"internally_reserved":0,"allocated":655360,"data_stored":300026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.34499999999999997,"15min":0.34100000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.55300000000000005}]},{"osd":2,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.49299999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.33800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42499999999999999,"15min":0.42799999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.62}]},{"osd":4,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80100000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.90200000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91900000000000004,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.877}]},{"osd":7,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.75600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9156,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362684,"statfs":{"total":96636764160,"available":96627388416,"internally_reserved":0,"allocated":651264,"data_stored":298954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.438,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36599999999999999,"15min":0.36699999999999999},"min":{"1min":0.27700000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.372}]},{"osd":1,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51400000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38100000000000001},"min":{"1min":0.27400000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.56100000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.42399999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.371,"15min":0.36899999999999999},"min":{"1min":0.27400000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52000000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.313},{"interface":"front","average":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.61599999999999999,"5min":0.748,"15min":0.748},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73299999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.499,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79300000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.621,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84099999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.39500000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81599999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79600000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.49099999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86699999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.64100000000000001,"15min":0.64300000000000002},"min":{"1min":0.48199999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.46800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9284,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362556,"statfs":{"total":96636764160,"available":96627257344,"internally_reserved":0,"allocated":651264,"data_stored":298954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.504,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.47699999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.255,"5min":0.246,"15min":0.246},"max":{"1min":0.45200000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39800000000000002,"15min":0.40400000000000003},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58399999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.56399999999999995,"15min":0.59599999999999997},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54700000000000004,"5min":5.0540000000000003,"15min":5.0540000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.45200000000000001,"15min":0.46300000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.53700000000000003,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85799999999999998,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.626,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84399999999999997,"5min":1.2,"15min":1.2},"last":0.97099999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":1.169,"15min":1.169},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81999999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.89500000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81499999999999995,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":1.23,"15min":1.23},"last":1.01}]},{"osd":7,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77800000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85199999999999998,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.018}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9352,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362488,"statfs":{"total":96636764160,"available":96627187712,"internally_reserved":0,"allocated":655360,"data_stored":300446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.45600000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.41999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.66300000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.495,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.49099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68400000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73799999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.75600000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81499999999999995,"5min":1.117,"15min":1.117},"last":0.80700000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.78400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9156,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362684,"statfs":{"total":96636764160,"available":96627388416,"internally_reserved":0,"allocated":651264,"data_stored":299534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71899999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.34899999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.435},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55000000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60999999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.625,"15min":0.64000000000000001},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.64000000000000001,"15min":0.66000000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75700000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69499999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.67900000000000005,"15min":0.69699999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.751,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66400000000000003,"15min":0.67700000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83299999999999996,"5min":1.052,"15min":1.052},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:02:36.061 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:02:36.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.219+0000 7f3ea801a700 1 Processor -- start 2023-11-28T13:02:36.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.219+0000 7f3ea801a700 1 -- start start 2023-11-28T13:02:36.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea801a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ea015a6c0 0x7f3ea00cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea801a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3ea00b2660 0x7f3ea00b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea801a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea00cfb90 con 0x7f3ea00d0020 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea801a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea00b3020 con 0x7f3ea00b2660 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea801a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00cb930 con 0x7f3ea015a6c0 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea55b5700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f3ea00d0020 0x7f3ea00ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48836/0 (socket says 172.21.15.6:48836) 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea65b7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3ea00b2660 0x7f3ea00b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea55b5700 1 -- 172.21.15.6:0/3304538600 learned_addr learned my addr 172.21.15.6:0/3304538600 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea5db6700 1 --2- 172.21.15.6:0/3304538600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ea015a6c0 0x7f3ea00cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea4db4700 1 -- 172.21.15.6:0/3304538600 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216024028 0 0) 0x7f3ea00cfb90 con 0x7f3ea00d0020 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea4db4700 1 -- 172.21.15.6:0/3304538600 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e8c0035b0 con 0x7f3ea00d0020 2023-11-28T13:02:36.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea65b7700 1 -- 172.21.15.6:0/3304538600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ea015a6c0 msgr2=0x7f3ea00cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea65b7700 1 --2- 172.21.15.6:0/3304538600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ea015a6c0 0x7f3ea00cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea65b7700 1 -- 172.21.15.6:0/3304538600 >> v1:172.21.15.6:6789/0 conn(0x7f3ea00d0020 legacy=0x7f3ea00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.220+0000 7f3ea65b7700 1 -- 172.21.15.6:0/3304538600 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00cf260 con 0x7f3ea00b2660 2023-11-28T13:02:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea65b7700 1 --2- 172.21.15.6:0/3304538600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea00b2660 0x7f3ea00b2ae0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009d90 tx=0x7f3e9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8503b02156682f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea4db4700 1 -- 172.21.15.6:0/3304538600 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800e070 con 0x7f3ea00b2660 2023-11-28T13:02:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea4db4700 1 -- 172.21.15.6:0/3304538600 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9800bb20 con 0x7f3ea00b2660 2023-11-28T13:02:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea4db4700 1 -- 172.21.15.6:0/3304538600 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98012430 con 0x7f3ea00b2660 2023-11-28T13:02:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea801a700 1 -- 172.21.15.6:0/3304538600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea00b2660 msgr2=0x7f3ea00b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea801a700 1 --2- 172.21.15.6:0/3304538600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea00b2660 0x7f3ea00b2ae0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009d90 tx=0x7f3e9800b330 comp rx=0 tx=0).stop 2023-11-28T13:02:36.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea801a700 1 -- 172.21.15.6:0/3304538600 shutdown_connections 2023-11-28T13:02:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea801a700 1 --2- 172.21.15.6:0/3304538600 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea00b2660 0x7f3ea00b2ae0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea801a700 1 --2- 172.21.15.6:0/3304538600 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ea015a6c0 0x7f3ea00cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea801a700 1 -- 172.21.15.6:0/3304538600 >> 172.21.15.6:0/3304538600 conn(0x7f3ea00cd660 msgr2=0x7f3ea00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea801a700 1 -- 172.21.15.6:0/3304538600 shutdown_connections 2023-11-28T13:02:36.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.221+0000 7f3ea801a700 1 -- 172.21.15.6:0/3304538600 wait complete. 2023-11-28T13:02:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.223+0000 7f3ea801a700 1 Processor -- start 2023-11-28T13:02:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.223+0000 7f3ea801a700 1 -- start start 2023-11-28T13:02:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.223+0000 7f3ea801a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ea00b2660 0x7f3ea0156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.223+0000 7f3ea801a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ea00d0020 0x7f3ea015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.223+0000 7f3ea801a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea015a6c0 0x7f3ea00c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.223+0000 7f3ea801a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea015bae0 con 0x7f3ea015a6c0 2023-11-28T13:02:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.223+0000 7f3ea801a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea015da10 con 0x7f3ea00d0020 2023-11-28T13:02:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.223+0000 7f3ea801a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00cac90 con 0x7f3ea00b2660 2023-11-28T13:02:36.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.223+0000 7f3ea65b7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea015a6c0 0x7f3ea00c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea65b7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea015a6c0 0x7f3ea00c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33294/0 (socket says 172.21.15.6:33294) 2023-11-28T13:02:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea65b7700 1 -- 172.21.15.6:0/3251465170 learned_addr learned my addr 172.21.15.6:0/3251465170 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea5db6700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ea00b2660 0x7f3ea0156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea55b5700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ea00d0020 0x7f3ea015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea65b7700 1 -- 172.21.15.6:0/3251465170 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ea00b2660 msgr2=0x7f3ea0156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea65b7700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ea00b2660 0x7f3ea0156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea65b7700 1 -- 172.21.15.6:0/3251465170 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ea00d0020 msgr2=0x7f3ea015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea65b7700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ea00d0020 0x7f3ea015c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea65b7700 1 -- 172.21.15.6:0/3251465170 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e98009a70 con 0x7f3ea015a6c0 2023-11-28T13:02:36.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea55b5700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ea00d0020 0x7f3ea015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:36.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea65b7700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea015a6c0 0x7f3ea00c7cb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3e98014040 tx=0x7f3e98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3e967fc700 1 -- 172.21.15.6:0/3251465170 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e980129f0 con 0x7f3ea015a6c0 2023-11-28T13:02:36.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3e967fc700 1 -- 172.21.15.6:0/3251465170 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e98020070 con 0x7f3ea015a6c0 2023-11-28T13:02:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea801a700 1 -- 172.21.15.6:0/3251465170 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea00cf820 con 0x7f3ea015a6c0 2023-11-28T13:02:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3e967fc700 1 -- 172.21.15.6:0/3251465170 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98012db0 con 0x7f3ea015a6c0 2023-11-28T13:02:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.224+0000 7f3ea801a700 1 -- 172.21.15.6:0/3251465170 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea00cf540 con 0x7f3ea015a6c0 2023-11-28T13:02:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.225+0000 7f3ea801a700 1 -- 172.21.15.6:0/3251465170 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea00d0470 con 0x7f3ea015a6c0 2023-11-28T13:02:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.226+0000 7f3e967fc700 1 -- 172.21.15.6:0/3251465170 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3e98004040 con 0x7f3ea015a6c0 2023-11-28T13:02:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.226+0000 7f3e967fc700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3e840682f0 0x7f3e8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.226+0000 7f3e967fc700 1 -- 172.21.15.6:0/3251465170 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3e98022070 con 0x7f3ea015a6c0 2023-11-28T13:02:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.226+0000 7f3ea5db6700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3e840682f0 0x7f3e8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.227+0000 7f3ea5db6700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3e840682f0 0x7f3e8406a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009ad0 tx=0x7f3e9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.231+0000 7f3e967fc700 1 -- 172.21.15.6:0/3251465170 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3e98054900 con 0x7f3ea015a6c0 2023-11-28T13:02:36.420 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.419+0000 7f3ea801a700 1 -- 172.21.15.6:0/3251465170 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ea00675a0 con 0x7f3e840682f0 2023-11-28T13:02:36.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.428+0000 7f3e967fc700 1 -- 172.21.15.6:0/3251465170 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311899 (secure 0 0 0) 0x7f3ea00675a0 con 0x7f3e840682f0 2023-11-28T13:02:36.432 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:36.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 -- 172.21.15.6:0/3251465170 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3e840682f0 msgr2=0x7f3e8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3e840682f0 0x7f3e8406a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009ad0 tx=0x7f3e9c008040 comp rx=0 tx=0).stop 2023-11-28T13:02:36.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 -- 172.21.15.6:0/3251465170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea015a6c0 msgr2=0x7f3ea00c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea015a6c0 0x7f3ea00c7cb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3e98014040 tx=0x7f3e98004ad0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 -- 172.21.15.6:0/3251465170 shutdown_connections 2023-11-28T13:02:36.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ea015a6c0 0x7f3ea00c7cb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3e840682f0 0x7f3e8406a7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ea00b2660 0x7f3ea0156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 --2- 172.21.15.6:0/3251465170 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ea00d0020 0x7f3ea015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 -- 172.21.15.6:0/3251465170 >> 172.21.15.6:0/3251465170 conn(0x7f3ea00cd660 msgr2=0x7f3ea00b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:36.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 -- 172.21.15.6:0/3251465170 shutdown_connections 2023-11-28T13:02:36.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.431+0000 7f3ea801a700 1 -- 172.21.15.6:0/3251465170 wait complete. 2023-11-28T13:02:36.436 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:02:36.444 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2023-11-28T13:02:34.970866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":76064,"kb_used_data":6240,"kb_used_omap":0,"kb_used_meta":69760,"kb_avail":754898656,"statfs":{"total":773094113280,"available":773016223744,"internally_reserved":0,"allocated":6389760,"data_stored":3574589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71434240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.837796"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392564+0000","last_change":"2023-11-28T13:02:31.591399+0000","last_active":"2023-11-28T13:02:33.392564+0000","last_peered":"2023-11-28T13:02:33.392564+0000","last_clean":"2023-11-28T13:02:33.392564+0000","last_became_active":"2023-11-28T13:02:31.591062+0000","last_became_peered":"2023-11-28T13:02:31.591062+0000","last_unstale":"2023-11-28T13:02:33.392564+0000","last_undegraded":"2023-11-28T13:02:33.392564+0000","last_fullsized":"2023-11-28T13:02:33.392564+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:32.296512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390747+0000","last_change":"2023-11-28T13:02:31.583920+0000","last_active":"2023-11-28T13:02:33.390747+0000","last_peered":"2023-11-28T13:02:33.390747+0000","last_clean":"2023-11-28T13:02:33.390747+0000","last_became_active":"2023-11-28T13:02:31.583643+0000","last_became_peered":"2023-11-28T13:02:31.583643+0000","last_unstale":"2023-11-28T13:02:33.390747+0000","last_undegraded":"2023-11-28T13:02:33.390747+0000","last_fullsized":"2023-11-28T13:02:33.390747+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:41.354914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571938+0000","last_change":"2023-11-28T13:02:31.585521+0000","last_active":"2023-11-28T13:02:32.571938+0000","last_peered":"2023-11-28T13:02:32.571938+0000","last_clean":"2023-11-28T13:02:32.571938+0000","last_became_active":"2023-11-28T13:02:31.585222+0000","last_became_peered":"2023-11-28T13:02:31.585222+0000","last_unstale":"2023-11-28T13:02:32.571938+0000","last_undegraded":"2023-11-28T13:02:32.571938+0000","last_fullsized":"2023-11-28T13:02:32.571938+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:57.159327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571873+0000","last_change":"2023-11-28T13:02:31.586065+0000","last_active":"2023-11-28T13:02:32.571873+0000","last_peered":"2023-11-28T13:02:32.571873+0000","last_clean":"2023-11-28T13:02:32.571873+0000","last_became_active":"2023-11-28T13:02:31.585791+0000","last_became_peered":"2023-11-28T13:02:31.585791+0000","last_unstale":"2023-11-28T13:02:32.571873+0000","last_undegraded":"2023-11-28T13:02:32.571873+0000","last_fullsized":"2023-11-28T13:02:32.571873+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:45.032643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392445+0000","last_change":"2023-11-28T13:02:31.588039+0000","last_active":"2023-11-28T13:02:33.392445+0000","last_peered":"2023-11-28T13:02:33.392445+0000","last_clean":"2023-11-28T13:02:33.392445+0000","last_became_active":"2023-11-28T13:02:31.587794+0000","last_became_peered":"2023-11-28T13:02:31.587794+0000","last_unstale":"2023-11-28T13:02:33.392445+0000","last_undegraded":"2023-11-28T13:02:33.392445+0000","last_fullsized":"2023-11-28T13:02:33.392445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:17:43.148353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391918+0000","last_change":"2023-11-28T13:02:31.590953+0000","last_active":"2023-11-28T13:02:33.391918+0000","last_peered":"2023-11-28T13:02:33.391918+0000","last_clean":"2023-11-28T13:02:33.391918+0000","last_became_active":"2023-11-28T13:02:31.586161+0000","last_became_peered":"2023-11-28T13:02:31.586161+0000","last_unstale":"2023-11-28T13:02:33.391918+0000","last_undegraded":"2023-11-28T13:02:33.391918+0000","last_fullsized":"2023-11-28T13:02:33.391918+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:41.354914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641741+0000","last_change":"2023-11-28T13:02:31.588103+0000","last_active":"2023-11-28T13:02:32.641741+0000","last_peered":"2023-11-28T13:02:32.641741+0000","last_clean":"2023-11-28T13:02:32.641741+0000","last_became_active":"2023-11-28T13:02:31.587650+0000","last_became_peered":"2023-11-28T13:02:31.587650+0000","last_unstale":"2023-11-28T13:02:32.641741+0000","last_undegraded":"2023-11-28T13:02:32.641741+0000","last_fullsized":"2023-11-28T13:02:32.641741+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:12:37.986813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390455+0000","last_change":"2023-11-28T13:02:31.594513+0000","last_active":"2023-11-28T13:02:33.390455+0000","last_peered":"2023-11-28T13:02:33.390455+0000","last_clean":"2023-11-28T13:02:33.390455+0000","last_became_active":"2023-11-28T13:02:31.594253+0000","last_became_peered":"2023-11-28T13:02:31.594253+0000","last_unstale":"2023-11-28T13:02:33.390455+0000","last_undegraded":"2023-11-28T13:02:33.390455+0000","last_fullsized":"2023-11-28T13:02:33.390455+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:57:17.900013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.642231+0000","last_change":"2023-11-28T13:02:31.592287+0000","last_active":"2023-11-28T13:02:32.642231+0000","last_peered":"2023-11-28T13:02:32.642231+0000","last_clean":"2023-11-28T13:02:32.642231+0000","last_became_active":"2023-11-28T13:02:31.591985+0000","last_became_peered":"2023-11-28T13:02:31.591985+0000","last_unstale":"2023-11-28T13:02:32.642231+0000","last_undegraded":"2023-11-28T13:02:32.642231+0000","last_fullsized":"2023-11-28T13:02:32.642231+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:53.469232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570672+0000","last_change":"2023-11-28T13:02:31.580992+0000","last_active":"2023-11-28T13:02:32.570672+0000","last_peered":"2023-11-28T13:02:32.570672+0000","last_clean":"2023-11-28T13:02:32.570672+0000","last_became_active":"2023-11-28T13:02:31.580780+0000","last_became_peered":"2023-11-28T13:02:31.580780+0000","last_unstale":"2023-11-28T13:02:32.570672+0000","last_undegraded":"2023-11-28T13:02:32.570672+0000","last_fullsized":"2023-11-28T13:02:32.570672+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:23:59.400367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570557+0000","last_change":"2023-11-28T13:02:31.593244+0000","last_active":"2023-11-28T13:02:32.570557+0000","last_peered":"2023-11-28T13:02:32.570557+0000","last_clean":"2023-11-28T13:02:32.570557+0000","last_became_active":"2023-11-28T13:02:31.592982+0000","last_became_peered":"2023-11-28T13:02:31.592982+0000","last_unstale":"2023-11-28T13:02:32.570557+0000","last_undegraded":"2023-11-28T13:02:32.570557+0000","last_fullsized":"2023-11-28T13:02:32.570557+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:22:21.600980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572257+0000","last_change":"2023-11-28T13:02:31.592028+0000","last_active":"2023-11-28T13:02:32.572257+0000","last_peered":"2023-11-28T13:02:32.572257+0000","last_clean":"2023-11-28T13:02:32.572257+0000","last_became_active":"2023-11-28T13:02:31.591490+0000","last_became_peered":"2023-11-28T13:02:31.591490+0000","last_unstale":"2023-11-28T13:02:32.572257+0000","last_undegraded":"2023-11-28T13:02:32.572257+0000","last_fullsized":"2023-11-28T13:02:32.572257+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:02:25.266256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856393+0000","last_change":"2023-11-28T13:02:31.588171+0000","last_active":"2023-11-28T13:02:32.856393+0000","last_peered":"2023-11-28T13:02:32.856393+0000","last_clean":"2023-11-28T13:02:32.856393+0000","last_became_active":"2023-11-28T13:02:31.587869+0000","last_became_peered":"2023-11-28T13:02:31.587869+0000","last_unstale":"2023-11-28T13:02:32.856393+0000","last_undegraded":"2023-11-28T13:02:32.856393+0000","last_fullsized":"2023-11-28T13:02:32.856393+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:08.799483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641872+0000","last_change":"2023-11-28T13:02:31.584572+0000","last_active":"2023-11-28T13:02:32.641872+0000","last_peered":"2023-11-28T13:02:32.641872+0000","last_clean":"2023-11-28T13:02:32.641872+0000","last_became_active":"2023-11-28T13:02:31.584002+0000","last_became_peered":"2023-11-28T13:02:31.584002+0000","last_unstale":"2023-11-28T13:02:32.641872+0000","last_undegraded":"2023-11-28T13:02:32.641872+0000","last_fullsized":"2023-11-28T13:02:32.641872+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:55.094506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641772+0000","last_change":"2023-11-28T13:02:31.570902+0000","last_active":"2023-11-28T13:02:32.641772+0000","last_peered":"2023-11-28T13:02:32.641772+0000","last_clean":"2023-11-28T13:02:32.641772+0000","last_became_active":"2023-11-28T13:02:31.570577+0000","last_became_peered":"2023-11-28T13:02:31.570577+0000","last_unstale":"2023-11-28T13:02:32.641772+0000","last_undegraded":"2023-11-28T13:02:32.641772+0000","last_fullsized":"2023-11-28T13:02:32.641772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:15.215786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391593+0000","last_change":"2023-11-28T13:02:31.586105+0000","last_active":"2023-11-28T13:02:33.391593+0000","last_peered":"2023-11-28T13:02:33.391593+0000","last_clean":"2023-11-28T13:02:33.391593+0000","last_became_active":"2023-11-28T13:02:31.585808+0000","last_became_peered":"2023-11-28T13:02:31.585808+0000","last_unstale":"2023-11-28T13:02:33.391593+0000","last_undegraded":"2023-11-28T13:02:33.391593+0000","last_fullsized":"2023-11-28T13:02:33.391593+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:56.872669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390985+0000","last_change":"2023-11-28T13:02:31.588041+0000","last_active":"2023-11-28T13:02:33.390985+0000","last_peered":"2023-11-28T13:02:33.390985+0000","last_clean":"2023-11-28T13:02:33.390985+0000","last_became_active":"2023-11-28T13:02:31.587782+0000","last_became_peered":"2023-11-28T13:02:31.587782+0000","last_unstale":"2023-11-28T13:02:33.390985+0000","last_undegraded":"2023-11-28T13:02:33.390985+0000","last_fullsized":"2023-11-28T13:02:33.390985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:13.019187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571668+0000","last_change":"2023-11-28T13:02:31.587891+0000","last_active":"2023-11-28T13:02:32.571668+0000","last_peered":"2023-11-28T13:02:32.571668+0000","last_clean":"2023-11-28T13:02:32.571668+0000","last_became_active":"2023-11-28T13:02:31.587355+0000","last_became_peered":"2023-11-28T13:02:31.587355+0000","last_unstale":"2023-11-28T13:02:32.571668+0000","last_undegraded":"2023-11-28T13:02:32.571668+0000","last_fullsized":"2023-11-28T13:02:32.571668+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:41:40.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391824+0000","last_change":"2023-11-28T13:02:31.591666+0000","last_active":"2023-11-28T13:02:33.391824+0000","last_peered":"2023-11-28T13:02:33.391824+0000","last_clean":"2023-11-28T13:02:33.391824+0000","last_became_active":"2023-11-28T13:02:31.591350+0000","last_became_peered":"2023-11-28T13:02:31.591350+0000","last_unstale":"2023-11-28T13:02:33.391824+0000","last_undegraded":"2023-11-28T13:02:33.391824+0000","last_fullsized":"2023-11-28T13:02:33.391824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:20:11.378168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390765+0000","last_change":"2023-11-28T13:02:31.597001+0000","last_active":"2023-11-28T13:02:33.390765+0000","last_peered":"2023-11-28T13:02:33.390765+0000","last_clean":"2023-11-28T13:02:33.390765+0000","last_became_active":"2023-11-28T13:02:31.596636+0000","last_became_peered":"2023-11-28T13:02:31.596636+0000","last_unstale":"2023-11-28T13:02:33.390765+0000","last_undegraded":"2023-11-28T13:02:33.390765+0000","last_fullsized":"2023-11-28T13:02:33.390765+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:16.872040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572069+0000","last_change":"2023-11-28T13:02:31.588505+0000","last_active":"2023-11-28T13:02:32.572069+0000","last_peered":"2023-11-28T13:02:32.572069+0000","last_clean":"2023-11-28T13:02:32.572069+0000","last_became_active":"2023-11-28T13:02:31.588200+0000","last_became_peered":"2023-11-28T13:02:31.588200+0000","last_unstale":"2023-11-28T13:02:32.572069+0000","last_undegraded":"2023-11-28T13:02:32.572069+0000","last_fullsized":"2023-11-28T13:02:32.572069+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:43:21.360546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391007+0000","last_change":"2023-11-28T13:02:31.588603+0000","last_active":"2023-11-28T13:02:33.391007+0000","last_peered":"2023-11-28T13:02:33.391007+0000","last_clean":"2023-11-28T13:02:33.391007+0000","last_became_active":"2023-11-28T13:02:31.588355+0000","last_became_peered":"2023-11-28T13:02:31.588355+0000","last_unstale":"2023-11-28T13:02:33.391007+0000","last_undegraded":"2023-11-28T13:02:33.391007+0000","last_fullsized":"2023-11-28T13:02:33.391007+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:17.335526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571170+0000","last_change":"2023-11-28T13:02:31.591969+0000","last_active":"2023-11-28T13:02:32.571170+0000","last_peered":"2023-11-28T13:02:32.571170+0000","last_clean":"2023-11-28T13:02:32.571170+0000","last_became_active":"2023-11-28T13:02:31.591563+0000","last_became_peered":"2023-11-28T13:02:31.591563+0000","last_unstale":"2023-11-28T13:02:32.571170+0000","last_undegraded":"2023-11-28T13:02:32.571170+0000","last_fullsized":"2023-11-28T13:02:32.571170+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:13.477548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856246+0000","last_change":"2023-11-28T13:02:31.596962+0000","last_active":"2023-11-28T13:02:32.856246+0000","last_peered":"2023-11-28T13:02:32.856246+0000","last_clean":"2023-11-28T13:02:32.856246+0000","last_became_active":"2023-11-28T13:02:31.596681+0000","last_became_peered":"2023-11-28T13:02:31.596681+0000","last_unstale":"2023-11-28T13:02:32.856246+0000","last_undegraded":"2023-11-28T13:02:32.856246+0000","last_fullsized":"2023-11-28T13:02:32.856246+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:29:52.301201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392290+0000","last_change":"2023-11-28T13:02:31.579462+0000","last_active":"2023-11-28T13:02:33.392290+0000","last_peered":"2023-11-28T13:02:33.392290+0000","last_clean":"2023-11-28T13:02:33.392290+0000","last_became_active":"2023-11-28T13:02:31.579189+0000","last_became_peered":"2023-11-28T13:02:31.579189+0000","last_unstale":"2023-11-28T13:02:33.392290+0000","last_undegraded":"2023-11-28T13:02:33.392290+0000","last_fullsized":"2023-11-28T13:02:33.392290+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:02:14.215069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571424+0000","last_change":"2023-11-28T13:02:31.586885+0000","last_active":"2023-11-28T13:02:32.571424+0000","last_peered":"2023-11-28T13:02:32.571424+0000","last_clean":"2023-11-28T13:02:32.571424+0000","last_became_active":"2023-11-28T13:02:31.586485+0000","last_became_peered":"2023-11-28T13:02:31.586485+0000","last_unstale":"2023-11-28T13:02:32.571424+0000","last_undegraded":"2023-11-28T13:02:32.571424+0000","last_fullsized":"2023-11-28T13:02:32.571424+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:53:55.094506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392211+0000","last_change":"2023-11-28T13:02:31.597906+0000","last_active":"2023-11-28T13:02:33.392211+0000","last_peered":"2023-11-28T13:02:33.392211+0000","last_clean":"2023-11-28T13:02:33.392211+0000","last_became_active":"2023-11-28T13:02:31.597087+0000","last_became_peered":"2023-11-28T13:02:31.597087+0000","last_unstale":"2023-11-28T13:02:33.392211+0000","last_undegraded":"2023-11-28T13:02:33.392211+0000","last_fullsized":"2023-11-28T13:02:33.392211+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:57.092839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392511+0000","last_change":"2023-11-28T13:02:31.586899+0000","last_active":"2023-11-28T13:02:33.392511+0000","last_peered":"2023-11-28T13:02:33.392511+0000","last_clean":"2023-11-28T13:02:33.392511+0000","last_became_active":"2023-11-28T13:02:31.586608+0000","last_became_peered":"2023-11-28T13:02:31.586608+0000","last_unstale":"2023-11-28T13:02:33.392511+0000","last_undegraded":"2023-11-28T13:02:33.392511+0000","last_fullsized":"2023-11-28T13:02:33.392511+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:04.210874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390596+0000","last_change":"2023-11-28T13:02:31.592049+0000","last_active":"2023-11-28T13:02:33.390596+0000","last_peered":"2023-11-28T13:02:33.390596+0000","last_clean":"2023-11-28T13:02:33.390596+0000","last_became_active":"2023-11-28T13:02:31.591751+0000","last_became_peered":"2023-11-28T13:02:31.591751+0000","last_unstale":"2023-11-28T13:02:33.390596+0000","last_undegraded":"2023-11-28T13:02:33.390596+0000","last_fullsized":"2023-11-28T13:02:33.390596+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:32:24.133607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571342+0000","last_change":"2023-11-28T13:02:31.580080+0000","last_active":"2023-11-28T13:02:32.571342+0000","last_peered":"2023-11-28T13:02:32.571342+0000","last_clean":"2023-11-28T13:02:32.571342+0000","last_became_active":"2023-11-28T13:02:31.579755+0000","last_became_peered":"2023-11-28T13:02:31.579755+0000","last_unstale":"2023-11-28T13:02:32.571342+0000","last_undegraded":"2023-11-28T13:02:32.571342+0000","last_fullsized":"2023-11-28T13:02:32.571342+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:14:48.513955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392013+0000","last_change":"2023-11-28T13:02:31.577331+0000","last_active":"2023-11-28T13:02:33.392013+0000","last_peered":"2023-11-28T13:02:33.392013+0000","last_clean":"2023-11-28T13:02:33.392013+0000","last_became_active":"2023-11-28T13:02:31.577070+0000","last_became_peered":"2023-11-28T13:02:31.577070+0000","last_unstale":"2023-11-28T13:02:33.392013+0000","last_undegraded":"2023-11-28T13:02:33.392013+0000","last_fullsized":"2023-11-28T13:02:33.392013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:02:46.233325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391624+0000","last_change":"2023-11-28T13:02:31.589485+0000","last_active":"2023-11-28T13:02:33.391624+0000","last_peered":"2023-11-28T13:02:33.391624+0000","last_clean":"2023-11-28T13:02:33.391624+0000","last_became_active":"2023-11-28T13:02:31.589233+0000","last_became_peered":"2023-11-28T13:02:31.589233+0000","last_unstale":"2023-11-28T13:02:33.391624+0000","last_undegraded":"2023-11-28T13:02:33.391624+0000","last_fullsized":"2023-11-28T13:02:33.391624+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:30.759324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392252+0000","last_change":"2023-11-28T13:02:31.589787+0000","last_active":"2023-11-28T13:02:33.392252+0000","last_peered":"2023-11-28T13:02:33.392252+0000","last_clean":"2023-11-28T13:02:33.392252+0000","last_became_active":"2023-11-28T13:02:31.586183+0000","last_became_peered":"2023-11-28T13:02:31.586183+0000","last_unstale":"2023-11-28T13:02:33.392252+0000","last_undegraded":"2023-11-28T13:02:33.392252+0000","last_fullsized":"2023-11-28T13:02:33.392252+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:02:07.071203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392392+0000","last_change":"2023-11-28T13:02:31.168749+0000","last_active":"2023-11-28T13:02:33.392392+0000","last_peered":"2023-11-28T13:02:33.392392+0000","last_clean":"2023-11-28T13:02:33.392392+0000","last_became_active":"2023-11-28T13:02:31.167729+0000","last_became_peered":"2023-11-28T13:02:31.167729+0000","last_unstale":"2023-11-28T13:02:33.392392+0000","last_undegraded":"2023-11-28T13:02:33.392392+0000","last_fullsized":"2023-11-28T13:02:33.392392+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:50:00.079781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570753+0000","last_change":"2023-11-28T13:02:31.583640+0000","last_active":"2023-11-28T13:02:32.570753+0000","last_peered":"2023-11-28T13:02:32.570753+0000","last_clean":"2023-11-28T13:02:32.570753+0000","last_became_active":"2023-11-28T13:02:31.583195+0000","last_became_peered":"2023-11-28T13:02:31.583195+0000","last_unstale":"2023-11-28T13:02:32.570753+0000","last_undegraded":"2023-11-28T13:02:32.570753+0000","last_fullsized":"2023-11-28T13:02:32.570753+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:04:31.030504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390886+0000","last_change":"2023-11-28T13:02:31.160037+0000","last_active":"2023-11-28T13:02:33.390886+0000","last_peered":"2023-11-28T13:02:33.390886+0000","last_clean":"2023-11-28T13:02:33.390886+0000","last_became_active":"2023-11-28T13:02:31.159581+0000","last_became_peered":"2023-11-28T13:02:31.159581+0000","last_unstale":"2023-11-28T13:02:33.390886+0000","last_undegraded":"2023-11-28T13:02:33.390886+0000","last_fullsized":"2023-11-28T13:02:33.390886+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:14.909317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391996+0000","last_change":"2023-11-28T13:02:31.595649+0000","last_active":"2023-11-28T13:02:33.391996+0000","last_peered":"2023-11-28T13:02:33.391996+0000","last_clean":"2023-11-28T13:02:33.391996+0000","last_became_active":"2023-11-28T13:02:31.586303+0000","last_became_peered":"2023-11-28T13:02:31.586303+0000","last_unstale":"2023-11-28T13:02:33.391996+0000","last_undegraded":"2023-11-28T13:02:33.391996+0000","last_fullsized":"2023-11-28T13:02:33.391996+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:29:23.609219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392568+0000","last_change":"2023-11-28T13:02:31.163301+0000","last_active":"2023-11-28T13:02:33.392568+0000","last_peered":"2023-11-28T13:02:33.392568+0000","last_clean":"2023-11-28T13:02:33.392568+0000","last_became_active":"2023-11-28T13:02:31.163004+0000","last_became_peered":"2023-11-28T13:02:31.163004+0000","last_unstale":"2023-11-28T13:02:33.392568+0000","last_undegraded":"2023-11-28T13:02:33.392568+0000","last_fullsized":"2023-11-28T13:02:33.392568+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:25:09.784960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392330+0000","last_change":"2023-11-28T13:02:31.598141+0000","last_active":"2023-11-28T13:02:33.392330+0000","last_peered":"2023-11-28T13:02:33.392330+0000","last_clean":"2023-11-28T13:02:33.392330+0000","last_became_active":"2023-11-28T13:02:31.596640+0000","last_became_peered":"2023-11-28T13:02:31.596640+0000","last_unstale":"2023-11-28T13:02:33.392330+0000","last_undegraded":"2023-11-28T13:02:33.392330+0000","last_fullsized":"2023-11-28T13:02:33.392330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:36:16.872040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572133+0000","last_change":"2023-11-28T13:02:31.158200+0000","last_active":"2023-11-28T13:02:32.572133+0000","last_peered":"2023-11-28T13:02:32.572133+0000","last_clean":"2023-11-28T13:02:32.572133+0000","last_became_active":"2023-11-28T13:02:31.157926+0000","last_became_peered":"2023-11-28T13:02:31.157926+0000","last_unstale":"2023-11-28T13:02:32.572133+0000","last_undegraded":"2023-11-28T13:02:32.572133+0000","last_fullsized":"2023-11-28T13:02:32.572133+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:59.239818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390979+0000","last_change":"2023-11-28T13:02:31.592098+0000","last_active":"2023-11-28T13:02:33.390979+0000","last_peered":"2023-11-28T13:02:33.390979+0000","last_clean":"2023-11-28T13:02:33.390979+0000","last_became_active":"2023-11-28T13:02:31.591833+0000","last_became_peered":"2023-11-28T13:02:31.591833+0000","last_unstale":"2023-11-28T13:02:33.390979+0000","last_undegraded":"2023-11-28T13:02:33.390979+0000","last_fullsized":"2023-11-28T13:02:33.390979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:06:56.872669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641921+0000","last_change":"2023-11-28T13:02:31.153946+0000","last_active":"2023-11-28T13:02:32.641921+0000","last_peered":"2023-11-28T13:02:32.641921+0000","last_clean":"2023-11-28T13:02:32.641921+0000","last_became_active":"2023-11-28T13:02:31.153698+0000","last_became_peered":"2023-11-28T13:02:31.153698+0000","last_unstale":"2023-11-28T13:02:32.641921+0000","last_undegraded":"2023-11-28T13:02:32.641921+0000","last_fullsized":"2023-11-28T13:02:32.641921+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:43:20.358576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390793+0000","last_change":"2023-11-28T13:02:31.597441+0000","last_active":"2023-11-28T13:02:33.390793+0000","last_peered":"2023-11-28T13:02:33.390793+0000","last_clean":"2023-11-28T13:02:33.390793+0000","last_became_active":"2023-11-28T13:02:31.597167+0000","last_became_peered":"2023-11-28T13:02:31.597167+0000","last_unstale":"2023-11-28T13:02:33.390793+0000","last_undegraded":"2023-11-28T13:02:33.390793+0000","last_fullsized":"2023-11-28T13:02:33.390793+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:29:23.609219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392033+0000","last_change":"2023-11-28T13:02:31.163261+0000","last_active":"2023-11-28T13:02:33.392033+0000","last_peered":"2023-11-28T13:02:33.392033+0000","last_clean":"2023-11-28T13:02:33.392033+0000","last_became_active":"2023-11-28T13:02:31.162980+0000","last_became_peered":"2023-11-28T13:02:31.162980+0000","last_unstale":"2023-11-28T13:02:33.392033+0000","last_undegraded":"2023-11-28T13:02:33.392033+0000","last_fullsized":"2023-11-28T13:02:33.392033+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:33.575223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856229+0000","last_change":"2023-11-28T13:02:31.591930+0000","last_active":"2023-11-28T13:02:32.856229+0000","last_peered":"2023-11-28T13:02:32.856229+0000","last_clean":"2023-11-28T13:02:32.856229+0000","last_became_active":"2023-11-28T13:02:31.591658+0000","last_became_peered":"2023-11-28T13:02:31.591658+0000","last_unstale":"2023-11-28T13:02:32.856229+0000","last_undegraded":"2023-11-28T13:02:32.856229+0000","last_fullsized":"2023-11-28T13:02:32.856229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:15:56.096171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571999+0000","last_change":"2023-11-28T13:02:31.154256+0000","last_active":"2023-11-28T13:02:32.571999+0000","last_peered":"2023-11-28T13:02:32.571999+0000","last_clean":"2023-11-28T13:02:32.571999+0000","last_became_active":"2023-11-28T13:02:31.153880+0000","last_became_peered":"2023-11-28T13:02:31.153880+0000","last_unstale":"2023-11-28T13:02:32.571999+0000","last_undegraded":"2023-11-28T13:02:32.571999+0000","last_fullsized":"2023-11-28T13:02:32.571999+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:32:19.194642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390458+0000","last_change":"2023-11-28T13:02:31.589235+0000","last_active":"2023-11-28T13:02:33.390458+0000","last_peered":"2023-11-28T13:02:33.390458+0000","last_clean":"2023-11-28T13:02:33.390458+0000","last_became_active":"2023-11-28T13:02:31.588946+0000","last_became_peered":"2023-11-28T13:02:31.588946+0000","last_unstale":"2023-11-28T13:02:33.390458+0000","last_undegraded":"2023-11-28T13:02:33.390458+0000","last_fullsized":"2023-11-28T13:02:33.390458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:48:30.759324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641862+0000","last_change":"2023-11-28T13:02:31.161186+0000","last_active":"2023-11-28T13:02:32.641862+0000","last_peered":"2023-11-28T13:02:32.641862+0000","last_clean":"2023-11-28T13:02:32.641862+0000","last_became_active":"2023-11-28T13:02:31.160828+0000","last_became_peered":"2023-11-28T13:02:31.160828+0000","last_unstale":"2023-11-28T13:02:32.641862+0000","last_undegraded":"2023-11-28T13:02:32.641862+0000","last_fullsized":"2023-11-28T13:02:32.641862+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:16:59.848012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.642235+0000","last_change":"2023-11-28T13:02:31.573929+0000","last_active":"2023-11-28T13:02:32.642235+0000","last_peered":"2023-11-28T13:02:32.642235+0000","last_clean":"2023-11-28T13:02:32.642235+0000","last_became_active":"2023-11-28T13:02:31.573604+0000","last_became_peered":"2023-11-28T13:02:31.573604+0000","last_unstale":"2023-11-28T13:02:32.642235+0000","last_undegraded":"2023-11-28T13:02:32.642235+0000","last_fullsized":"2023-11-28T13:02:32.642235+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:07.955246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571501+0000","last_change":"2023-11-28T13:02:31.168769+0000","last_active":"2023-11-28T13:02:32.571501+0000","last_peered":"2023-11-28T13:02:32.571501+0000","last_clean":"2023-11-28T13:02:32.571501+0000","last_became_active":"2023-11-28T13:02:31.168462+0000","last_became_peered":"2023-11-28T13:02:31.168462+0000","last_unstale":"2023-11-28T13:02:32.571501+0000","last_undegraded":"2023-11-28T13:02:32.571501+0000","last_fullsized":"2023-11-28T13:02:32.571501+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:50.251928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571428+0000","last_change":"2023-11-28T13:02:31.581348+0000","last_active":"2023-11-28T13:02:32.571428+0000","last_peered":"2023-11-28T13:02:32.571428+0000","last_clean":"2023-11-28T13:02:32.571428+0000","last_became_active":"2023-11-28T13:02:31.581082+0000","last_became_peered":"2023-11-28T13:02:31.581082+0000","last_unstale":"2023-11-28T13:02:32.571428+0000","last_undegraded":"2023-11-28T13:02:32.571428+0000","last_fullsized":"2023-11-28T13:02:32.571428+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:07:22.317413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390996+0000","last_change":"2023-11-28T13:02:31.159847+0000","last_active":"2023-11-28T13:02:33.390996+0000","last_peered":"2023-11-28T13:02:33.390996+0000","last_clean":"2023-11-28T13:02:33.390996+0000","last_became_active":"2023-11-28T13:02:31.159227+0000","last_became_peered":"2023-11-28T13:02:31.159227+0000","last_unstale":"2023-11-28T13:02:33.390996+0000","last_undegraded":"2023-11-28T13:02:33.390996+0000","last_fullsized":"2023-11-28T13:02:33.390996+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:46.056976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570472+0000","last_change":"2023-11-28T13:02:31.570890+0000","last_active":"2023-11-28T13:02:32.570472+0000","last_peered":"2023-11-28T13:02:32.570472+0000","last_clean":"2023-11-28T13:02:32.570472+0000","last_became_active":"2023-11-28T13:02:31.570573+0000","last_became_peered":"2023-11-28T13:02:31.570573+0000","last_unstale":"2023-11-28T13:02:32.570472+0000","last_undegraded":"2023-11-28T13:02:32.570472+0000","last_fullsized":"2023-11-28T13:02:32.570472+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:39:38.704425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571870+0000","last_change":"2023-11-28T13:02:31.157987+0000","last_active":"2023-11-28T13:02:32.571870+0000","last_peered":"2023-11-28T13:02:32.571870+0000","last_clean":"2023-11-28T13:02:32.571870+0000","last_became_active":"2023-11-28T13:02:31.157605+0000","last_became_peered":"2023-11-28T13:02:31.157605+0000","last_unstale":"2023-11-28T13:02:32.571870+0000","last_undegraded":"2023-11-28T13:02:32.571870+0000","last_fullsized":"2023-11-28T13:02:32.571870+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:05:08.041508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572042+0000","last_change":"2023-11-28T13:02:31.581849+0000","last_active":"2023-11-28T13:02:32.572042+0000","last_peered":"2023-11-28T13:02:32.572042+0000","last_clean":"2023-11-28T13:02:32.572042+0000","last_became_active":"2023-11-28T13:02:31.581378+0000","last_became_peered":"2023-11-28T13:02:31.581378+0000","last_unstale":"2023-11-28T13:02:32.572042+0000","last_undegraded":"2023-11-28T13:02:32.572042+0000","last_fullsized":"2023-11-28T13:02:32.572042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:00.849982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.642111+0000","last_change":"2023-11-28T13:02:31.168730+0000","last_active":"2023-11-28T13:02:32.642111+0000","last_peered":"2023-11-28T13:02:32.642111+0000","last_clean":"2023-11-28T13:02:32.642111+0000","last_became_active":"2023-11-28T13:02:31.168366+0000","last_became_peered":"2023-11-28T13:02:31.168366+0000","last_unstale":"2023-11-28T13:02:32.642111+0000","last_undegraded":"2023-11-28T13:02:32.642111+0000","last_fullsized":"2023-11-28T13:02:32.642111+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:07:21.315443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572491+0000","last_change":"2023-11-28T13:02:31.586586+0000","last_active":"2023-11-28T13:02:32.572491+0000","last_peered":"2023-11-28T13:02:32.572491+0000","last_clean":"2023-11-28T13:02:32.572491+0000","last_became_active":"2023-11-28T13:02:31.586073+0000","last_became_peered":"2023-11-28T13:02:31.586073+0000","last_unstale":"2023-11-28T13:02:32.572491+0000","last_undegraded":"2023-11-28T13:02:32.572491+0000","last_fullsized":"2023-11-28T13:02:32.572491+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:07.955246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391032+0000","last_change":"2023-11-28T13:02:31.160978+0000","last_active":"2023-11-28T13:02:33.391032+0000","last_peered":"2023-11-28T13:02:33.391032+0000","last_clean":"2023-11-28T13:02:33.391032+0000","last_became_active":"2023-11-28T13:02:31.160401+0000","last_became_peered":"2023-11-28T13:02:31.160401+0000","last_unstale":"2023-11-28T13:02:33.391032+0000","last_undegraded":"2023-11-28T13:02:33.391032+0000","last_fullsized":"2023-11-28T13:02:33.391032+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:11:17.271316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571510+0000","last_change":"2023-11-28T13:02:31.587922+0000","last_active":"2023-11-28T13:02:32.571510+0000","last_peered":"2023-11-28T13:02:32.571510+0000","last_clean":"2023-11-28T13:02:32.571510+0000","last_became_active":"2023-11-28T13:02:31.587621+0000","last_became_peered":"2023-11-28T13:02:31.587621+0000","last_unstale":"2023-11-28T13:02:32.571510+0000","last_undegraded":"2023-11-28T13:02:32.571510+0000","last_fullsized":"2023-11-28T13:02:32.571510+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:23.864039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856254+0000","last_change":"2023-11-28T13:02:31.158209+0000","last_active":"2023-11-28T13:02:32.856254+0000","last_peered":"2023-11-28T13:02:32.856254+0000","last_clean":"2023-11-28T13:02:32.856254+0000","last_became_active":"2023-11-28T13:02:31.157931+0000","last_became_peered":"2023-11-28T13:02:31.157931+0000","last_unstale":"2023-11-28T13:02:32.856254+0000","last_undegraded":"2023-11-28T13:02:32.856254+0000","last_fullsized":"2023-11-28T13:02:32.856254+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:42.997557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641740+0000","last_change":"2023-11-28T13:02:31.584276+0000","last_active":"2023-11-28T13:02:32.641740+0000","last_peered":"2023-11-28T13:02:32.641740+0000","last_clean":"2023-11-28T13:02:32.641740+0000","last_became_active":"2023-11-28T13:02:31.583989+0000","last_became_peered":"2023-11-28T13:02:31.583989+0000","last_unstale":"2023-11-28T13:02:32.641740+0000","last_undegraded":"2023-11-28T13:02:32.641740+0000","last_fullsized":"2023-11-28T13:02:32.641740+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:00.075913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571740+0000","last_change":"2023-11-28T13:02:31.156773+0000","last_active":"2023-11-28T13:02:32.571740+0000","last_peered":"2023-11-28T13:02:32.571740+0000","last_clean":"2023-11-28T13:02:32.571740+0000","last_became_active":"2023-11-28T13:02:31.156454+0000","last_became_peered":"2023-11-28T13:02:31.156454+0000","last_unstale":"2023-11-28T13:02:32.571740+0000","last_undegraded":"2023-11-28T13:02:32.571740+0000","last_fullsized":"2023-11-28T13:02:32.571740+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:51:26.161562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571284+0000","last_change":"2023-11-28T13:02:31.585050+0000","last_active":"2023-11-28T13:02:32.571284+0000","last_peered":"2023-11-28T13:02:32.571284+0000","last_clean":"2023-11-28T13:02:32.571284+0000","last_became_active":"2023-11-28T13:02:31.584713+0000","last_became_peered":"2023-11-28T13:02:31.584713+0000","last_unstale":"2023-11-28T13:02:32.571284+0000","last_undegraded":"2023-11-28T13:02:32.571284+0000","last_fullsized":"2023-11-28T13:02:32.571284+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:54:15.802346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":19,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570971+0000","last_change":"2023-11-28T13:02:31.157623+0000","last_active":"2023-11-28T13:02:32.570971+0000","last_peered":"2023-11-28T13:02:32.570971+0000","last_clean":"2023-11-28T13:02:32.570971+0000","last_became_active":"2023-11-28T13:02:31.157281+0000","last_became_peered":"2023-11-28T13:02:31.157281+0000","last_unstale":"2023-11-28T13:02:32.570971+0000","last_undegraded":"2023-11-28T13:02:32.570971+0000","last_fullsized":"2023-11-28T13:02:32.570971+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:19:12.819597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571540+0000","last_change":"2023-11-28T13:02:31.587125+0000","last_active":"2023-11-28T13:02:32.571540+0000","last_peered":"2023-11-28T13:02:32.571540+0000","last_clean":"2023-11-28T13:02:32.571540+0000","last_became_active":"2023-11-28T13:02:31.586818+0000","last_became_peered":"2023-11-28T13:02:31.586818+0000","last_unstale":"2023-11-28T13:02:32.571540+0000","last_undegraded":"2023-11-28T13:02:32.571540+0000","last_fullsized":"2023-11-28T13:02:32.571540+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:13.477548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570812+0000","last_change":"2023-11-28T13:02:31.163041+0000","last_active":"2023-11-28T13:02:32.570812+0000","last_peered":"2023-11-28T13:02:32.570812+0000","last_clean":"2023-11-28T13:02:32.570812+0000","last_became_active":"2023-11-28T13:02:31.162740+0000","last_became_peered":"2023-11-28T13:02:31.162740+0000","last_unstale":"2023-11-28T13:02:32.570812+0000","last_undegraded":"2023-11-28T13:02:32.570812+0000","last_fullsized":"2023-11-28T13:02:32.570812+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:09:06.353454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571756+0000","last_change":"2023-11-28T13:02:31.583605+0000","last_active":"2023-11-28T13:02:32.571756+0000","last_peered":"2023-11-28T13:02:32.571756+0000","last_clean":"2023-11-28T13:02:32.571756+0000","last_became_active":"2023-11-28T13:02:31.583312+0000","last_became_peered":"2023-11-28T13:02:31.583312+0000","last_unstale":"2023-11-28T13:02:32.571756+0000","last_undegraded":"2023-11-28T13:02:32.571756+0000","last_fullsized":"2023-11-28T13:02:32.571756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:53.469232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392644+0000","last_change":"2023-11-28T13:02:31.168129+0000","last_active":"2023-11-28T13:02:33.392644+0000","last_peered":"2023-11-28T13:02:33.392644+0000","last_clean":"2023-11-28T13:02:33.392644+0000","last_became_active":"2023-11-28T13:02:31.167679+0000","last_became_peered":"2023-11-28T13:02:31.167679+0000","last_unstale":"2023-11-28T13:02:33.392644+0000","last_undegraded":"2023-11-28T13:02:33.392644+0000","last_fullsized":"2023-11-28T13:02:33.392644+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:19:02.510602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570511+0000","last_change":"2023-11-28T13:02:31.573089+0000","last_active":"2023-11-28T13:02:32.570511+0000","last_peered":"2023-11-28T13:02:32.570511+0000","last_clean":"2023-11-28T13:02:32.570511+0000","last_became_active":"2023-11-28T13:02:31.572854+0000","last_became_peered":"2023-11-28T13:02:31.572854+0000","last_unstale":"2023-11-28T13:02:32.570511+0000","last_undegraded":"2023-11-28T13:02:32.570511+0000","last_fullsized":"2023-11-28T13:02:32.570511+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:47.613849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856443+0000","last_change":"2023-11-28T13:02:31.155385+0000","last_active":"2023-11-28T13:02:32.856443+0000","last_peered":"2023-11-28T13:02:32.856443+0000","last_clean":"2023-11-28T13:02:32.856443+0000","last_became_active":"2023-11-28T13:02:31.155110+0000","last_became_peered":"2023-11-28T13:02:31.155110+0000","last_unstale":"2023-11-28T13:02:32.856443+0000","last_undegraded":"2023-11-28T13:02:32.856443+0000","last_fullsized":"2023-11-28T13:02:32.856443+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:31.087515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392471+0000","last_change":"2023-11-28T13:02:31.597730+0000","last_active":"2023-11-28T13:02:33.392471+0000","last_peered":"2023-11-28T13:02:33.392471+0000","last_clean":"2023-11-28T13:02:33.392471+0000","last_became_active":"2023-11-28T13:02:31.596819+0000","last_became_peered":"2023-11-28T13:02:31.596819+0000","last_unstale":"2023-11-28T13:02:33.392471+0000","last_undegraded":"2023-11-28T13:02:33.392471+0000","last_fullsized":"2023-11-28T13:02:33.392471+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:12:32.498368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572030+0000","last_change":"2023-11-28T13:02:31.153020+0000","last_active":"2023-11-28T13:02:32.572030+0000","last_peered":"2023-11-28T13:02:32.572030+0000","last_clean":"2023-11-28T13:02:32.572030+0000","last_became_active":"2023-11-28T13:02:31.152726+0000","last_became_peered":"2023-11-28T13:02:31.152726+0000","last_unstale":"2023-11-28T13:02:32.572030+0000","last_undegraded":"2023-11-28T13:02:32.572030+0000","last_fullsized":"2023-11-28T13:02:32.572030+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:59:46.981104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390779+0000","last_change":"2023-11-28T13:02:31.587261+0000","last_active":"2023-11-28T13:02:33.390779+0000","last_peered":"2023-11-28T13:02:33.390779+0000","last_clean":"2023-11-28T13:02:33.390779+0000","last_became_active":"2023-11-28T13:02:31.584178+0000","last_became_peered":"2023-11-28T13:02:31.584178+0000","last_unstale":"2023-11-28T13:02:33.390779+0000","last_undegraded":"2023-11-28T13:02:33.390779+0000","last_fullsized":"2023-11-28T13:02:33.390779+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:12:32.498368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570027+0000","last_change":"2023-11-28T13:02:31.156804+0000","last_active":"2023-11-28T13:02:32.570027+0000","last_peered":"2023-11-28T13:02:32.570027+0000","last_clean":"2023-11-28T13:02:32.570027+0000","last_became_active":"2023-11-28T13:02:31.156515+0000","last_became_peered":"2023-11-28T13:02:31.156515+0000","last_unstale":"2023-11-28T13:02:32.570027+0000","last_undegraded":"2023-11-28T13:02:32.570027+0000","last_fullsized":"2023-11-28T13:02:32.570027+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:28:56.259977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571626+0000","last_change":"2023-11-28T13:02:31.592265+0000","last_active":"2023-11-28T13:02:32.571626+0000","last_peered":"2023-11-28T13:02:32.571626+0000","last_clean":"2023-11-28T13:02:32.571626+0000","last_became_active":"2023-11-28T13:02:31.591625+0000","last_became_peered":"2023-11-28T13:02:31.591625+0000","last_unstale":"2023-11-28T13:02:32.571626+0000","last_undegraded":"2023-11-28T13:02:32.571626+0000","last_fullsized":"2023-11-28T13:02:32.571626+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:11:45.032643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.641909+0000","last_change":"2023-11-28T13:02:31.169320+0000","last_active":"2023-11-28T13:02:32.641909+0000","last_peered":"2023-11-28T13:02:32.641909+0000","last_clean":"2023-11-28T13:02:32.641909+0000","last_became_active":"2023-11-28T13:02:31.168966+0000","last_became_peered":"2023-11-28T13:02:31.168966+0000","last_unstale":"2023-11-28T13:02:32.641909+0000","last_undegraded":"2023-11-28T13:02:32.641909+0000","last_fullsized":"2023-11-28T13:02:32.641909+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:23:56.157357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.569965+0000","last_change":"2023-11-28T13:02:31.582200+0000","last_active":"2023-11-28T13:02:32.569965+0000","last_peered":"2023-11-28T13:02:32.569965+0000","last_clean":"2023-11-28T13:02:32.569965+0000","last_became_active":"2023-11-28T13:02:31.581927+0000","last_became_peered":"2023-11-28T13:02:31.581927+0000","last_unstale":"2023-11-28T13:02:32.569965+0000","last_undegraded":"2023-11-28T13:02:32.569965+0000","last_fullsized":"2023-11-28T13:02:32.569965+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:58.821868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570129+0000","last_change":"2023-11-28T13:02:31.151164+0000","last_active":"2023-11-28T13:02:32.570129+0000","last_peered":"2023-11-28T13:02:32.570129+0000","last_clean":"2023-11-28T13:02:32.570129+0000","last_became_active":"2023-11-28T13:02:31.150880+0000","last_became_peered":"2023-11-28T13:02:31.150880+0000","last_unstale":"2023-11-28T13:02:32.570129+0000","last_undegraded":"2023-11-28T13:02:32.570129+0000","last_fullsized":"2023-11-28T13:02:32.570129+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:20:24.962468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390465+0000","last_change":"2023-11-28T13:02:31.588209+0000","last_active":"2023-11-28T13:02:33.390465+0000","last_peered":"2023-11-28T13:02:33.390465+0000","last_clean":"2023-11-28T13:02:33.390465+0000","last_became_active":"2023-11-28T13:02:31.587833+0000","last_became_peered":"2023-11-28T13:02:31.587833+0000","last_unstale":"2023-11-28T13:02:33.390465+0000","last_undegraded":"2023-11-28T13:02:33.390465+0000","last_fullsized":"2023-11-28T13:02:33.390465+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:02:07.071203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390575+0000","last_change":"2023-11-28T13:02:31.157931+0000","last_active":"2023-11-28T13:02:33.390575+0000","last_peered":"2023-11-28T13:02:33.390575+0000","last_clean":"2023-11-28T13:02:33.390575+0000","last_became_active":"2023-11-28T13:02:31.157478+0000","last_became_peered":"2023-11-28T13:02:31.157478+0000","last_unstale":"2023-11-28T13:02:33.390575+0000","last_undegraded":"2023-11-28T13:02:33.390575+0000","last_fullsized":"2023-11-28T13:02:33.390575+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:50:00.079781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571992+0000","last_change":"2023-11-28T13:02:31.581663+0000","last_active":"2023-11-28T13:02:32.571992+0000","last_peered":"2023-11-28T13:02:32.571992+0000","last_clean":"2023-11-28T13:02:32.571992+0000","last_became_active":"2023-11-28T13:02:31.581031+0000","last_became_peered":"2023-11-28T13:02:31.581031+0000","last_unstale":"2023-11-28T13:02:32.571992+0000","last_undegraded":"2023-11-28T13:02:32.571992+0000","last_fullsized":"2023-11-28T13:02:32.571992+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:42:53.469232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392437+0000","last_change":"2023-11-28T13:02:31.168319+0000","last_active":"2023-11-28T13:02:33.392437+0000","last_peered":"2023-11-28T13:02:33.392437+0000","last_clean":"2023-11-28T13:02:33.392437+0000","last_became_active":"2023-11-28T13:02:31.167709+0000","last_became_peered":"2023-11-28T13:02:31.167709+0000","last_unstale":"2023-11-28T13:02:33.392437+0000","last_undegraded":"2023-11-28T13:02:33.392437+0000","last_fullsized":"2023-11-28T13:02:33.392437+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:44:09.741137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.567424+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:02:31.567424+0000","last_peered":"2023-11-28T13:02:31.567424+0000","last_clean":"2023-11-28T13:02:31.567424+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:02:31.567424+0000","last_undegraded":"2023-11-28T13:02:31.567424+0000","last_fullsized":"2023-11-28T13:02:31.567424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390834+0000","last_change":"2023-11-28T13:02:31.591456+0000","last_active":"2023-11-28T13:02:33.390834+0000","last_peered":"2023-11-28T13:02:33.390834+0000","last_clean":"2023-11-28T13:02:33.390834+0000","last_became_active":"2023-11-28T13:02:31.584118+0000","last_became_peered":"2023-11-28T13:02:31.584118+0000","last_unstale":"2023-11-28T13:02:33.390834+0000","last_undegraded":"2023-11-28T13:02:33.390834+0000","last_fullsized":"2023-11-28T13:02:33.390834+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:04.890441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572533+0000","last_change":"2023-11-28T13:02:31.156754+0000","last_active":"2023-11-28T13:02:32.572533+0000","last_peered":"2023-11-28T13:02:32.572533+0000","last_clean":"2023-11-28T13:02:32.572533+0000","last_became_active":"2023-11-28T13:02:31.156415+0000","last_became_peered":"2023-11-28T13:02:31.156415+0000","last_unstale":"2023-11-28T13:02:32.572533+0000","last_undegraded":"2023-11-28T13:02:32.572533+0000","last_fullsized":"2023-11-28T13:02:32.572533+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:32:04.093914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":44,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.581970+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:02:31.581970+0000","last_peered":"2023-11-28T13:02:31.581970+0000","last_clean":"2023-11-28T13:02:31.581970+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:02:31.581970+0000","last_undegraded":"2023-11-28T13:02:31.581970+0000","last_fullsized":"2023-11-28T13:02:31.581970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392572+0000","last_change":"2023-11-28T13:02:31.590718+0000","last_active":"2023-11-28T13:02:33.392572+0000","last_peered":"2023-11-28T13:02:33.392572+0000","last_clean":"2023-11-28T13:02:33.392572+0000","last_became_active":"2023-11-28T13:02:31.590275+0000","last_became_peered":"2023-11-28T13:02:31.590275+0000","last_unstale":"2023-11-28T13:02:33.392572+0000","last_undegraded":"2023-11-28T13:02:33.392572+0000","last_fullsized":"2023-11-28T13:02:33.392572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:30:04.890441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.570275+0000","last_change":"2023-11-28T13:02:31.153173+0000","last_active":"2023-11-28T13:02:32.570275+0000","last_peered":"2023-11-28T13:02:32.570275+0000","last_clean":"2023-11-28T13:02:32.570275+0000","last_became_active":"2023-11-28T13:02:31.152787+0000","last_became_peered":"2023-11-28T13:02:31.152787+0000","last_unstale":"2023-11-28T13:02:32.570275+0000","last_undegraded":"2023-11-28T13:02:32.570275+0000","last_fullsized":"2023-11-28T13:02:32.570275+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:39.835807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.567895+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:02:31.567895+0000","last_peered":"2023-11-28T13:02:31.567895+0000","last_clean":"2023-11-28T13:02:31.567895+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:02:31.567895+0000","last_undegraded":"2023-11-28T13:02:31.567895+0000","last_fullsized":"2023-11-28T13:02:31.567895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":198,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.572883+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:02:31.572883+0000","last_peered":"2023-11-28T13:02:31.572883+0000","last_clean":"2023-11-28T13:02:31.572883+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:02:31.572883+0000","last_undegraded":"2023-11-28T13:02:31.572883+0000","last_fullsized":"2023-11-28T13:02:31.572883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.572573+0000","last_change":"2023-11-28T13:02:31.586394+0000","last_active":"2023-11-28T13:02:32.572573+0000","last_peered":"2023-11-28T13:02:32.572573+0000","last_clean":"2023-11-28T13:02:32.572573+0000","last_became_active":"2023-11-28T13:02:31.585743+0000","last_became_peered":"2023-11-28T13:02:31.585743+0000","last_unstale":"2023-11-28T13:02:32.572573+0000","last_undegraded":"2023-11-28T13:02:32.572573+0000","last_fullsized":"2023-11-28T13:02:32.572573+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:00.075913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391162+0000","last_change":"2023-11-28T13:02:31.160616+0000","last_active":"2023-11-28T13:02:33.391162+0000","last_peered":"2023-11-28T13:02:33.391162+0000","last_clean":"2023-11-28T13:02:33.391162+0000","last_became_active":"2023-11-28T13:02:31.159814+0000","last_became_peered":"2023-11-28T13:02:31.159814+0000","last_unstale":"2023-11-28T13:02:33.391162+0000","last_undegraded":"2023-11-28T13:02:33.391162+0000","last_fullsized":"2023-11-28T13:02:33.391162+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:43.400645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.572098+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:02:31.572098+0000","last_peered":"2023-11-28T13:02:31.572098+0000","last_clean":"2023-11-28T13:02:31.572098+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:02:31.572098+0000","last_undegraded":"2023-11-28T13:02:31.572098+0000","last_fullsized":"2023-11-28T13:02:31.572098+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.390545+0000","last_change":"2023-11-28T13:02:31.596874+0000","last_active":"2023-11-28T13:02:33.390545+0000","last_peered":"2023-11-28T13:02:33.390545+0000","last_clean":"2023-11-28T13:02:33.390545+0000","last_became_active":"2023-11-28T13:02:31.596620+0000","last_became_peered":"2023-11-28T13:02:31.596620+0000","last_unstale":"2023-11-28T13:02:33.390545+0000","last_undegraded":"2023-11-28T13:02:33.390545+0000","last_fullsized":"2023-11-28T13:02:33.390545+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:19:57.092839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.856253+0000","last_change":"2023-11-28T13:02:31.152084+0000","last_active":"2023-11-28T13:02:32.856253+0000","last_peered":"2023-11-28T13:02:32.856253+0000","last_clean":"2023-11-28T13:02:32.856253+0000","last_became_active":"2023-11-28T13:02:31.151769+0000","last_became_peered":"2023-11-28T13:02:31.151769+0000","last_unstale":"2023-11-28T13:02:32.856253+0000","last_undegraded":"2023-11-28T13:02:32.856253+0000","last_fullsized":"2023-11-28T13:02:32.856253+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:46.802595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.572778+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:02:31.572778+0000","last_peered":"2023-11-28T13:02:31.572778+0000","last_clean":"2023-11-28T13:02:31.572778+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:02:31.572778+0000","last_undegraded":"2023-11-28T13:02:31.572778+0000","last_fullsized":"2023-11-28T13:02:31.572778+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391715+0000","last_change":"2023-11-28T13:02:31.586237+0000","last_active":"2023-11-28T13:02:33.391715+0000","last_peered":"2023-11-28T13:02:33.391715+0000","last_clean":"2023-11-28T13:02:33.391715+0000","last_became_active":"2023-11-28T13:02:31.585914+0000","last_became_peered":"2023-11-28T13:02:31.585914+0000","last_unstale":"2023-11-28T13:02:33.391715+0000","last_undegraded":"2023-11-28T13:02:33.391715+0000","last_fullsized":"2023-11-28T13:02:33.391715+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:08:40.289844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571405+0000","last_change":"2023-11-28T13:02:31.164931+0000","last_active":"2023-11-28T13:02:32.571405+0000","last_peered":"2023-11-28T13:02:32.571405+0000","last_clean":"2023-11-28T13:02:32.571405+0000","last_became_active":"2023-11-28T13:02:31.164606+0000","last_became_peered":"2023-11-28T13:02:31.164606+0000","last_unstale":"2023-11-28T13:02:32.571405+0000","last_undegraded":"2023-11-28T13:02:32.571405+0000","last_fullsized":"2023-11-28T13:02:32.571405+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:00:03.898632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.576376+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:02:31.576376+0000","last_peered":"2023-11-28T13:02:31.576376+0000","last_clean":"2023-11-28T13:02:31.576376+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:02:31.576376+0000","last_undegraded":"2023-11-28T13:02:31.576376+0000","last_fullsized":"2023-11-28T13:02:31.576376+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571387+0000","last_change":"2023-11-28T13:02:31.585412+0000","last_active":"2023-11-28T13:02:32.571387+0000","last_peered":"2023-11-28T13:02:32.571387+0000","last_clean":"2023-11-28T13:02:32.571387+0000","last_became_active":"2023-11-28T13:02:31.585091+0000","last_became_peered":"2023-11-28T13:02:31.585091+0000","last_unstale":"2023-11-28T13:02:32.571387+0000","last_undegraded":"2023-11-28T13:02:32.571387+0000","last_fullsized":"2023-11-28T13:02:32.571387+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:09.625935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.391133+0000","last_change":"2023-11-28T13:02:31.159531+0000","last_active":"2023-11-28T13:02:33.391133+0000","last_peered":"2023-11-28T13:02:33.391133+0000","last_clean":"2023-11-28T13:02:33.391133+0000","last_became_active":"2023-11-28T13:02:31.159249+0000","last_became_peered":"2023-11-28T13:02:31.159249+0000","last_unstale":"2023-11-28T13:02:33.391133+0000","last_undegraded":"2023-11-28T13:02:33.391133+0000","last_fullsized":"2023-11-28T13:02:33.391133+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:23.856955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.576373+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:02:31.576373+0000","last_peered":"2023-11-28T13:02:31.576373+0000","last_clean":"2023-11-28T13:02:31.576373+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:02:31.576373+0000","last_undegraded":"2023-11-28T13:02:31.576373+0000","last_fullsized":"2023-11-28T13:02:31.576373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:32.571667+0000","last_change":"2023-11-28T13:02:31.586198+0000","last_active":"2023-11-28T13:02:32.571667+0000","last_peered":"2023-11-28T13:02:32.571667+0000","last_clean":"2023-11-28T13:02:32.571667+0000","last_became_active":"2023-11-28T13:02:31.585863+0000","last_became_peered":"2023-11-28T13:02:31.585863+0000","last_unstale":"2023-11-28T13:02:32.571667+0000","last_undegraded":"2023-11-28T13:02:32.571667+0000","last_fullsized":"2023-11-28T13:02:32.571667+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:31.132598+0000","last_clean_scrub_stamp":"2023-11-28T13:02:31.132598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:15.215786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T13:02:33.392321+0000","last_change":"2023-11-28T13:02:31.168641+0000","last_active":"2023-11-28T13:02:33.392321+0000","last_peered":"2023-11-28T13:02:33.392321+0000","last_clean":"2023-11-28T13:02:33.392321+0000","last_became_active":"2023-11-28T13:02:31.168363+0000","last_became_peered":"2023-11-28T13:02:31.168363+0000","last_unstale":"2023-11-28T13:02:33.392321+0000","last_undegraded":"2023-11-28T13:02:33.392321+0000","last_fullsized":"2023-11-28T13:02:33.392321+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:02:30.130628+0000","last_clean_scrub_stamp":"2023-11-28T13:02:30.130628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:43.400645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T13:02:31.576014+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:02:31.576014+0000","last_peered":"2023-11-28T13:02:31.576014+0000","last_clean":"2023-11-28T13:02:31.576014+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:02:31.576014+0000","last_undegraded":"2023-11-28T13:02:31.576014+0000","last_fullsized":"2023-11-28T13:02:31.576014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10376,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361464,"statfs":{"total":96636764160,"available":96626139136,"internally_reserved":0,"allocated":1245184,"data_stored":889426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.45700000000000002,"15min":0.442},"min":{"1min":0.32200000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69899999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.495}]},{"osd":1,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.36499999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.432}]},{"osd":2,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.40799999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64800000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51800000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77100000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56799999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65200000000000002,"15min":0.65500000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63200000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.46500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.79000000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85299999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9148,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362692,"statfs":{"total":96636764160,"available":96627396608,"internally_reserved":0,"allocated":643072,"data_stored":298898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.33900000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.49399999999999999,"15min":0.48199999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58299999999999996}]},{"osd":1,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.629},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72799999999999998,"5min":0.87,"15min":0.87},"last":0.60399999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.71399999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.54100000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":1.2829999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.38900000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.74299999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.626},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.77100000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.755}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10240,"kb_used_data":1208,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361600,"statfs":{"total":96636764160,"available":96626278400,"internally_reserved":0,"allocated":1236992,"data_stored":888351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.313,"15min":0.309},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56799999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.373}]},{"osd":2,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.307},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38200000000000001,"15min":0.38},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.29099999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.32700000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.82899999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.71999999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.47299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":1.157,"15min":1.157},"last":0.51100000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86799999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.495},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9352,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362488,"statfs":{"total":96636764160,"available":96627187712,"internally_reserved":0,"allocated":655360,"data_stored":300026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.37,"5min":0.34499999999999997,"15min":0.34100000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.55300000000000005}]},{"osd":2,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.49299999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.33800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42499999999999999,"15min":0.42799999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.62}]},{"osd":4,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80100000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.90200000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91900000000000004,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.877}]},{"osd":7,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.75600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9156,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362684,"statfs":{"total":96636764160,"available":96627388416,"internally_reserved":0,"allocated":651264,"data_stored":298954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.438,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36599999999999999,"15min":0.36699999999999999},"min":{"1min":0.27700000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.372}]},{"osd":1,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51400000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38100000000000001},"min":{"1min":0.27400000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.56100000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.42399999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.371,"15min":0.36899999999999999},"min":{"1min":0.27400000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52000000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.313},{"interface":"front","average":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.61599999999999999,"5min":0.748,"15min":0.748},"last":0.35299999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73299999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.499,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79300000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.621,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84099999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.39500000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81599999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79600000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.49099999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86699999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.64100000000000001,"15min":0.64300000000000002},"min":{"1min":0.48199999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.46800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9284,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362556,"statfs":{"total":96636764160,"available":96627257344,"internally_reserved":0,"allocated":651264,"data_stored":298954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.504,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.47699999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.255,"5min":0.246,"15min":0.246},"max":{"1min":0.45200000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39800000000000002,"15min":0.40400000000000003},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58399999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54600000000000004}]},{"osd":2,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.56399999999999995,"15min":0.59599999999999997},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54700000000000004,"5min":5.0540000000000003,"15min":5.0540000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.45200000000000001,"15min":0.46300000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.53700000000000003,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85799999999999998,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.626,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84399999999999997,"5min":1.2,"15min":1.2},"last":0.97099999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":1.169,"15min":1.169},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81999999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.89500000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81499999999999995,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":1.23,"15min":1.23},"last":1.01}]},{"osd":7,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77800000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85199999999999998,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.018}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9352,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362488,"statfs":{"total":96636764160,"available":96627187712,"internally_reserved":0,"allocated":655360,"data_stored":300446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.45600000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.41999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.66300000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.495,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.49099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68400000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73799999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.75600000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81499999999999995,"5min":1.117,"15min":1.117},"last":0.80700000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.78400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9156,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362684,"statfs":{"total":96636764160,"available":96627388416,"internally_reserved":0,"allocated":651264,"data_stored":299534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71899999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.34899999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.378}]},{"osd":2,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.435},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55000000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60999999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.625,"15min":0.64000000000000001},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.64000000000000001,"15min":0.66000000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75700000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69499999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.67900000000000005,"15min":0.69699999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.751,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66400000000000003,"15min":0.67700000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83299999999999996,"5min":1.052,"15min":1.052},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:02:36.449 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:02:36.450 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T13:02:36.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.616+0000 7f4d2487b700 1 Processor -- start 2023-11-28T13:02:36.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.616+0000 7f4d2487b700 1 -- start start 2023-11-28T13:02:36.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.617+0000 7f4d2487b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4d200b62d0 0x7f4d200cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.617+0000 7f4d2487b700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d200cd2a0 con 0x7f4d200b2d90 2023-11-28T13:02:36.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.617+0000 7f4d2487b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d200d0400 con 0x7f4d2006f8b0 2023-11-28T13:02:36.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.617+0000 7f4d2487b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d200d06e0 con 0x7f4d200b62d0 2023-11-28T13:02:36.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.617+0000 7f4d1ed9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f4d200b2d90 0x7f4d200ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48852/0 (socket says 172.21.15.6:48852) 2023-11-28T13:02:36.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.617+0000 7f4d1ed9d700 1 -- 172.21.15.6:0/29987786 learned_addr learned my addr 172.21.15.6:0/29987786 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:36.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1e59c700 1 --2- 172.21.15.6:0/29987786 >> v2:172.21.15.61:3300/0 conn(0x7f4d200b62d0 0x7f4d200cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333485767 0 0) 0x7f4d200cd2a0 con 0x7f4d200b2d90 2023-11-28T13:02:36.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d080035b0 con 0x7f4d200b2d90 2023-11-28T13:02:36.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 453639798 0 0) 0x7f4d080035b0 con 0x7f4d200b2d90 2023-11-28T13:02:36.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 >> v2:172.21.15.61:3300/0 conn(0x7f4d200b62d0 msgr2=0x7f4d200cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:36.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 --2- 172.21.15.6:0/29987786 >> v2:172.21.15.61:3300/0 conn(0x7f4d200b62d0 0x7f4d200cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 >> v1:172.21.15.61:6789/0 conn(0x7f4d2006f8b0 legacy=0x7f4d200b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d200cffb0 con 0x7f4d200b2d90 2023-11-28T13:02:36.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f4d180019f0 con 0x7f4d200b2d90 2023-11-28T13:02:36.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 >> v1:172.21.15.6:6789/0 conn(0x7f4d200b2d90 legacy=0x7f4d200ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d080043f0 0x7f4d080048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d080050b0 0x7f4d0800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d200d06e0 con 0x7f4d08003a00 2023-11-28T13:02:36.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0800c000 con 0x7f4d080043f0 2023-11-28T13:02:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0800c1c0 con 0x7f4d080050b0 2023-11-28T13:02:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1e59c700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1dd9b700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d080043f0 0x7f4d080048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1ed9d700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d080050b0 0x7f4d0800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1e59c700 1 -- 172.21.15.6:0/29987786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d080050b0 msgr2=0x7f4d0800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1e59c700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d080050b0 0x7f4d0800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1e59c700 1 -- 172.21.15.6:0/29987786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d080043f0 msgr2=0x7f4d080048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1e59c700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d080043f0 0x7f4d080048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1e59c700 1 -- 172.21.15.6:0/29987786 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d080035b0 con 0x7f4d08003a00 2023-11-28T13:02:36.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1ed9d700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d080050b0 0x7f4d0800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:36.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.618+0000 7f4d1ed9d700 1 -- 172.21.15.6:0/29987786 reap_dead start 2023-11-28T13:02:36.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d1e59c700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d08003dd0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009b60 tx=0x7f4d08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45114d13d173d7e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400e6c0 con 0x7f4d08003a00 2023-11-28T13:02:36.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1400e820 con 0x7f4d08003a00 2023-11-28T13:02:36.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d1d59a700 1 -- 172.21.15.6:0/29987786 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400eb90 con 0x7f4d08003a00 2023-11-28T13:02:36.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d2487b700 1 -- 172.21.15.6:0/29987786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 msgr2=0x7f4d08003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d2487b700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d08003dd0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009b60 tx=0x7f4d08005880 comp rx=0 tx=0).stop 2023-11-28T13:02:36.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d2487b700 1 -- 172.21.15.6:0/29987786 shutdown_connections 2023-11-28T13:02:36.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d2487b700 1 --2- 172.21.15.6:0/29987786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d08003dd0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d2487b700 1 -- 172.21.15.6:0/29987786 >> 172.21.15.6:0/29987786 conn(0x7f4d200cd680 msgr2=0x7f4d200cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:36.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d2487b700 1 -- 172.21.15.6:0/29987786 shutdown_connections 2023-11-28T13:02:36.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.619+0000 7f4d2487b700 1 -- 172.21.15.6:0/29987786 wait complete. 2023-11-28T13:02:36.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.621+0000 7f4d2487b700 1 Processor -- start 2023-11-28T13:02:36.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.621+0000 7f4d2487b700 1 -- start start 2023-11-28T13:02:36.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.621+0000 7f4d2487b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d200ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.631 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d2487b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d201581b0 0x7f4d2014d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d2487b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d200c5b40 0x7f4d200c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d2487b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2014d580 con 0x7f4d08003a00 2023-11-28T13:02:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d2487b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d200c4aa0 con 0x7f4d200c5b40 2023-11-28T13:02:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d2487b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d200c4d80 con 0x7f4d201581b0 2023-11-28T13:02:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d200ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d200ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33308/0 (socket says 172.21.15.6:33308) 2023-11-28T13:02:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1e59c700 1 -- 172.21.15.6:0/4246333751 learned_addr learned my addr 172.21.15.6:0/4246333751 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1ed9d700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d200c5b40 0x7f4d200c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1dd9b700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d201581b0 0x7f4d2014d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1e59c700 1 -- 172.21.15.6:0/4246333751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d201581b0 msgr2=0x7f4d2014d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1e59c700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d201581b0 0x7f4d2014d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1e59c700 1 -- 172.21.15.6:0/4246333751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d200c5b40 msgr2=0x7f4d200c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1e59c700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d200c5b40 0x7f4d200c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1e59c700 1 -- 172.21.15.6:0/4246333751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d200b66a0 con 0x7f4d08003a00 2023-11-28T13:02:36.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d1e59c700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d200ca780 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4d08005810 tx=0x7f4d140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d0effd700 1 -- 172.21.15.6:0/4246333751 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400e6c0 con 0x7f4d08003a00 2023-11-28T13:02:36.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d0effd700 1 -- 172.21.15.6:0/4246333751 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d14003ad0 con 0x7f4d08003a00 2023-11-28T13:02:36.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d2487b700 1 -- 172.21.15.6:0/4246333751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2006fd30 con 0x7f4d08003a00 2023-11-28T13:02:36.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d0effd700 1 -- 172.21.15.6:0/4246333751 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14003e80 con 0x7f4d08003a00 2023-11-28T13:02:36.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.622+0000 7f4d2487b700 1 -- 172.21.15.6:0/4246333751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d200cce50 con 0x7f4d08003a00 2023-11-28T13:02:36.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.623+0000 7f4d2487b700 1 -- 172.21.15.6:0/4246333751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d200739f0 con 0x7f4d08003a00 2023-11-28T13:02:36.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.624+0000 7f4d0effd700 1 -- 172.21.15.6:0/4246333751 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d14019030 con 0x7f4d08003a00 2023-11-28T13:02:36.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.624+0000 7f4d0effd700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 0x7f4d0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:36.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.624+0000 7f4d0effd700 1 -- 172.21.15.6:0/4246333751 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4d14087c30 con 0x7f4d08003a00 2023-11-28T13:02:36.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.625+0000 7f4d1dd9b700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 0x7f4d0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:36.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.625+0000 7f4d1dd9b700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 0x7f4d0006a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4d200c6610 tx=0x7f4d10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:36.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.628+0000 7f4d0effd700 1 -- 172.21.15.6:0/4246333751 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d140543f0 con 0x7f4d08003a00 2023-11-28T13:02:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.901+0000 7f4d2487b700 1 -- 172.21.15.6:0/4246333751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f4d2014d580 con 0x7f4d08003a00 2023-11-28T13:02:36.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.905+0000 7f4d0effd700 1 -- 172.21.15.6:0/4246333751 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f4d140170e0 con 0x7f4d08003a00 2023-11-28T13:02:36.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 -- 172.21.15.6:0/4246333751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 msgr2=0x7f4d0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 0x7f4d0006a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4d200c6610 tx=0x7f4d10009040 comp rx=0 tx=0).stop 2023-11-28T13:02:36.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 -- 172.21.15.6:0/4246333751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 msgr2=0x7f4d200ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:36.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d200ca780 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4d08005810 tx=0x7f4d140047c0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 -- 172.21.15.6:0/4246333751 shutdown_connections 2023-11-28T13:02:36.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4d200c5b40 0x7f4d200c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d08003a00 0x7f4d200ca780 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4d000682f0 0x7f4d0006a7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 --2- 172.21.15.6:0/4246333751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4d201581b0 0x7f4d2014d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:36.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 -- 172.21.15.6:0/4246333751 >> 172.21.15.6:0/4246333751 conn(0x7f4d200cd680 msgr2=0x7f4d200b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:36.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 -- 172.21.15.6:0/4246333751 shutdown_connections 2023-11-28T13:02:36.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:36.907+0000 7f4d2487b700 1 -- 172.21.15.6:0/4246333751 wait complete. 2023-11-28T13:02:36.910 INFO:teuthology.orchestra.run.smithi006.stderr:updated caps for client.0 2023-11-28T13:02:36.921 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:37.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.044+0000 7fa1f447a700 1 Processor -- start 2023-11-28T13:02:37.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.044+0000 7fa1f447a700 1 -- start start 2023-11-28T13:02:37.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.045+0000 7fa1f447a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa1ec0cefa0 0x7fa1ec15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.045+0000 7fa1f447a700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec15e0b0 con 0x7fa1ec159590 2023-11-28T13:02:37.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.045+0000 7fa1f447a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec15d120 con 0x7fa1ec06e400 2023-11-28T13:02:37.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.045+0000 7fa1f447a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ec15d6e0 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.045+0000 7fa1f2216700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fa1ec159590 0x7fa1ec0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48868/0 (socket says 172.21.15.6:48868) 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.045+0000 7fa1f2216700 1 -- 172.21.15.6:0/4132211810 learned_addr learned my addr 172.21.15.6:0/4132211810 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.045+0000 7fa1f2a17700 1 --2- 172.21.15.6:0/4132211810 >> v2:172.21.15.61:3300/0 conn(0x7fa1ec0cefa0 0x7fa1ec15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.046+0000 7fa1f1214700 1 -- 172.21.15.6:0/4132211810 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093170344 0 0) 0x7fa1dc001110 con 0x7fa1ec159590 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.046+0000 7fa1f1214700 1 -- 172.21.15.6:0/4132211810 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d80035b0 con 0x7fa1ec159590 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.046+0000 7fa1f2a17700 1 -- 172.21.15.6:0/4132211810 >> v1:172.21.15.61:6789/0 conn(0x7fa1ec06e400 legacy=0x7fa1ec0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.046+0000 7fa1f2a17700 1 -- 172.21.15.6:0/4132211810 >> v1:172.21.15.6:6789/0 conn(0x7fa1ec159590 legacy=0x7fa1ec0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.046+0000 7fa1f2a17700 1 -- 172.21.15.6:0/4132211810 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec0aef60 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.046+0000 7fa1f2a17700 1 --2- 172.21.15.6:0/4132211810 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1ec0cefa0 0x7fa1ec15db70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009d90 tx=0x7fa1e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e63f38b2e858c1f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.046+0000 7fa1f1214700 1 -- 172.21.15.6:0/4132211810 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e400e070 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.046+0000 7fa1f1214700 1 -- 172.21.15.6:0/4132211810 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e400bb20 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.046+0000 7fa1f1214700 1 -- 172.21.15.6:0/4132211810 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4012430 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.047+0000 7fa1f447a700 1 -- 172.21.15.6:0/4132211810 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1ec0cefa0 msgr2=0x7fa1ec15db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.047+0000 7fa1f447a700 1 --2- 172.21.15.6:0/4132211810 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1ec0cefa0 0x7fa1ec15db70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009d90 tx=0x7fa1e400b330 comp rx=0 tx=0).stop 2023-11-28T13:02:37.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.047+0000 7fa1f447a700 1 -- 172.21.15.6:0/4132211810 shutdown_connections 2023-11-28T13:02:37.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.047+0000 7fa1f447a700 1 --2- 172.21.15.6:0/4132211810 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1ec0cefa0 0x7fa1ec15db70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.047+0000 7fa1f447a700 1 -- 172.21.15.6:0/4132211810 >> 172.21.15.6:0/4132211810 conn(0x7fa1ec0b1650 msgr2=0x7fa1ec0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.047+0000 7fa1f447a700 1 -- 172.21.15.6:0/4132211810 shutdown_connections 2023-11-28T13:02:37.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.047+0000 7fa1f447a700 1 -- 172.21.15.6:0/4132211810 wait complete. 2023-11-28T13:02:37.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.048+0000 7fa1f447a700 1 Processor -- start 2023-11-28T13:02:37.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.048+0000 7fa1f447a700 1 -- start start 2023-11-28T13:02:37.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f447a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1ec06e400 0x7fa1ec1518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f447a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ec0cefa0 0x7fa1ec0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f447a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1ec159590 0x7fa1ec0c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f447a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec1523e0 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.052 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f447a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec0b3010 con 0x7fa1ec159590 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f447a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec0b2d30 con 0x7fa1ec06e400 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f1a15700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ec0cefa0 0x7fa1ec0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f1a15700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ec0cefa0 0x7fa1ec0b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33324/0 (socket says 172.21.15.6:33324) 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f1a15700 1 -- 172.21.15.6:0/3902703172 learned_addr learned my addr 172.21.15.6:0/3902703172 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f2a17700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1ec159590 0x7fa1ec0c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f2216700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1ec06e400 0x7fa1ec1518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f1a15700 1 -- 172.21.15.6:0/3902703172 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1ec06e400 msgr2=0x7fa1ec1518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f1a15700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1ec06e400 0x7fa1ec1518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f1a15700 1 -- 172.21.15.6:0/3902703172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1ec159590 msgr2=0x7fa1ec0c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f1a15700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1ec159590 0x7fa1ec0c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f1a15700 1 -- 172.21.15.6:0/3902703172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e4009a70 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1f1a15700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ec0cefa0 0x7fa1ec0b5330 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c4e0 tx=0x7fa1e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.049+0000 7fa1e2ffd700 1 -- 172.21.15.6:0/3902703172 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80052d0 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.050+0000 7fa1e2ffd700 1 -- 172.21.15.6:0/3902703172 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e8005430 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.050+0000 7fa1f447a700 1 -- 172.21.15.6:0/3902703172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec15d370 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.050+0000 7fa1e2ffd700 1 -- 172.21.15.6:0/3902703172 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80038c0 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.050+0000 7fa1f447a700 1 -- 172.21.15.6:0/3902703172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec0af240 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.050+0000 7fa1f447a700 1 -- 172.21.15.6:0/3902703172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec159960 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.051+0000 7fa1e2ffd700 1 -- 172.21.15.6:0/3902703172 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa1e8019030 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.051+0000 7fa1e2ffd700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1d00682f0 0x7fa1d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.051+0000 7fa1e2ffd700 1 -- 172.21.15.6:0/3902703172 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa1e8085030 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.054 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.051+0000 7fa1f2216700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1d00682f0 0x7fa1d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.052+0000 7fa1f2216700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1d00682f0 0x7fa1d006a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc007fd0 tx=0x7fa1dc007c40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.056+0000 7fa1e2ffd700 1 -- 172.21.15.6:0/3902703172 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa1e8051750 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.130 INFO:tasks.ceph.mds.a.smithi006.stderr:2023-11-28T13:02:37.129+0000 7fae1385f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:02:37.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.291+0000 7fa1f447a700 1 -- 172.21.15.6:0/3902703172 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1ec0675a0 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.292+0000 7fa1e2ffd700 1 -- 172.21.15.6:0/3902703172 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4636 (secure 0 0 0) 0x7fa1e8017090 con 0x7fa1ec0cefa0 2023-11-28T13:02:37.293 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:37.293 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4004512137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4004512137},{"type":"v1","addr":"172.21.15.61:6833","nonce":4004512137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/1313775424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":1313775424},{"type":"v1","addr":"172.21.15.61:6835","nonce":1313775424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/233781200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":233781200},{"type":"v1","addr":"172.21.15.6:6836","nonce":233781200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:31.562404+0000","modified":"2023-11-28T13:02:34.939442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/451011578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":451011578},{"type":"v1","addr":"172.21.15.6:6837","nonce":451011578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:37.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 -- 172.21.15.6:0/3902703172 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1d00682f0 msgr2=0x7fa1d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1d00682f0 0x7fa1d006a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc007fd0 tx=0x7fa1dc007c40 comp rx=0 tx=0).stop 2023-11-28T13:02:37.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 -- 172.21.15.6:0/3902703172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ec0cefa0 msgr2=0x7fa1ec0b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ec0cefa0 0x7fa1ec0b5330 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800c4e0 tx=0x7fa1e800c7f0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 -- 172.21.15.6:0/3902703172 shutdown_connections 2023-11-28T13:02:37.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1ec159590 0x7fa1ec0c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1d00682f0 0x7fa1d006a7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1ec06e400 0x7fa1ec1518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 --2- 172.21.15.6:0/3902703172 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1ec0cefa0 0x7fa1ec0b5330 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 -- 172.21.15.6:0/3902703172 >> 172.21.15.6:0/3902703172 conn(0x7fa1ec0b1650 msgr2=0x7fa1ec154530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 -- 172.21.15.6:0/3902703172 shutdown_connections 2023-11-28T13:02:37.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.294+0000 7fa1f447a700 1 -- 172.21.15.6:0/3902703172 wait complete. 2023-11-28T13:02:37.297 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2023-11-28T13:02:37.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:02:31.562404+0000', 'modified': '2023-11-28T13:02:34.939442+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6003}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6003': {'gid': 6003, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6837/451011578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6835', 'nonce': 451011578}, {'type': 'v1', 'addr': '172.21.15.6:6837', 'nonce': 451011578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:02:37.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:02:37.710 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T13:02:37.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.845+0000 7f30ff5e0700 1 Processor -- start 2023-11-28T13:02:37.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.846+0000 7f30ff5e0700 1 -- start start 2023-11-28T13:02:37.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.846+0000 7f30ff5e0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815a6c0 0x7f30f80cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.846+0000 7f30ff5e0700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f80cfb90 con 0x7f30f80b2660 2023-11-28T13:02:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.846+0000 7f30ff5e0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f80cb930 con 0x7f30f80d0020 2023-11-28T13:02:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.846+0000 7f30ff5e0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80cf260 con 0x7f30f815a6c0 2023-11-28T13:02:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30fdb7d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f30f80b2660 0x7f30f80b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:48884/0 (socket says 172.21.15.6:48884) 2023-11-28T13:02:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30fdb7d700 1 -- 172.21.15.6:0/3324155133 learned_addr learned my addr 172.21.15.6:0/3324155133 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30fd37c700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815a6c0 0x7f30f80cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627287869 0 0) 0x7f30f80cfb90 con 0x7f30f80b2660 2023-11-28T13:02:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e40035b0 con 0x7f30f80b2660 2023-11-28T13:02:37.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451532712 0 0) 0x7f30f80cb930 con 0x7f30f80d0020 2023-11-28T13:02:37.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f80cfb90 con 0x7f30f80d0020 2023-11-28T13:02:37.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2808618562 0 0) 0x7f30e40035b0 con 0x7f30f80b2660 2023-11-28T13:02:37.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815a6c0 msgr2=0x7f30f80cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815a6c0 0x7f30f80cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 >> v1:172.21.15.61:6789/0 conn(0x7f30f80d0020 legacy=0x7f30f80ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f80cf540 con 0x7f30f80b2660 2023-11-28T13:02:37.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f30f00019f0 con 0x7f30f80b2660 2023-11-28T13:02:37.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30fd37c700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815a6c0 0x7f30f80cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:37.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 >> v1:172.21.15.6:6789/0 conn(0x7f30f80b2660 legacy=0x7f30f80b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815a6c0 0x7f30e4003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f30e4004150 0x7f30e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30e4004d90 0x7f30e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80cf260 con 0x7f30e4004d90 2023-11-28T13:02:37.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e400baf0 con 0x7f30e4004150 2023-11-28T13:02:37.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.847+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e400bcb0 con 0x7f30f815a6c0 2023-11-28T13:02:37.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30e4004d90 0x7f30e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30fd37c700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815a6c0 0x7f30e4003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30fcb7b700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f30e4004150 0x7f30e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30fdb7d700 1 -- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815a6c0 msgr2=0x7f30e4003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815a6c0 0x7f30e4003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30fdb7d700 1 -- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f30e4004150 msgr2=0x7f30e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f30e4004150 0x7f30e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30fdb7d700 1 -- 172.21.15.6:0/3324155133 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40094a0 con 0x7f30e4004d90 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30e4004d90 0x7f30e4005200 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f30f000a6e0 tx=0x7f30e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d23b69852d848c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f00072d0 con 0x7f30e4004d90 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f0007430 con 0x7f30e4004d90 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.848+0000 7f30ebfff700 1 -- 172.21.15.6:0/3324155133 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f00077c0 con 0x7f30e4004d90 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.849+0000 7f30ff5e0700 1 -- 172.21.15.6:0/3324155133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30e4004d90 msgr2=0x7f30e4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.849+0000 7f30ff5e0700 1 --2- 172.21.15.6:0/3324155133 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30e4004d90 0x7f30e4005200 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f30f000a6e0 tx=0x7f30e4005880 comp rx=0 tx=0).stop 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.849+0000 7f30fdb7d700 1 -- 172.21.15.6:0/3324155133 reap_dead start 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.849+0000 7f30ff5e0700 1 -- 172.21.15.6:0/3324155133 shutdown_connections 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.849+0000 7f30ff5e0700 1 -- 172.21.15.6:0/3324155133 >> 172.21.15.6:0/3324155133 conn(0x7f30f80cd660 msgr2=0x7f30f80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.849+0000 7f30ff5e0700 1 -- 172.21.15.6:0/3324155133 shutdown_connections 2023-11-28T13:02:37.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.849+0000 7f30ff5e0700 1 -- 172.21.15.6:0/3324155133 wait complete. 2023-11-28T13:02:37.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.850+0000 7f30ff5e0700 1 Processor -- start 2023-11-28T13:02:37.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.850+0000 7f30ff5e0700 1 -- start start 2023-11-28T13:02:37.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30ff5e0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f30f81ef670 0x7f30f81efa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30ff5e0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815bae0 0x7f30f815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30ff5e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30f80b1680 0x7f30f80b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30ff5e0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f815cc60 con 0x7f30f80b1680 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30ff5e0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f815c490 con 0x7f30f81ef670 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30ff5e0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80b2030 con 0x7f30f815bae0 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fdb7d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30f80b1680 0x7f30f80b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fdb7d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30f80b1680 0x7f30f80b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33346/0 (socket says 172.21.15.6:33346) 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4214777470 learned_addr learned my addr 172.21.15.6:0/4214777470 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fcb7b700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815bae0 0x7f30f815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fd37c700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f30f81ef670 0x7f30f81efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4214777470 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815bae0 msgr2=0x7f30f815bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815bae0 0x7f30f815bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4214777470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f30f81ef670 msgr2=0x7f30f81efa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f30f81ef670 0x7f30f81efa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4214777470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f80d0470 con 0x7f30f80b1680 2023-11-28T13:02:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30f80b1680 0x7f30f80b1af0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f30e400bac0 tx=0x7f30f0001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30e9ffb700 1 -- 172.21.15.6:0/4214777470 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f00011c0 con 0x7f30f80b1680 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30e9ffb700 1 -- 172.21.15.6:0/4214777470 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f0012400 con 0x7f30f80b1680 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.851+0000 7f30ff5e0700 1 -- 172.21.15.6:0/4214777470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f80cf820 con 0x7f30f80b1680 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.852+0000 7f30e9ffb700 1 -- 172.21.15.6:0/4214777470 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0012770 con 0x7f30f80b1680 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.852+0000 7f30ff5e0700 1 -- 172.21.15.6:0/4214777470 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f80d0600 con 0x7f30f80b1680 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.853+0000 7f30e9ffb700 1 -- 172.21.15.6:0/4214777470 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f30f0010070 con 0x7f30f80b1680 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.854+0000 7f30e9ffb700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30dc068320 0x7f30dc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.854+0000 7f30e9ffb700 1 -- 172.21.15.6:0/4214777470 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f30f0087760 con 0x7f30f80b1680 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.854+0000 7f30fd37c700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30dc068320 0x7f30dc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.854+0000 7f30fd37c700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30dc068320 0x7f30dc06a7d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009de0 tx=0x7f30f4009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f310b769b80 1 Processor -- start 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f310b769b80 1 -- start start 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f310b769b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5caebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f310b769b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5caf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f310b769b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5ca7ce0 0x55c8f5ca8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f310b769b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5b6ebc0 con 0x55c8f5cae810 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f310b769b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5bdd350 con 0x55c8f5c831c0 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f310b769b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5bd6880 con 0x55c8f5ca7ce0 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f30fd37c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5caebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f30fd37c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5caebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33350/0 (socket says 172.21.15.6:33350) 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.856+0000 7f30fd37c700 1 -- 172.21.15.6:0/443838197 learned_addr learned my addr 172.21.15.6:0/443838197 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30fcb7b700 1 --2- 172.21.15.6:0/443838197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5caf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/443838197 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5ca7ce0 0x55c8f5ca8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30fd37c700 1 -- 172.21.15.6:0/443838197 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5ca7ce0 msgr2=0x55c8f5ca8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30fd37c700 1 --2- 172.21.15.6:0/443838197 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5ca7ce0 0x55c8f5ca8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30fd37c700 1 -- 172.21.15.6:0/443838197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 msgr2=0x55c8f5caf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30fd37c700 1 --2- 172.21.15.6:0/443838197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5caf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30fd37c700 1 -- 172.21.15.6:0/443838197 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8f5a585e0 con 0x55c8f5cae810 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30fd37c700 1 --2- 172.21.15.6:0/443838197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5caebe0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f30f4004500 tx=0x7f30f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e962c5850451e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30e2ffd700 1 -- 172.21.15.6:0/443838197 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f40045b0 con 0x55c8f5cae810 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30e2ffd700 1 -- 172.21.15.6:0/443838197 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f40038b0 con 0x55c8f5cae810 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.857+0000 7f30e2ffd700 1 -- 172.21.15.6:0/443838197 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4003c40 con 0x55c8f5cae810 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 -- 172.21.15.6:0/443838197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 msgr2=0x55c8f5caebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 --2- 172.21.15.6:0/443838197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5caebe0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f30f4004500 tx=0x7f30f4017000 comp rx=0 tx=0).stop 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 -- 172.21.15.6:0/443838197 shutdown_connections 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 --2- 172.21.15.6:0/443838197 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5ca7ce0 0x55c8f5ca8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 --2- 172.21.15.6:0/443838197 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5caebe0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 --2- 172.21.15.6:0/443838197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5caf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 -- 172.21.15.6:0/443838197 >> 172.21.15.6:0/443838197 conn(0x55c8f5c81d60 msgr2=0x55c8f5c826f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 -- 172.21.15.6:0/443838197 shutdown_connections 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 -- 172.21.15.6:0/443838197 wait complete. 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 -- start start 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.858+0000 7f310b769b80 1 Processor -- start 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.859+0000 7f310b769b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5ca4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.859+0000 7f310b769b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5ca7ce0 0x55c8f5c9f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.859+0000 7f310b769b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5ccb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.859+0000 7f310b769b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5a5a8d0 con 0x55c8f5cae810 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.859+0000 7f310b769b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5cc0fd0 con 0x55c8f5c831c0 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.859+0000 7f310b769b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5ca04c0 con 0x55c8f5ca7ce0 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.859+0000 7f310b769b80 10 client.0 _ll_register_callbacks cb 0x55c8f5a7c240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fdb7d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5ccb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fdb7d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5ccb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33360/0 (socket says 172.21.15.6:33360) 2023-11-28T13:02:37.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4031008725 learned_addr learned my addr 172.21.15.6:0/4031008725 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fd37c700 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5ca4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fcb7b700 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5ca7ce0 0x55c8f5c9f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4031008725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5ca7ce0 msgr2=0x55c8f5c9f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5ca7ce0 0x55c8f5c9f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4031008725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 msgr2=0x55c8f5ca4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5ca4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4031008725 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f40041e0 con 0x55c8f5cae810 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5ccb1f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f30f0016040 tx=0x7f30f000a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423b633e08f4b475 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30d7fff700 1 -- 172.21.15.6:0/4031008725 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f005f850 con 0x55c8f5cae810 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30d7fff700 1 -- 172.21.15.6:0/4031008725 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f30f006c030 con 0x55c8f5cae810 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f310b769b80 1 -- 172.21.15.6:0/4031008725 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8f5ba4140 con 0x55c8f5cae810 2023-11-28T13:02:37.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30d7fff700 1 -- 172.21.15.6:0/4031008725 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30f005fae0 con 0x55c8f5cae810 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f310b769b80 10 client.6165 fetch_fsmap learned FSMap version 24 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f310b769b80 1 -- 172.21.15.6:0/4031008725 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55c8f5a04d90 con 0x55c8f5cae810 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.860+0000 7f30d7fff700 1 -- 172.21.15.6:0/4031008725 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8f5ba4140 con 0x55c8f5cae810 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.861+0000 7f30d7fff700 1 -- 172.21.15.6:0/4031008725 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f30f0053840 con 0x55c8f5cae810 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.861+0000 7f310b769b80 10 client.6165 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.861+0000 7f310b769b80 10 client.6165 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.861+0000 7f310b769b80 20 client.6165 populate_metadata read hostname 'smithi006' 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.861+0000 7f310b769b80 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55c8f5ca3c40 0x55c8f5ca4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.861+0000 7f310b769b80 4 client.6165 mds_command: new command op to 6003 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.861+0000 7f310b769b80 1 -- 172.21.15.6:0/4031008725 --> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8f5a5af10 con 0x55c8f5ca3c40 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.861+0000 7f30fd37c700 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55c8f5ca3c40 0x55c8f5ca4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.861+0000 7f30d7fff700 10 client.6165 ms_handle_connect on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.861+0000 7f30fd37c700 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55c8f5ca3c40 0x55c8f5ca4010 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19996ee6ea46fce server_cookie=762472434d035371 in_seq=0 out_seq=0 2023-11-28T13:02:37.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f30d7fff700 1 -- 172.21.15.6:0/4031008725 <== mds.0 v2:172.21.15.6:6835/451011578 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55c8f5a5af10 con 0x55c8f5ca3c40 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f30d7fff700 10 client.6165 handle_command_reply: tid=0 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 client.6165 shutdown 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 -- 172.21.15.6:0/4031008725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 msgr2=0x55c8f5ccb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5ccb1f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f30f0016040 tx=0x7f30f000a470 comp rx=0 tx=0).stop 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 -- 172.21.15.6:0/4031008725 shutdown_connections 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55c8f5ca3c40 0x55c8f5ca4010 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5ccb1f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5ca4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 --2- 172.21.15.6:0/4031008725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5ca7ce0 0x55c8f5c9f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 -- 172.21.15.6:0/4031008725 >> 172.21.15.6:0/4031008725 conn(0x55c8f5c81d60 msgr2=0x55c8f5c826f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f30d7fff700 0 client.6165 ms_handle_reset on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 -- 172.21.15.6:0/4031008725 shutdown_connections 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 1 -- 172.21.15.6:0/4031008725 wait complete. 2023-11-28T13:02:37.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.862+0000 7f310b769b80 20 client.6165 trim_cache size 0 max 16384 2023-11-28T13:02:37.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.880+0000 7f310b769b80 1 Processor -- start 2023-11-28T13:02:37.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.880+0000 7f310b769b80 1 -- start start 2023-11-28T13:02:37.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.880+0000 7f310b769b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5ca4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.880+0000 7f310b769b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5ca7ce0 0x55c8f5c82650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.880+0000 7f310b769b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5cae810 0x55c8f5c82b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.880+0000 7f310b769b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5c7de10 con 0x55c8f5ca7ce0 2023-11-28T13:02:37.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.880+0000 7f310b769b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5cc9a10 con 0x55c8f5c831c0 2023-11-28T13:02:37.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.880+0000 7f310b769b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5ca1630 con 0x55c8f5cae810 2023-11-28T13:02:37.884 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30fcb7b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5ca7ce0 0x55c8f5c82650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30fcb7b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5ca7ce0 0x55c8f5c82650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33366/0 (socket says 172.21.15.6:33366) 2023-11-28T13:02:37.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30fcb7b700 1 -- 172.21.15.6:0/2308390685 learned_addr learned my addr 172.21.15.6:0/2308390685 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:37.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30fcb7b700 1 -- 172.21.15.6:0/2308390685 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5cae810 msgr2=0x55c8f5c82b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30fcb7b700 1 --2- 172.21.15.6:0/2308390685 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5cae810 0x55c8f5c82b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30fcb7b700 1 -- 172.21.15.6:0/2308390685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 msgr2=0x55c8f5ca4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30fcb7b700 1 --2- 172.21.15.6:0/2308390685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5ca4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.885 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30fcb7b700 1 -- 172.21.15.6:0/2308390685 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8f5b9d140 con 0x55c8f5ca7ce0 2023-11-28T13:02:37.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30fcb7b700 1 --2- 172.21.15.6:0/2308390685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5ca7ce0 0x55c8f5c82650 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005440 tx=0x7f30ec005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e845301f8ba3c7fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30e37fe700 1 -- 172.21.15.6:0/2308390685 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec001ca0 con 0x55c8f5ca7ce0 2023-11-28T13:02:37.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30e37fe700 1 -- 172.21.15.6:0/2308390685 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec006860 con 0x55c8f5ca7ce0 2023-11-28T13:02:37.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.881+0000 7f30e37fe700 1 -- 172.21.15.6:0/2308390685 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec006bd0 con 0x55c8f5ca7ce0 2023-11-28T13:02:37.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 -- 172.21.15.6:0/2308390685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5ca7ce0 msgr2=0x55c8f5c82650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 --2- 172.21.15.6:0/2308390685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5ca7ce0 0x55c8f5c82650 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005440 tx=0x7f30ec005470 comp rx=0 tx=0).stop 2023-11-28T13:02:37.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 -- 172.21.15.6:0/2308390685 shutdown_connections 2023-11-28T13:02:37.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 --2- 172.21.15.6:0/2308390685 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5cae810 0x55c8f5c82b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 --2- 172.21.15.6:0/2308390685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5c831c0 0x55c8f5ca4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 --2- 172.21.15.6:0/2308390685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5ca7ce0 0x55c8f5c82650 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 -- 172.21.15.6:0/2308390685 >> 172.21.15.6:0/2308390685 conn(0x55c8f5c81d60 msgr2=0x55c8f5c7e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 -- 172.21.15.6:0/2308390685 shutdown_connections 2023-11-28T13:02:37.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 -- 172.21.15.6:0/2308390685 wait complete. 2023-11-28T13:02:37.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 -- start start 2023-11-28T13:02:37.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.882+0000 7f310b769b80 1 Processor -- start 2023-11-28T13:02:37.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f310b769b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5c831c0 0x55c8f5ca1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f310b769b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5ca7ce0 0x55c8f5c82630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f310b769b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5c82b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f310b769b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5ca2810 con 0x55c8f5cae810 2023-11-28T13:02:37.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f310b769b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5cccca0 con 0x55c8f5ca7ce0 2023-11-28T13:02:37.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f310b769b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8f5ca25c0 con 0x55c8f5c831c0 2023-11-28T13:02:37.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f310b769b80 10 client.0 _ll_register_callbacks cb 0x55c8f5a7c240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:37.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f30fdb7d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5c82b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f30fdb7d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5c82b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33374/0 (socket says 172.21.15.6:33374) 2023-11-28T13:02:37.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4727053 learned_addr learned my addr 172.21.15.6:0/4727053 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:37.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f30fcb7b700 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5ca7ce0 0x55c8f5c82630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.883+0000 7f30fd37c700 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5c831c0 0x55c8f5ca1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4727053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5c831c0 msgr2=0x55c8f5ca1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5c831c0 0x55c8f5ca1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4727053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5ca7ce0 msgr2=0x55c8f5c82630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:37.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5ca7ce0 0x55c8f5c82630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30fdb7d700 1 -- 172.21.15.6:0/4727053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f80cfb90 con 0x55c8f5cae810 2023-11-28T13:02:37.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30fd37c700 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5c831c0 0x55c8f5ca1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:37.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30fdb7d700 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5c82b70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f30f0057980 tx=0x7f30f0004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0239a34825cdc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:37.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:02:37.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30d7fff700 1 -- 172.21.15.6:0/4727053 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f00569b0 con 0x55c8f5cae810 2023-11-28T13:02:37.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30d7fff700 1 -- 172.21.15.6:0/4727053 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f30f006c030 con 0x55c8f5cae810 2023-11-28T13:02:37.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f310b769b80 1 -- 172.21.15.6:0/4727053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8f5b94160 con 0x55c8f5cae810 2023-11-28T13:02:37.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30d7fff700 1 -- 172.21.15.6:0/4727053 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30f005fae0 con 0x55c8f5cae810 2023-11-28T13:02:37.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f310b769b80 10 client.6171 fetch_fsmap learned FSMap version 24 2023-11-28T13:02:37.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f310b769b80 1 -- 172.21.15.6:0/4727053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55c8f5a2b200 con 0x55c8f5cae810 2023-11-28T13:02:37.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30d7fff700 1 -- 172.21.15.6:0/4727053 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8f5b94160 con 0x55c8f5cae810 2023-11-28T13:02:37.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f30d7fff700 1 -- 172.21.15.6:0/4727053 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f30f00579f0 con 0x55c8f5cae810 2023-11-28T13:02:37.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f310b769b80 10 client.6171 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T13:02:37.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f310b769b80 10 client.6171 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T13:02:37.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f310b769b80 20 client.6171 populate_metadata read hostname 'smithi006' 2023-11-28T13:02:37.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f310b769b80 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55c8f5cb19e0 0x55c8f5cb1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:37.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f310b769b80 4 client.6171 mds_command: new command op to 6003 tid=0[{"prefix": "status"}] 2023-11-28T13:02:37.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.884+0000 7f310b769b80 1 -- 172.21.15.6:0/4727053 --> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c8f5b7e170 con 0x55c8f5cb19e0 2023-11-28T13:02:37.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f30fd37c700 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55c8f5cb19e0 0x55c8f5cb1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:37.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f30d7fff700 10 client.6171 ms_handle_connect on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:37.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f30fd37c700 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55c8f5cb19e0 0x55c8f5cb1db0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da276c27bbc12bd3 server_cookie=eafac2a8be1c415f in_seq=0 out_seq=0 2023-11-28T13:02:37.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f30d7fff700 1 -- 172.21.15.6:0/4727053 <== mds.0 v2:172.21.15.6:6835/451011578 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55c8f5b7e170 con 0x55c8f5cb19e0 2023-11-28T13:02:37.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f30d7fff700 10 client.6171 handle_command_reply: tid=0 2023-11-28T13:02:37.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 client.6171 shutdown 2023-11-28T13:02:37.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 -- 172.21.15.6:0/4727053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 msgr2=0x55c8f5c82b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5c82b70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f30f0057980 tx=0x7f30f0004bf0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 -- 172.21.15.6:0/4727053 shutdown_connections 2023-11-28T13:02:37.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55c8f5cb19e0 0x55c8f5cb1db0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55c8f5cae810 0x55c8f5c82b70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f30d7fff700 0 client.6171 ms_handle_reset on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:37.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55c8f5c831c0 0x55c8f5ca1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 --2- 172.21.15.6:0/4727053 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c8f5ca7ce0 0x55c8f5c82630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 -- 172.21.15.6:0/4727053 >> 172.21.15.6:0/4727053 conn(0x55c8f5c81d60 msgr2=0x55c8f5ca3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 -- 172.21.15.6:0/4727053 shutdown_connections 2023-11-28T13:02:37.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.885+0000 7f310b769b80 1 -- 172.21.15.6:0/4727053 wait complete. 2023-11-28T13:02:37.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f310b769b80 20 client.6171 trim_cache size 0 max 16384 2023-11-28T13:02:37.899 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:02:37.899 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "dc643718-e974-4d8a-b829-ff6ced554687", 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 6003, 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 6.322480584, 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 24, 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 54, 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 54, 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 11.662617726000001 2023-11-28T13:02:37.900 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:02:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 -- 172.21.15.6:0/4214777470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30dc068320 msgr2=0x7f30dc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30dc068320 0x7f30dc06a7d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009de0 tx=0x7f30f4009210 comp rx=0 tx=0).stop 2023-11-28T13:02:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 -- 172.21.15.6:0/4214777470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30f80b1680 msgr2=0x7f30f80b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30f80b1680 0x7f30f80b1af0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f30e400bac0 tx=0x7f30f0001cb0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 -- 172.21.15.6:0/4214777470 shutdown_connections 2023-11-28T13:02:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30f80b1680 0x7f30f80b1af0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f30f81ef670 0x7f30f81efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f30dc068320 0x7f30dc06a7d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 --2- 172.21.15.6:0/4214777470 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f30f815bae0 0x7f30f815bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:37.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 -- 172.21.15.6:0/4214777470 >> 172.21.15.6:0/4214777470 conn(0x7f30f80cd660 msgr2=0x7f30f80b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:37.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 -- 172.21.15.6:0/4214777470 shutdown_connections 2023-11-28T13:02:37.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:37.886+0000 7f30ff5e0700 1 -- 172.21.15.6:0/4214777470 wait complete. 2023-11-28T13:02:37.903 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:02:38.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.027+0000 7f045b29d700 1 Processor -- start 2023-11-28T13:02:38.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.027+0000 7f045b29d700 1 -- start start 2023-11-28T13:02:38.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.028+0000 7f045b29d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f045415a6c0 0x7f04540cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.028+0000 7f045b29d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f04540d0020 0x7f04540ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.028+0000 7f045b29d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04540cfb90 con 0x7f04540b2660 2023-11-28T13:02:38.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.028+0000 7f045b29d700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04540cb930 con 0x7f04540d0020 2023-11-28T13:02:38.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.028+0000 7f045b29d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04540cf260 con 0x7f045415a6c0 2023-11-28T13:02:38.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.028+0000 7f0458838700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f04540d0020 0x7f04540ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.028+0000 7f0458838700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f04540d0020 0x7f04540ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33378/0 (socket says 172.21.15.6:33378) 2023-11-28T13:02:38.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.028+0000 7f0458838700 1 -- 172.21.15.6:0/3946423575 learned_addr learned my addr 172.21.15.6:0/3946423575 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:38.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f0459039700 1 --2- 172.21.15.6:0/3946423575 >> v2:172.21.15.61:3300/0 conn(0x7f045415a6c0 0x7f04540cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f0458838700 1 -- 172.21.15.6:0/3946423575 >> v2:172.21.15.61:3300/0 conn(0x7f045415a6c0 msgr2=0x7f04540cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:38.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f0458838700 1 --2- 172.21.15.6:0/3946423575 >> v2:172.21.15.61:3300/0 conn(0x7f045415a6c0 0x7f04540cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f0458838700 1 -- 172.21.15.6:0/3946423575 >> v1:172.21.15.61:6789/0 conn(0x7f04540b2660 legacy=0x7f04540b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:38.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f0458838700 1 -- 172.21.15.6:0/3946423575 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04540cf540 con 0x7f04540d0020 2023-11-28T13:02:38.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f0458838700 1 --2- 172.21.15.6:0/3946423575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04540d0020 0x7f04540ccea0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f045406ffd0 tx=0x7f0450009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2577e8eb3113aff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:38.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f044bfff700 1 -- 172.21.15.6:0/3946423575 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450004230 con 0x7f04540d0020 2023-11-28T13:02:38.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f044bfff700 1 -- 172.21.15.6:0/3946423575 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0450004390 con 0x7f04540d0020 2023-11-28T13:02:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f044bfff700 1 -- 172.21.15.6:0/3946423575 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04500036a0 con 0x7f04540d0020 2023-11-28T13:02:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f045b29d700 1 -- 172.21.15.6:0/3946423575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04540d0020 msgr2=0x7f04540ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.029+0000 7f045b29d700 1 --2- 172.21.15.6:0/3946423575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04540d0020 0x7f04540ccea0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f045406ffd0 tx=0x7f0450009bd0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.045 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.030+0000 7f045b29d700 1 -- 172.21.15.6:0/3946423575 shutdown_connections 2023-11-28T13:02:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.030+0000 7f045b29d700 1 --2- 172.21.15.6:0/3946423575 >> v2:172.21.15.61:3300/0 conn(0x7f045415a6c0 0x7f04540cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.030+0000 7f045b29d700 1 --2- 172.21.15.6:0/3946423575 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04540d0020 0x7f04540ccea0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.030+0000 7f045b29d700 1 -- 172.21.15.6:0/3946423575 >> 172.21.15.6:0/3946423575 conn(0x7f04540cd660 msgr2=0x7f04540cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.030+0000 7f045b29d700 1 -- 172.21.15.6:0/3946423575 shutdown_connections 2023-11-28T13:02:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.030+0000 7f045b29d700 1 -- 172.21.15.6:0/3946423575 wait complete. 2023-11-28T13:02:38.046 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.031+0000 7f045b29d700 1 Processor -- start 2023-11-28T13:02:38.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.031+0000 7f045b29d700 1 -- start start 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045b29d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04540b2660 0x7f04540c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045b29d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04540d0020 0x7f0454153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045b29d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f045415a6c0 0x7f04541558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045b29d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0454153f60 con 0x7f045415a6c0 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045b29d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04541561f0 con 0x7f04540b2660 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045b29d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04541564d0 con 0x7f04540d0020 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045983a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f045415a6c0 0x7f04541558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045983a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f045415a6c0 0x7f04541558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33388/0 (socket says 172.21.15.6:33388) 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045983a700 1 -- 172.21.15.6:0/3375565227 learned_addr learned my addr 172.21.15.6:0/3375565227 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f0459039700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04540b2660 0x7f04540c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f0458838700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04540d0020 0x7f0454153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045983a700 1 -- 172.21.15.6:0/3375565227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04540d0020 msgr2=0x7f0454153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045983a700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04540d0020 0x7f0454153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045983a700 1 -- 172.21.15.6:0/3375565227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04540b2660 msgr2=0x7f04540c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045983a700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04540b2660 0x7f04540c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045983a700 1 -- 172.21.15.6:0/3375565227 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0450009550 con 0x7f045415a6c0 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f0458838700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04540d0020 0x7f0454153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045983a700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f045415a6c0 0x7f04541558a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f044c00cc10 tx=0x7f044c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f0449ffb700 1 -- 172.21.15.6:0/3375565227 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c00ab10 con 0x7f045415a6c0 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f0449ffb700 1 -- 172.21.15.6:0/3375565227 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c00ac70 con 0x7f045415a6c0 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f045b29d700 1 -- 172.21.15.6:0/3375565227 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04540cd3e0 con 0x7f045415a6c0 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.032+0000 7f0449ffb700 1 -- 172.21.15.6:0/3375565227 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c012420 con 0x7f045415a6c0 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.033+0000 7f045b29d700 1 -- 172.21.15.6:0/3375565227 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04540cf820 con 0x7f045415a6c0 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.033+0000 7f045b29d700 1 -- 172.21.15.6:0/3375565227 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04540d0470 con 0x7f045415a6c0 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.034+0000 7f0449ffb700 1 -- 172.21.15.6:0/3375565227 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f044c010050 con 0x7f045415a6c0 2023-11-28T13:02:38.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.034+0000 7f0449ffb700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04380682f0 0x7f043806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:38.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.034+0000 7f0449ffb700 1 -- 172.21.15.6:0/3375565227 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f044c086120 con 0x7f045415a6c0 2023-11-28T13:02:38.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.034+0000 7f0459039700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04380682f0 0x7f043806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:38.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.035+0000 7f0459039700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04380682f0 0x7f043806a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0444006b20 tx=0x7f0444008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:38.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.039+0000 7f0449ffb700 1 -- 172.21.15.6:0/3375565227 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f044c0526f0 con 0x7f045415a6c0 2023-11-28T13:02:38.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.278+0000 7f045b29d700 1 -- 172.21.15.6:0/3375565227 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f04540675a0 con 0x7f045415a6c0 2023-11-28T13:02:38.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.278+0000 7f0449ffb700 1 -- 172.21.15.6:0/3375565227 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f044c055d10 con 0x7f045415a6c0 2023-11-28T13:02:38.280 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:02:38.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 -- 172.21.15.6:0/3375565227 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04380682f0 msgr2=0x7f043806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04380682f0 0x7f043806a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0444006b20 tx=0x7f0444008040 comp rx=0 tx=0).stop 2023-11-28T13:02:38.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 -- 172.21.15.6:0/3375565227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f045415a6c0 msgr2=0x7f04541558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:38.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f045415a6c0 0x7f04541558a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f044c00cc10 tx=0x7f044c00cf20 comp rx=0 tx=0).stop 2023-11-28T13:02:38.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 -- 172.21.15.6:0/3375565227 shutdown_connections 2023-11-28T13:02:38.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f045415a6c0 0x7f04541558a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f04540b2660 0x7f04540c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04380682f0 0x7f043806a7a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 --2- 172.21.15.6:0/3375565227 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f04540d0020 0x7f0454153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:38.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 -- 172.21.15.6:0/3375565227 >> 172.21.15.6:0/3375565227 conn(0x7f04540cd660 msgr2=0x7f04540b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:38.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 -- 172.21.15.6:0/3375565227 shutdown_connections 2023-11-28T13:02:38.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:38.280+0000 7f045b29d700 1 -- 172.21.15.6:0/3375565227 wait complete. 2023-11-28T13:02:38.294 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:02:38.294 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:02:38.294 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-28T13:02:38.294 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:02:38.294 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:38.294 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:02:38.294 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T13:02:38.294 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:02:38.294 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:02:38.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:38.295 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-28T13:02:38.312 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:02:38.313 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42205sec preferred_lft 42205sec 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 8e:02:21:fe:ce:1c brd ff:ff:ff:ff:ff:ff 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::68d7:25ff:fe50:263/64 scope link 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:38.314 INFO:teuthology.orchestra.run.smithi006.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:02:38.315 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether ba:07:72:c8:31:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:02:38.315 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::b807:72ff:fec8:3118/64 scope link 2023-11-28T13:02:38.315 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:38.315 INFO:teuthology.orchestra.run.smithi006.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:02:38.315 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 8e:02:21:fe:ce:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:02:38.315 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::8c02:21ff:fefe:ce1c/64 scope link 2023-11-28T13:02:38.315 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:02:38.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:38.315 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:02:38.379 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-28T13:02:38.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:38.381 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-28T13:02:38.440 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-28T13:02:38.440 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:02:38.440 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T13:02:38.440 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T13:02:38.440 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:02:38.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:38.441 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:02:38.441 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:02:38.441 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T13:02:38.441 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T13:02:38.441 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T13:02:38.441 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:02:38.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:38.548 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-28T13:02:38.566 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:02:38.566 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:02:38.567 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:02:38.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:38.567 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2023-11-28T13:02:38.633 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T13:02:38.634 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:02:38.634 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:38.691 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:02:38.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:02:38.691 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:38.751 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:02:38.819 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:02:38.819 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:02:38.886 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:02:38.886 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:02:38.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:38.887 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:02:38.944 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T13:02:38.944 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T13:02:38.986 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:02:39.013 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:02:39.014 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:02:39.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:2023-11-28T13:02:39.075+0000 7f53f1e225c0 -1 init, newargv = 0x557dc346fc00 newargc=15 2023-11-28T13:02:39.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[126073]: starting ceph client 2023-11-28T13:02:39.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[126073]: starting fuse 2023-11-28T13:02:39.082 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:02:39.084 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:02:39.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:39.084 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:02:39.143 INFO:teuthology.orchestra.run.smithi006.stdout:47 2023-11-28T13:02:39.143 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> import re 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2023-11-28T13:02:39.144 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> return f 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.0")) 2023-11-28T13:02:39.145 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:02:39.268 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.0.126073.asok 2023-11-28T13:02:39.449 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.126073.asok 2023-11-28T13:02:39.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:39.450 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.126073.asok status 2023-11-28T13:02:39.614 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:02:39.614 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2023-11-28T13:02:39.614 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T13:02:39.614 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T13:02:39.614 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "0", 2023-11-28T13:02:39.614 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "126073", 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 6180, 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 6180 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:39.615 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1747716784 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1747716784 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.6180 192.168.0.1:0/1747716784", 2023-11-28T13:02:39.616 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.1:0/1747716784", 2023-11-28T13:02:39.617 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2023-11-28T13:02:39.617 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 24, 2023-11-28T13:02:39.617 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 54, 2023-11-28T13:02:39.617 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2023-11-28T13:02:39.617 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2023-11-28T13:02:39.617 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2023-11-28T13:02:39.617 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:02:39.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:39.624 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2023-11-28T13:02:39.624 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:39.642 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2023-11-28T13:02:39.642 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:39.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:39.642 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:39.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:39.711 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-28T13:02:39.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:39.774 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-28T13:02:39.834 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:02:39.834 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:02:39.834 DEBUG:teuthology.orchestra.run.smithi006:> import stat 2023-11-28T13:02:39.834 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:02:39.834 DEBUG:teuthology.orchestra.run.smithi006:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-28T13:02:39.834 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:02:39.941 INFO:teuthology.orchestra.run.smithi006.stdout:1099511627777 2023-11-28T13:02:40.130 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.295+0000 7f0ebd854700 1 Processor -- start 2023-11-28T13:02:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.295+0000 7f0ebd854700 1 -- start start 2023-11-28T13:02:40.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.295+0000 7f0ebd854700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0eb815c640 0x7f0eb80cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.295+0000 7f0ebd854700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0eb806bf30 0x7f0eb806c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.295+0000 7f0ebd854700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb80b6350 con 0x7f0eb80d2820 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.295+0000 7f0ebd854700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb815ca10 con 0x7f0eb806bf30 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.295+0000 7f0ebd854700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb80d1470 con 0x7f0eb815c640 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb77fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0eb806bf30 0x7f0eb806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb77fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0eb806bf30 0x7f0eb806c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33410/0 (socket says 172.21.15.6:33410) 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb77fe700 1 -- 172.21.15.6:0/1791039396 learned_addr learned my addr 172.21.15.6:0/1791039396 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb6ffd700 1 --2- 172.21.15.6:0/1791039396 >> v2:172.21.15.61:3300/0 conn(0x7f0eb815c640 0x7f0eb80cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb77fe700 1 -- 172.21.15.6:0/1791039396 >> v2:172.21.15.61:3300/0 conn(0x7f0eb815c640 msgr2=0x7f0eb80cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb77fe700 1 --2- 172.21.15.6:0/1791039396 >> v2:172.21.15.61:3300/0 conn(0x7f0eb815c640 0x7f0eb80cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb77fe700 1 -- 172.21.15.6:0/1791039396 >> v1:172.21.15.61:6789/0 conn(0x7f0eb80d2820 legacy=0x7f0eb80b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb77fe700 1 -- 172.21.15.6:0/1791039396 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb806c840 con 0x7f0eb806bf30 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb6ffd700 1 --2- 172.21.15.6:0/1791039396 >> v2:172.21.15.61:3300/0 conn(0x7f0eb815c640 0x7f0eb80cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb77fe700 1 --2- 172.21.15.6:0/1791039396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 0x7f0eb806c300 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0eb806d420 tx=0x7f0eb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262be90100ed87be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.299 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb5ffb700 1 -- 172.21.15.6:0/1791039396 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000e070 con 0x7f0eb806bf30 2023-11-28T13:02:40.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb5ffb700 1 -- 172.21.15.6:0/1791039396 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb0003a20 con 0x7f0eb806bf30 2023-11-28T13:02:40.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.296+0000 7f0eb5ffb700 1 -- 172.21.15.6:0/1791039396 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0003dd0 con 0x7f0eb806bf30 2023-11-28T13:02:40.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.297+0000 7f0ebd854700 1 -- 172.21.15.6:0/1791039396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 msgr2=0x7f0eb806c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.297+0000 7f0ebd854700 1 --2- 172.21.15.6:0/1791039396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 0x7f0eb806c300 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0eb806d420 tx=0x7f0eb000b330 comp rx=0 tx=0).stop 2023-11-28T13:02:40.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.297+0000 7f0ebd854700 1 -- 172.21.15.6:0/1791039396 shutdown_connections 2023-11-28T13:02:40.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.297+0000 7f0ebd854700 1 --2- 172.21.15.6:0/1791039396 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 0x7f0eb806c300 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.297+0000 7f0ebd854700 1 --2- 172.21.15.6:0/1791039396 >> v2:172.21.15.61:3300/0 conn(0x7f0eb815c640 0x7f0eb80cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.297+0000 7f0ebd854700 1 -- 172.21.15.6:0/1791039396 >> 172.21.15.6:0/1791039396 conn(0x7f0eb80d2df0 msgr2=0x7f0eb80d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.297+0000 7f0ebd854700 1 -- 172.21.15.6:0/1791039396 shutdown_connections 2023-11-28T13:02:40.300 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.297+0000 7f0ebd854700 1 -- 172.21.15.6:0/1791039396 wait complete. 2023-11-28T13:02:40.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.298+0000 7f0ebd854700 1 Processor -- start 2023-11-28T13:02:40.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.298+0000 7f0ebd854700 1 -- start start 2023-11-28T13:02:40.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0ebd854700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 0x7f0eb815f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0ebd854700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0eb80d2820 0x7f0eb80c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0ebd854700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0eb815c640 0x7f0eb80c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0ebd854700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb815ec40 con 0x7f0eb806bf30 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0ebd854700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb815f5a0 con 0x7f0eb815c640 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0ebd854700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb815dfc0 con 0x7f0eb80d2820 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 0x7f0eb815f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 0x7f0eb815f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33412/0 (socket says 172.21.15.6:33412) 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb6ffd700 1 -- 172.21.15.6:0/1313748344 learned_addr learned my addr 172.21.15.6:0/1313748344 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb77fe700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0eb815c640 0x7f0eb80c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb67fc700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0eb80d2820 0x7f0eb80c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb6ffd700 1 -- 172.21.15.6:0/1313748344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0eb80d2820 msgr2=0x7f0eb80c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb6ffd700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0eb80d2820 0x7f0eb80c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb6ffd700 1 -- 172.21.15.6:0/1313748344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0eb815c640 msgr2=0x7f0eb80c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb6ffd700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0eb815c640 0x7f0eb80c6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb6ffd700 1 -- 172.21.15.6:0/1313748344 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb0009b30 con 0x7f0eb806bf30 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb67fc700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0eb80d2820 0x7f0eb80c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb77fe700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0eb815c640 0x7f0eb80c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.299+0000 7f0eb6ffd700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 0x7f0eb815f060 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800c7f0 tx=0x7f0ea800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.300+0000 7f0e9ffff700 1 -- 172.21.15.6:0/1313748344 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8011800 con 0x7f0eb806bf30 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.300+0000 7f0e9ffff700 1 -- 172.21.15.6:0/1313748344 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea8011960 con 0x7f0eb806bf30 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.300+0000 7f0ebd854700 1 -- 172.21.15.6:0/1313748344 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb80cd7c0 con 0x7f0eb806bf30 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.300+0000 7f0e9ffff700 1 -- 172.21.15.6:0/1313748344 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8011d10 con 0x7f0eb806bf30 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.300+0000 7f0ebd854700 1 -- 172.21.15.6:0/1313748344 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb80cd4e0 con 0x7f0eb806bf30 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.301+0000 7f0ebd854700 1 -- 172.21.15.6:0/1313748344 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb80d1710 con 0x7f0eb806bf30 2023-11-28T13:02:40.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.301+0000 7f0e9ffff700 1 -- 172.21.15.6:0/1313748344 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0ea8021070 con 0x7f0eb806bf30 2023-11-28T13:02:40.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.301+0000 7f0e9ffff700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ea40682f0 0x7f0ea406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.302+0000 7f0e9ffff700 1 -- 172.21.15.6:0/1313748344 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0ea8085cb0 con 0x7f0eb806bf30 2023-11-28T13:02:40.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.302+0000 7f0eb67fc700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ea40682f0 0x7f0ea406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.302+0000 7f0eb67fc700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ea40682f0 0x7f0ea406a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0eb806ac30 tx=0x7f0eac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.306+0000 7f0e9ffff700 1 -- 172.21.15.6:0/1313748344 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0ea8052350 con 0x7f0eb806bf30 2023-11-28T13:02:40.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.543+0000 7f0ebd854700 1 -- 172.21.15.6:0/1313748344 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0eb815eeb0 con 0x7f0eb806bf30 2023-11-28T13:02:40.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.544+0000 7f0e9ffff700 1 -- 172.21.15.6:0/1313748344 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4636 (secure 0 0 0) 0x7f0ea801f070 con 0x7f0eb806bf30 2023-11-28T13:02:40.545 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:40.545 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4004512137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4004512137},{"type":"v1","addr":"172.21.15.61:6833","nonce":4004512137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/1313775424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":1313775424},{"type":"v1","addr":"172.21.15.61:6835","nonce":1313775424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/233781200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":233781200},{"type":"v1","addr":"172.21.15.6:6836","nonce":233781200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:31.562404+0000","modified":"2023-11-28T13:02:34.939442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/451011578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":451011578},{"type":"v1","addr":"172.21.15.6:6837","nonce":451011578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:40.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 -- 172.21.15.6:0/1313748344 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ea40682f0 msgr2=0x7f0ea406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ea40682f0 0x7f0ea406a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0eb806ac30 tx=0x7f0eac009040 comp rx=0 tx=0).stop 2023-11-28T13:02:40.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 -- 172.21.15.6:0/1313748344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 msgr2=0x7f0eb815f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 0x7f0eb815f060 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800c7f0 tx=0x7f0ea800cbb0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 -- 172.21.15.6:0/1313748344 shutdown_connections 2023-11-28T13:02:40.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0eb815c640 0x7f0eb80c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0eb806bf30 0x7f0eb815f060 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ea40682f0 0x7f0ea406a7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 --2- 172.21.15.6:0/1313748344 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0eb80d2820 0x7f0eb80c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 -- 172.21.15.6:0/1313748344 >> 172.21.15.6:0/1313748344 conn(0x7f0eb80d2df0 msgr2=0x7f0eb80d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 -- 172.21.15.6:0/1313748344 shutdown_connections 2023-11-28T13:02:40.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.546+0000 7f0ebd854700 1 -- 172.21.15.6:0/1313748344 wait complete. 2023-11-28T13:02:40.550 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2023-11-28T13:02:40.559 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T13:02:40.718 INFO:teuthology.orchestra.run.smithi006.stdout:{"message":"","return_code":0} 2023-11-28T13:02:40.718 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:02:40.718 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T13:02:40.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.763+0000 7fe0ff785080 1 Processor -- start 2023-11-28T13:02:40.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.763+0000 7fe0ff785080 1 -- start start 2023-11-28T13:02:40.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.764+0000 7fe0ff785080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0e4003b80 0x55ce6b6a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.764+0000 7fe0ff785080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b68c1f0 0x55ce6b5fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.764+0000 7fe0ff785080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ce6b5d03d0 0x55ce6b5fd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.764+0000 7fe0ff785080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce6b577000 con 0x55ce6b68c1f0 2023-11-28T13:02:40.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.764+0000 7fe0ff785080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce6b509fe0 con 0x7fe0e4003b80 2023-11-28T13:02:40.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.764+0000 7fe0ff785080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce6b68c020 con 0x55ce6b5d03d0 2023-11-28T13:02:40.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.764+0000 7fe0f5087700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b68c1f0 0x55ce6b5fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.764+0000 7fe0f5087700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b68c1f0 0x55ce6b5fd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33434/0 (socket says 172.21.15.6:33434) 2023-11-28T13:02:40.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.764+0000 7fe0f5087700 1 -- 172.21.15.6:0/923124495 learned_addr learned my addr 172.21.15.6:0/923124495 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:40.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.764+0000 7fe0f5888700 1 --2- 172.21.15.6:0/923124495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0e4003b80 0x55ce6b6a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0f6089700 1 --2- 172.21.15.6:0/923124495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ce6b5d03d0 0x55ce6b5fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0f5087700 1 -- 172.21.15.6:0/923124495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ce6b5d03d0 msgr2=0x55ce6b5fd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0f5087700 1 --2- 172.21.15.6:0/923124495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ce6b5d03d0 0x55ce6b5fd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0f5087700 1 -- 172.21.15.6:0/923124495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0e4003b80 msgr2=0x55ce6b6a66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0f5087700 1 --2- 172.21.15.6:0/923124495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0e4003b80 0x55ce6b6a66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0f5087700 1 -- 172.21.15.6:0/923124495 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce6b522360 con 0x55ce6b68c1f0 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0f6089700 1 --2- 172.21.15.6:0/923124495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ce6b5d03d0 0x55ce6b5fd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0f5087700 1 --2- 172.21.15.6:0/923124495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b68c1f0 0x55ce6b5fd410 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80097b0 tx=0x7fe0e8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa67402537b3257 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0effff700 1 -- 172.21.15.6:0/923124495 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800f040 con 0x55ce6b68c1f0 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0effff700 1 -- 172.21.15.6:0/923124495 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e8008870 con 0x55ce6b68c1f0 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0effff700 1 -- 172.21.15.6:0/923124495 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8008be0 con 0x55ce6b68c1f0 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0ff785080 1 -- 172.21.15.6:0/923124495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b68c1f0 msgr2=0x55ce6b5fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0ff785080 1 --2- 172.21.15.6:0/923124495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b68c1f0 0x55ce6b5fd410 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80097b0 tx=0x7fe0e8005a00 comp rx=0 tx=0).stop 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0ff785080 1 -- 172.21.15.6:0/923124495 shutdown_connections 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0ff785080 1 --2- 172.21.15.6:0/923124495 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ce6b5d03d0 0x55ce6b5fd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0ff785080 1 --2- 172.21.15.6:0/923124495 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe0e4003b80 0x55ce6b6a66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0ff785080 1 --2- 172.21.15.6:0/923124495 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b68c1f0 0x55ce6b5fd410 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.765+0000 7fe0ff785080 1 -- 172.21.15.6:0/923124495 >> 172.21.15.6:0/923124495 conn(0x55ce6b6167e0 msgr2=0x55ce6b5cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.766+0000 7fe0ff785080 1 -- 172.21.15.6:0/923124495 shutdown_connections 2023-11-28T13:02:40.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.766+0000 7fe0ff785080 1 -- 172.21.15.6:0/923124495 wait complete. 2023-11-28T13:02:40.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.766+0000 7fe0ff785080 1 Processor -- start 2023-11-28T13:02:40.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.766+0000 7fe0ff785080 1 -- start start 2023-11-28T13:02:40.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0ff785080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b5d03d0 0x55ce6b689bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0ff785080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ce6b68c1f0 0x55ce6b5f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0ff785080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0e4003b80 0x55ce6b5f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0ff785080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce6b5fad30 con 0x55ce6b5d03d0 2023-11-28T13:02:40.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0ff785080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce6b5cfa70 con 0x55ce6b68c1f0 2023-11-28T13:02:40.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0ff785080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce6b5cfd10 con 0x7fe0e4003b80 2023-11-28T13:02:40.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5888700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b5d03d0 0x55ce6b689bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5888700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b5d03d0 0x55ce6b689bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33438/0 (socket says 172.21.15.6:33438) 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5888700 1 -- 172.21.15.6:0/2563555776 learned_addr learned my addr 172.21.15.6:0/2563555776 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f6089700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0e4003b80 0x55ce6b5f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5087700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ce6b68c1f0 0x55ce6b5f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5888700 1 -- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0e4003b80 msgr2=0x55ce6b5f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5888700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe0e4003b80 0x55ce6b5f2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5888700 1 -- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ce6b68c1f0 msgr2=0x55ce6b5f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5888700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ce6b68c1f0 0x55ce6b5f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5888700 1 -- 172.21.15.6:0/2563555776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e8009410 con 0x55ce6b5d03d0 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5087700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ce6b68c1f0 0x55ce6b5f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.767+0000 7fe0f5888700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b5d03d0 0x55ce6b689bd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0011ec0 tx=0x7fe0e000f620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.768+0000 7fe0eeffd700 1 -- 172.21.15.6:0/2563555776 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0012430 con 0x55ce6b5d03d0 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.768+0000 7fe0eeffd700 1 -- 172.21.15.6:0/2563555776 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e0008030 con 0x55ce6b5d03d0 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.768+0000 7fe0ff785080 1 -- 172.21.15.6:0/2563555776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ce6b6143b0 con 0x55ce6b5d03d0 2023-11-28T13:02:40.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.768+0000 7fe0eeffd700 1 -- 172.21.15.6:0/2563555776 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00127b0 con 0x55ce6b5d03d0 2023-11-28T13:02:40.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.768+0000 7fe0ff785080 1 -- 172.21.15.6:0/2563555776 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce6b515830 con 0x55ce6b5d03d0 2023-11-28T13:02:40.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.769+0000 7fe0eeffd700 1 -- 172.21.15.6:0/2563555776 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe0e0022070 con 0x55ce6b5d03d0 2023-11-28T13:02:40.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.769+0000 7fe0eeffd700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8066c60 0x7fe0d8069110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.769+0000 7fe0eeffd700 1 -- 172.21.15.6:0/2563555776 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe0e0084d60 con 0x55ce6b5d03d0 2023-11-28T13:02:40.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.770+0000 7fe0f5087700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8066c60 0x7fe0d8069110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.770+0000 7fe0ff785080 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55ce6b5f3e40 0x55ce6b5f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.770+0000 7fe0ff785080 1 -- 172.21.15.6:0/2563555776 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54) v8 -- 0x55ce6b73dc30 con 0x55ce6b5f3e40 2023-11-28T13:02:40.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.770+0000 7fe0f6089700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55ce6b5f3e40 0x55ce6b5f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.770+0000 7fe0f5087700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8066c60 0x7fe0d8069110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55ce6b6a70b0 tx=0x7fe0e8007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.770+0000 7fe0f6089700 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55ce6b5f3e40 0x55ce6b5f3280 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0f6089700 1 -- 172.21.15.6:0/2563555776 <== osd.5 v2:172.21.15.61:6808/103780 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fe0f0001250 con 0x55ce6b5f3e40 2023-11-28T13:02:40.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0ff785080 1 -- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55ce6b5f3e40 msgr2=0x55ce6b5f3280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0ff785080 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55ce6b5f3e40 0x55ce6b5f3280 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0ff785080 1 -- 172.21.15.6:0/2563555776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8066c60 msgr2=0x7fe0d8069110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0ff785080 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe0d8066c60 0x7fe0d8069110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55ce6b6a70b0 tx=0x7fe0e8007000 comp rx=0 tx=0).stop 2023-11-28T13:02:40.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0ff785080 1 -- 172.21.15.6:0/2563555776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b5d03d0 msgr2=0x55ce6b689bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0ff785080 1 --2- 172.21.15.6:0/2563555776 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ce6b5d03d0 0x55ce6b689bd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0011ec0 tx=0x7fe0e000f620 comp rx=0 tx=0).stop 2023-11-28T13:02:40.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0f6089700 1 -- 172.21.15.6:0/2563555776 reap_dead start 2023-11-28T13:02:40.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0ff785080 1 -- 172.21.15.6:0/2563555776 shutdown_connections 2023-11-28T13:02:40.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0ff785080 1 -- 172.21.15.6:0/2563555776 >> 172.21.15.6:0/2563555776 conn(0x55ce6b6167e0 msgr2=0x55ce6b6a66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0ff785080 1 -- 172.21.15.6:0/2563555776 shutdown_connections 2023-11-28T13:02:40.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.771+0000 7fe0ff785080 1 -- 172.21.15.6:0/2563555776 wait complete. 2023-11-28T13:02:40.781 INFO:teuthology.orchestra.run.smithi006.stdout:Yalphaparent_a 2023-11-28T13:02:40.781 DEBUG:teuthology.orchestra.run.smithi006:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T13:02:40.896 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:02:40.896 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511627777, 2023-11-28T13:02:40.896 INFO:teuthology.orchestra.run.smithi006.stdout: "ancestors": [ 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: "dirino": 1099511627776, 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: "dname": "alpha", 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 2 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: "dirino": 1, 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: "dname": "parent_a", 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 4 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:40.897 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-28T13:02:40.898 INFO:teuthology.orchestra.run.smithi006.stdout: "pool": 9, 2023-11-28T13:02:40.898 INFO:teuthology.orchestra.run.smithi006.stdout: "old_pools": [] 2023-11-28T13:02:40.898 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:02:40.898 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:40.899 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-28T13:02:40.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.943+0000 7fdcaed8f080 1 Processor -- start 2023-11-28T13:02:40.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.943+0000 7fdcaed8f080 1 -- start start 2023-11-28T13:02:40.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdcaed8f080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559dddd86880 0x559ddde53da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdcaed8f080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x559ddddb2d80 0x559ddddaf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdcaed8f080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddddb3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdcaed8f080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dddcbefe0 con 0x559ddde5c6f0 2023-11-28T13:02:40.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdcaed8f080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddddb0790 con 0x559dddd86880 2023-11-28T13:02:40.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdcaed8f080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ddde5c460 con 0x559ddddb2d80 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca5693700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddddb3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca5693700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddddb3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33458/0 (socket says 172.21.15.6:33458) 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca5693700 1 -- 172.21.15.6:0/12362990 learned_addr learned my addr 172.21.15.6:0/12362990 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca4e92700 1 --2- 172.21.15.6:0/12362990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559dddd86880 0x559ddde53da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca4691700 1 --2- 172.21.15.6:0/12362990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x559ddddb2d80 0x559ddddaf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca5693700 1 -- 172.21.15.6:0/12362990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x559ddddb2d80 msgr2=0x559ddddaf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca5693700 1 --2- 172.21.15.6:0/12362990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x559ddddb2d80 0x559ddddaf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca5693700 1 -- 172.21.15.6:0/12362990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559dddd86880 msgr2=0x559ddde53da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca5693700 1 --2- 172.21.15.6:0/12362990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559dddd86880 0x559ddde53da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca5693700 1 -- 172.21.15.6:0/12362990 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98005de0 con 0x559ddde5c6f0 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca4e92700 1 --2- 172.21.15.6:0/12362990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559dddd86880 0x559ddde53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:40.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca5693700 1 --2- 172.21.15.6:0/12362990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddddb3220 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdc94002850 tx=0x7fdc94002b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc972c965f8ff10 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca368f700 1 -- 172.21.15.6:0/12362990 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94012680 con 0x559ddde5c6f0 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.944+0000 7fdca368f700 1 -- 172.21.15.6:0/12362990 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc940127e0 con 0x559ddde5c6f0 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.945+0000 7fdca368f700 1 -- 172.21.15.6:0/12362990 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94012b90 con 0x559ddde5c6f0 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.945+0000 7fdcaed8f080 1 -- 172.21.15.6:0/12362990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 msgr2=0x559ddddb3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.945+0000 7fdcaed8f080 1 --2- 172.21.15.6:0/12362990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddddb3220 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdc94002850 tx=0x7fdc94002b60 comp rx=0 tx=0).stop 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.945+0000 7fdcaed8f080 1 -- 172.21.15.6:0/12362990 shutdown_connections 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.945+0000 7fdcaed8f080 1 --2- 172.21.15.6:0/12362990 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddddb3220 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.945+0000 7fdcaed8f080 1 --2- 172.21.15.6:0/12362990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559dddd86880 0x559ddde53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.945+0000 7fdcaed8f080 1 --2- 172.21.15.6:0/12362990 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x559ddddb2d80 0x559ddddaf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.945+0000 7fdcaed8f080 1 -- 172.21.15.6:0/12362990 >> 172.21.15.6:0/12362990 conn(0x559ddddb1310 msgr2=0x559dddd84bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.945+0000 7fdcaed8f080 1 -- 172.21.15.6:0/12362990 shutdown_connections 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.945+0000 7fdcaed8f080 1 -- 172.21.15.6:0/12362990 wait complete. 2023-11-28T13:02:40.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.946+0000 7fdcaed8f080 1 Processor -- start 2023-11-28T13:02:40.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.946+0000 7fdcaed8f080 1 -- start start 2023-11-28T13:02:40.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdcaed8f080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559dddd86880 0x559ddde48360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdcaed8f080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x559ddddb2d80 0x559ddde54960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdcaed8f080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddde54ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.952 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdcaed8f080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddde4a360 con 0x559ddde5c6f0 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdcaed8f080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddde48f50 con 0x559dddd86880 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdcaed8f080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ddde49120 con 0x559ddddb2d80 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca5693700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddde54ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca5693700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddde54ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33462/0 (socket says 172.21.15.6:33462) 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca5693700 1 -- 172.21.15.6:0/1092236050 learned_addr learned my addr 172.21.15.6:0/1092236050 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca4e92700 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559dddd86880 0x559ddde48360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca4691700 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x559ddddb2d80 0x559ddde54960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca5693700 1 -- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x559ddddb2d80 msgr2=0x559ddde54960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca5693700 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x559ddddb2d80 0x559ddde54960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca5693700 1 -- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559dddd86880 msgr2=0x559ddde48360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca5693700 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559dddd86880 0x559ddde48360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca5693700 1 -- 172.21.15.6:0/1092236050 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc94002530 con 0x559ddde5c6f0 2023-11-28T13:02:40.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.947+0000 7fdca5693700 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddde54ea0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdc94014040 tx=0x7fdc940033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.948+0000 7fdca268d700 1 -- 172.21.15.6:0/1092236050 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400d820 con 0x559ddde5c6f0 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.948+0000 7fdca268d700 1 -- 172.21.15.6:0/1092236050 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9401f070 con 0x559ddde5c6f0 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.948+0000 7fdca268d700 1 -- 172.21.15.6:0/1092236050 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400dba0 con 0x559ddde5c6f0 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.948+0000 7fdcaed8f080 1 -- 172.21.15.6:0/1092236050 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559dddcd7120 con 0x559ddde5c6f0 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.948+0000 7fdcaed8f080 1 -- 172.21.15.6:0/1092236050 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559dddcca830 con 0x559ddde5c6f0 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.949+0000 7fdca268d700 1 -- 172.21.15.6:0/1092236050 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdc9400dd00 con 0x559ddde5c6f0 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.949+0000 7fdca268d700 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdc90066c10 0x7fdc900690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.949+0000 7fdca268d700 1 -- 172.21.15.6:0/1092236050 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fdc94021070 con 0x559ddde5c6f0 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.949+0000 7fdca4e92700 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdc90066c10 0x7fdc900690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.950+0000 7fdcaed8f080 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x559ddde53ef0 0x559dddefb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:40.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.950+0000 7fdcaed8f080 1 -- 172.21.15.6:0/1092236050 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) v8 -- 0x559ddddb4350 con 0x559ddde53ef0 2023-11-28T13:02:40.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.950+0000 7fdca4e92700 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdc90066c10 0x7fdc900690c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c003f10 tx=0x7fdc9c003870 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.950+0000 7fdca4691700 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x559ddde53ef0 0x559dddefb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:40.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.950+0000 7fdca4691700 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x559ddde53ef0 0x559dddefb9d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:40.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdca4691700 1 -- 172.21.15.6:0/1092236050 <== osd.5 v2:172.21.15.61:6808/103780 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v54'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fdc98019040 con 0x559ddde53ef0 2023-11-28T13:02:40.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdcaed8f080 1 -- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x559ddde53ef0 msgr2=0x559dddefb9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdcaed8f080 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x559ddde53ef0 0x559dddefb9d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdcaed8f080 1 -- 172.21.15.6:0/1092236050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdc90066c10 msgr2=0x7fdc900690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdcaed8f080 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdc90066c10 0x7fdc900690c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c003f10 tx=0x7fdc9c003870 comp rx=0 tx=0).stop 2023-11-28T13:02:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdcaed8f080 1 -- 172.21.15.6:0/1092236050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 msgr2=0x559ddde54ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdcaed8f080 1 --2- 172.21.15.6:0/1092236050 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x559ddde5c6f0 0x559ddde54ea0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdc94014040 tx=0x7fdc940033a0 comp rx=0 tx=0).stop 2023-11-28T13:02:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdca5693700 1 -- 172.21.15.6:0/1092236050 reap_dead start 2023-11-28T13:02:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdcaed8f080 1 -- 172.21.15.6:0/1092236050 shutdown_connections 2023-11-28T13:02:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdcaed8f080 1 -- 172.21.15.6:0/1092236050 >> 172.21.15.6:0/1092236050 conn(0x559ddddb1310 msgr2=0x559dddd84bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdcaed8f080 1 -- 172.21.15.6:0/1092236050 shutdown_connections 2023-11-28T13:02:40.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:40.952+0000 7fdcaed8f080 1 -- 172.21.15.6:0/1092236050 wait complete. 2023-11-28T13:02:40.958 DEBUG:teuthology.orchestra.run.smithi006:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-28T13:02:40.959 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-28T13:02:41.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.088+0000 7ff70afa2700 1 Processor -- start 2023-11-28T13:02:41.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.088+0000 7ff70afa2700 1 -- start start 2023-11-28T13:02:41.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff70afa2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 0x7ff70415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff70afa2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff704160f60 0x7ff70415d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff70afa2700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff704160ad0 con 0x7ff70415c740 2023-11-28T13:02:41.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff70afa2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70415d9d0 con 0x7ff704160f60 2023-11-28T13:02:41.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff70afa2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff704160480 con 0x7ff70406c640 2023-11-28T13:02:41.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff708d3e700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff70415c740 0x7ff70406bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49022/0 (socket says 172.21.15.6:49022) 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff708d3e700 1 -- 172.21.15.6:0/4018463801 learned_addr learned my addr 172.21.15.6:0/4018463801 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff703fff700 1 --2- 172.21.15.6:0/4018463801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 0x7ff70415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff70953f700 1 --2- 172.21.15.6:0/4018463801 >> v2:172.21.15.61:3300/0 conn(0x7ff704160f60 0x7ff70415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff7037fe700 1 -- 172.21.15.6:0/4018463801 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281694853 0 0) 0x7ff704160ad0 con 0x7ff70415c740 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff7037fe700 1 -- 172.21.15.6:0/4018463801 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f40035b0 con 0x7ff70415c740 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff70953f700 1 -- 172.21.15.6:0/4018463801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 msgr2=0x7ff70415cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff70953f700 1 --2- 172.21.15.6:0/4018463801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 0x7ff70415cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff70953f700 1 -- 172.21.15.6:0/4018463801 >> v1:172.21.15.6:6789/0 conn(0x7ff70415c740 legacy=0x7ff70406bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff70953f700 1 -- 172.21.15.6:0/4018463801 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff704160760 con 0x7ff704160f60 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.089+0000 7ff703fff700 1 --2- 172.21.15.6:0/4018463801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 0x7ff70415cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.090+0000 7ff70953f700 1 --2- 172.21.15.6:0/4018463801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff704160f60 0x7ff70415d050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009d90 tx=0x7ff6fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278a1811183c53c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.094 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.090+0000 7ff7037fe700 1 -- 172.21.15.6:0/4018463801 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00e070 con 0x7ff704160f60 2023-11-28T13:02:41.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.090+0000 7ff7037fe700 1 -- 172.21.15.6:0/4018463801 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc00bb20 con 0x7ff704160f60 2023-11-28T13:02:41.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.090+0000 7ff7037fe700 1 -- 172.21.15.6:0/4018463801 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00be90 con 0x7ff704160f60 2023-11-28T13:02:41.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.090+0000 7ff70afa2700 1 -- 172.21.15.6:0/4018463801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff704160f60 msgr2=0x7ff70415d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.090+0000 7ff70afa2700 1 --2- 172.21.15.6:0/4018463801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff704160f60 0x7ff70415d050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009d90 tx=0x7ff6fc00b330 comp rx=0 tx=0).stop 2023-11-28T13:02:41.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.091+0000 7ff70afa2700 1 -- 172.21.15.6:0/4018463801 shutdown_connections 2023-11-28T13:02:41.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.091+0000 7ff70afa2700 1 --2- 172.21.15.6:0/4018463801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff704160f60 0x7ff70415d050 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.091+0000 7ff70afa2700 1 --2- 172.21.15.6:0/4018463801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 0x7ff70415cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.091+0000 7ff70afa2700 1 -- 172.21.15.6:0/4018463801 >> 172.21.15.6:0/4018463801 conn(0x7ff7040d2e70 msgr2=0x7ff7040d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.091+0000 7ff70afa2700 1 -- 172.21.15.6:0/4018463801 shutdown_connections 2023-11-28T13:02:41.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.091+0000 7ff70afa2700 1 -- 172.21.15.6:0/4018463801 wait complete. 2023-11-28T13:02:41.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.092+0000 7ff70afa2700 1 Processor -- start 2023-11-28T13:02:41.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.092+0000 7ff70afa2700 1 -- start start 2023-11-28T13:02:41.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff70afa2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 0x7ff704158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff70afa2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff70415c740 0x7ff7040b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff70afa2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff704160f60 0x7ff7040b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff70afa2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7040b70b0 con 0x7ff70415c740 2023-11-28T13:02:41.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff70afa2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7041589a0 con 0x7ff704160f60 2023-11-28T13:02:41.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff70afa2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7040cbf10 con 0x7ff70406c640 2023-11-28T13:02:41.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff703fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff70415c740 0x7ff7040b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff703fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff70415c740 0x7ff7040b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33478/0 (socket says 172.21.15.6:33478) 2023-11-28T13:02:41.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff703fff700 1 -- 172.21.15.6:0/131558282 learned_addr learned my addr 172.21.15.6:0/131558282 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff708d3e700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 0x7ff704158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff70953f700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff704160f60 0x7ff7040b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff703fff700 1 -- 172.21.15.6:0/131558282 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 msgr2=0x7ff704158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff703fff700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 0x7ff704158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff703fff700 1 -- 172.21.15.6:0/131558282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff704160f60 msgr2=0x7ff7040b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff703fff700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff704160f60 0x7ff7040b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff703fff700 1 -- 172.21.15.6:0/131558282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f00094a0 con 0x7ff70415c740 2023-11-28T13:02:41.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.093+0000 7ff703fff700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff70415c740 0x7ff7040b5ce0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000ebe0 tx=0x7ff6f0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.094+0000 7ff7017fa700 1 -- 172.21.15.6:0/131558282 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000c890 con 0x7ff70415c740 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.094+0000 7ff7017fa700 1 -- 172.21.15.6:0/131558282 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f000c9f0 con 0x7ff70415c740 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.094+0000 7ff70afa2700 1 -- 172.21.15.6:0/131558282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc009a70 con 0x7ff70415c740 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.094+0000 7ff7017fa700 1 -- 172.21.15.6:0/131558282 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000cda0 con 0x7ff70415c740 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.094+0000 7ff70afa2700 1 -- 172.21.15.6:0/131558282 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70415d720 con 0x7ff70415c740 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.095+0000 7ff7017fa700 1 -- 172.21.15.6:0/131558282 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff6f0019030 con 0x7ff70415c740 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.096+0000 7ff7017fa700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6e80682f0 0x7ff6e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.096+0000 7ff7017fa700 1 -- 172.21.15.6:0/131558282 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff6f0085f50 con 0x7ff70415c740 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.096+0000 7ff708d3e700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6e80682f0 0x7ff6e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.096+0000 7ff708d3e700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6e80682f0 0x7ff6e806a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8005910 tx=0x7ff6f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.098+0000 7ff71712bb80 1 Processor -- start 2023-11-28T13:02:41.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.098+0000 7ff71712bb80 1 -- start start 2023-11-28T13:02:41.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d44af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 0x55dc51d6ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.108 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6f1f0 0x55dc51d8a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff71712bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc51c2ec90 con 0x55dc51d6e8e0 2023-11-28T13:02:41.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff71712bb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc51c9d420 con 0x55dc51d6c200 2023-11-28T13:02:41.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff71712bb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc51c96950 con 0x55dc51d6f1f0 2023-11-28T13:02:41.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff708d3e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d44af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff708d3e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d44af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33484/0 (socket says 172.21.15.6:33484) 2023-11-28T13:02:41.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff708d3e700 1 -- 172.21.15.6:0/97571962 learned_addr learned my addr 172.21.15.6:0/97571962 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff70953f700 1 --2- 172.21.15.6:0/97571962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6f1f0 0x55dc51d8a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff703fff700 1 --2- 172.21.15.6:0/97571962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 0x55dc51d6ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.109 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff708d3e700 1 -- 172.21.15.6:0/97571962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6f1f0 msgr2=0x55dc51d8a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff708d3e700 1 --2- 172.21.15.6:0/97571962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6f1f0 0x55dc51d8a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff708d3e700 1 -- 172.21.15.6:0/97571962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 msgr2=0x55dc51d6ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff708d3e700 1 --2- 172.21.15.6:0/97571962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 0x55dc51d6ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff708d3e700 1 -- 172.21.15.6:0/97571962 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc51b186b0 con 0x55dc51d6e8e0 2023-11-28T13:02:41.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff703fff700 1 --2- 172.21.15.6:0/97571962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 0x55dc51d6ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:02:41.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff708d3e700 1 --2- 172.21.15.6:0/97571962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d44af0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8005a70 tx=0x7ff6f8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b777d00a581a4beb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff6ee7fc700 1 -- 172.21.15.6:0/97571962 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8001520 con 0x55dc51d6e8e0 2023-11-28T13:02:41.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff6ee7fc700 1 -- 172.21.15.6:0/97571962 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f8005e30 con 0x55dc51d6e8e0 2023-11-28T13:02:41.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.099+0000 7ff6ee7fc700 1 -- 172.21.15.6:0/97571962 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800e5c0 con 0x55dc51d6e8e0 2023-11-28T13:02:41.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.100+0000 7ff71712bb80 1 -- 172.21.15.6:0/97571962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 msgr2=0x55dc51d44af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.100+0000 7ff71712bb80 1 --2- 172.21.15.6:0/97571962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d44af0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8005a70 tx=0x7ff6f8011000 comp rx=0 tx=0).stop 2023-11-28T13:02:41.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.100+0000 7ff71712bb80 1 -- 172.21.15.6:0/97571962 shutdown_connections 2023-11-28T13:02:41.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.100+0000 7ff71712bb80 1 --2- 172.21.15.6:0/97571962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6f1f0 0x55dc51d8a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.100+0000 7ff71712bb80 1 --2- 172.21.15.6:0/97571962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d44af0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.100+0000 7ff71712bb80 1 --2- 172.21.15.6:0/97571962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 0x55dc51d6ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.100+0000 7ff71712bb80 1 -- 172.21.15.6:0/97571962 >> 172.21.15.6:0/97571962 conn(0x55dc51d41e30 msgr2=0x55dc51d427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.100+0000 7ff71712bb80 1 -- 172.21.15.6:0/97571962 shutdown_connections 2023-11-28T13:02:41.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.100+0000 7ff71712bb80 1 -- 172.21.15.6:0/97571962 wait complete. 2023-11-28T13:02:41.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.101+0000 7ff71712bb80 1 -- start start 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.101+0000 7ff71712bb80 1 Processor -- start 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 0x55dc51d6cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d64560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6f1f0 0x55dc51d64aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff71712bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc51b1a9a0 con 0x55dc51d6e8e0 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff71712bb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc51d919f0 con 0x55dc51d6c200 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff71712bb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc51d91e10 con 0x55dc51d6f1f0 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff71712bb80 10 client.0 _ll_register_callbacks cb 0x55dc51b3c310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff703fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d64560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff703fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d64560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33492/0 (socket says 172.21.15.6:33492) 2023-11-28T13:02:41.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff703fff700 1 -- 172.21.15.6:0/13282040 learned_addr learned my addr 172.21.15.6:0/13282040 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff708d3e700 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 0x55dc51d6cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff70953f700 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6f1f0 0x55dc51d64aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff703fff700 1 -- 172.21.15.6:0/13282040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6f1f0 msgr2=0x55dc51d64aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff703fff700 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6f1f0 0x55dc51d64aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff703fff700 1 -- 172.21.15.6:0/13282040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 msgr2=0x55dc51d6cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff703fff700 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 0x55dc51d6cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.102+0000 7ff703fff700 1 -- 172.21.15.6:0/13282040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f800f040 con 0x55dc51d6e8e0 2023-11-28T13:02:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff703fff700 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d64560 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0054590 tx=0x7ff6f005e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be18957d079cfd50 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff6e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:02:41.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff6e37fe700 1 -- 172.21.15.6:0/13282040 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0052230 con 0x55dc51d6e8e0 2023-11-28T13:02:41.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff6e37fe700 1 -- 172.21.15.6:0/13282040 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff6f0052390 con 0x55dc51d6e8e0 2023-11-28T13:02:41.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff71712bb80 1 -- 172.21.15.6:0/13282040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc51c64210 con 0x55dc51d6e8e0 2023-11-28T13:02:41.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff6e37fe700 1 -- 172.21.15.6:0/13282040 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6f00551a0 con 0x55dc51d6e8e0 2023-11-28T13:02:41.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff71712bb80 10 client.6216 fetch_fsmap learned FSMap version 24 2023-11-28T13:02:41.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff71712bb80 1 -- 172.21.15.6:0/13282040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55dc51ac4e60 con 0x55dc51d6e8e0 2023-11-28T13:02:41.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff6e37fe700 1 -- 172.21.15.6:0/13282040 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc51c64210 con 0x55dc51d6e8e0 2023-11-28T13:02:41.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff6e37fe700 1 -- 172.21.15.6:0/13282040 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff6f0055300 con 0x55dc51d6e8e0 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff71712bb80 10 client.6216 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff71712bb80 10 client.6216 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff71712bb80 20 client.6216 populate_metadata read hostname 'smithi006' 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff71712bb80 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55dc51d8f290 0x55dc51d71350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff71712bb80 4 client.6216 mds_command: new command op to 6003 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff71712bb80 1 -- 172.21.15.6:0/13282040 --> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc51b1afe0 con 0x55dc51d8f290 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.103+0000 7ff708d3e700 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55dc51d8f290 0x55dc51d71350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff6e37fe700 10 client.6216 ms_handle_connect on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff708d3e700 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55dc51d8f290 0x55dc51d71350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb49ed0677aba4ae server_cookie=eb490dd1e534382f in_seq=0 out_seq=0 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff6e37fe700 1 -- 172.21.15.6:0/13282040 <== mds.0 v2:172.21.15.6:6835/451011578 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55dc51b1afe0 con 0x55dc51d8f290 2023-11-28T13:02:41.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff6e37fe700 10 client.6216 handle_command_reply: tid=0 2023-11-28T13:02:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff71712bb80 1 client.6216 shutdown 2023-11-28T13:02:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff71712bb80 1 -- 172.21.15.6:0/13282040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 msgr2=0x55dc51d64560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff71712bb80 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d64560 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0054590 tx=0x7ff6f005e3c0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff71712bb80 1 -- 172.21.15.6:0/13282040 shutdown_connections 2023-11-28T13:02:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff71712bb80 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6f1f0 0x55dc51d64aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff71712bb80 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6c200 0x55dc51d6cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff71712bb80 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55dc51d8f290 0x55dc51d71350 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff71712bb80 1 --2- 172.21.15.6:0/13282040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51d64560 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff6e37fe700 0 client.6216 ms_handle_reset on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:41.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.104+0000 7ff71712bb80 1 -- 172.21.15.6:0/13282040 >> 172.21.15.6:0/13282040 conn(0x55dc51d41e30 msgr2=0x55dc51d44a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.105+0000 7ff71712bb80 1 -- 172.21.15.6:0/13282040 shutdown_connections 2023-11-28T13:02:41.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.105+0000 7ff71712bb80 1 -- 172.21.15.6:0/13282040 wait complete. 2023-11-28T13:02:41.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.105+0000 7ff71712bb80 20 client.6216 trim_cache size 0 max 16384 2023-11-28T13:02:41.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.112+0000 7f1797afa700 1 Processor -- start 2023-11-28T13:02:41.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.112+0000 7f1797afa700 1 -- start start 2023-11-28T13:02:41.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1797afa700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f17900b8590 0x7f179006a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1797afa700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f179006b310 0x7f179006a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1797afa700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17900d00c0 con 0x7f179006ac60 2023-11-28T13:02:41.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1797afa700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179006b070 con 0x7f17900b8590 2023-11-28T13:02:41.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1797afa700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17900cdae0 con 0x7f179006b310 2023-11-28T13:02:41.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1795896700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f17900b8590 0x7f179006a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1796097700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f179006ac60 0x7f1790160040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49054/0 (socket says 172.21.15.6:49054) 2023-11-28T13:02:41.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1796097700 1 -- 172.21.15.6:0/1188945704 learned_addr learned my addr 172.21.15.6:0/1188945704 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:41.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1795095700 1 --2- 172.21.15.6:0/1188945704 >> v2:172.21.15.61:3300/0 conn(0x7f179006b310 0x7f179006a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1795896700 1 -- 172.21.15.6:0/1188945704 >> v2:172.21.15.61:3300/0 conn(0x7f179006b310 msgr2=0x7f179006a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1795896700 1 --2- 172.21.15.6:0/1188945704 >> v2:172.21.15.61:3300/0 conn(0x7f179006b310 0x7f179006a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1795896700 1 -- 172.21.15.6:0/1188945704 >> v1:172.21.15.6:6789/0 conn(0x7f179006ac60 legacy=0x7f1790160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1795896700 1 -- 172.21.15.6:0/1188945704 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17900cfde0 con 0x7f17900b8590 2023-11-28T13:02:41.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.113+0000 7f1795896700 1 --2- 172.21.15.6:0/1188945704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17900b8590 0x7f179006a1e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f179006b280 tx=0x7f178c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bba402232e2b56 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1794894700 1 -- 172.21.15.6:0/1188945704 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c011830 con 0x7f17900b8590 2023-11-28T13:02:41.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1794894700 1 -- 172.21.15.6:0/1188945704 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c011990 con 0x7f17900b8590 2023-11-28T13:02:41.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1794894700 1 -- 172.21.15.6:0/1188945704 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c011d20 con 0x7f17900b8590 2023-11-28T13:02:41.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1797afa700 1 -- 172.21.15.6:0/1188945704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17900b8590 msgr2=0x7f179006a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1797afa700 1 --2- 172.21.15.6:0/1188945704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17900b8590 0x7f179006a1e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f179006b280 tx=0x7f178c009b20 comp rx=0 tx=0).stop 2023-11-28T13:02:41.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1797afa700 1 -- 172.21.15.6:0/1188945704 shutdown_connections 2023-11-28T13:02:41.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1797afa700 1 --2- 172.21.15.6:0/1188945704 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17900b8590 0x7f179006a1e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1797afa700 1 --2- 172.21.15.6:0/1188945704 >> v2:172.21.15.61:3300/0 conn(0x7f179006b310 0x7f179006a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1797afa700 1 -- 172.21.15.6:0/1188945704 >> 172.21.15.6:0/1188945704 conn(0x7f179006cde0 msgr2=0x7f17900b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1797afa700 1 -- 172.21.15.6:0/1188945704 shutdown_connections 2023-11-28T13:02:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.114+0000 7f1797afa700 1 -- 172.21.15.6:0/1188945704 wait complete. 2023-11-28T13:02:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.115+0000 7f1797afa700 1 Processor -- start 2023-11-28T13:02:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.115+0000 7f1797afa700 1 -- start start 2023-11-28T13:02:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1797afa700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f179006ac60 0x7f1790157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1797afa700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f179006b310 0x7f17900b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1797afa700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17900b8590 0x7f1790156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1797afa700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900cbfa0 con 0x7f179006ac60 2023-11-28T13:02:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1797afa700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900cc280 con 0x7f17900b8590 2023-11-28T13:02:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1797afa700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17900cc560 con 0x7f179006b310 2023-11-28T13:02:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1795896700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f179006ac60 0x7f1790157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1795896700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f179006ac60 0x7f1790157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33504/0 (socket says 172.21.15.6:33504) 2023-11-28T13:02:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1795896700 1 -- 172.21.15.6:0/2501487955 learned_addr learned my addr 172.21.15.6:0/2501487955 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1795095700 1 --2- 172.21.15.6:0/2501487955 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f179006b310 0x7f17900b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.116+0000 7f1796097700 1 --2- 172.21.15.6:0/2501487955 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17900b8590 0x7f1790156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f1795896700 1 -- 172.21.15.6:0/2501487955 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f179006b310 msgr2=0x7f17900b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f1795896700 1 --2- 172.21.15.6:0/2501487955 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f179006b310 0x7f17900b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f1795896700 1 -- 172.21.15.6:0/2501487955 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17900b8590 msgr2=0x7f1790156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f1795896700 1 --2- 172.21.15.6:0/2501487955 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17900b8590 0x7f1790156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f1795896700 1 -- 172.21.15.6:0/2501487955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c0094a0 con 0x7f179006ac60 2023-11-28T13:02:41.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f1795896700 1 --2- 172.21.15.6:0/2501487955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f179006ac60 0x7f1790157ee0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f178c009b60 tx=0x7f178c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f17867fc700 1 -- 172.21.15.6:0/2501487955 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c0036a0 con 0x7f179006ac60 2023-11-28T13:02:41.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f17867fc700 1 -- 172.21.15.6:0/2501487955 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c01f070 con 0x7f179006ac60 2023-11-28T13:02:41.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f1797afa700 1 -- 172.21.15.6:0/2501487955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17900cd770 con 0x7f179006ac60 2023-11-28T13:02:41.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f17867fc700 1 -- 172.21.15.6:0/2501487955 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c003a60 con 0x7f179006ac60 2023-11-28T13:02:41.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.117+0000 7f1797afa700 1 -- 172.21.15.6:0/2501487955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17900cd490 con 0x7f179006ac60 2023-11-28T13:02:41.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.118+0000 7f1797afa700 1 -- 172.21.15.6:0/2501487955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f17900cfa70 con 0x7f179006ac60 2023-11-28T13:02:41.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.118+0000 7f17867fc700 1 -- 172.21.15.6:0/2501487955 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f178c00f050 con 0x7f179006ac60 2023-11-28T13:02:41.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.119+0000 7f17867fc700 1 --2- 172.21.15.6:0/2501487955 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f17740682f0 0x7f177406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.119+0000 7f17867fc700 1 -- 172.21.15.6:0/2501487955 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f178c021070 con 0x7f179006ac60 2023-11-28T13:02:41.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.119+0000 7f1795095700 1 --2- 172.21.15.6:0/2501487955 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f17740682f0 0x7f177406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.119+0000 7f17867fc700 1 -- 172.21.15.6:0/2501487955 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+502 (secure 0 0 0) 0x7f178c087a60 con 0x7f179006ac60 2023-11-28T13:02:41.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.119+0000 7f1797afa700 1 -- 172.21.15.6:0/2501487955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f1790067860 con 0x7f179006ac60 2023-11-28T13:02:41.132 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.119+0000 7f1795095700 1 --2- 172.21.15.6:0/2501487955 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f17740682f0 0x7f177406a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f17900cb550 tx=0x7f1780009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.123+0000 7ff71712bb80 1 Processor -- start 2023-11-28T13:02:41.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.123+0000 7ff71712bb80 1 -- start start 2023-11-28T13:02:41.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.123+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6c200 0x55dc51d43c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.123+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6e8e0 0x55dc51d42200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.123+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6f1f0 0x55dc51d625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.123+0000 7ff71712bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc51d8fa60 con 0x55dc51d6c200 2023-11-28T13:02:41.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.123+0000 7ff71712bb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc51d721c0 con 0x55dc51d6f1f0 2023-11-28T13:02:41.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.123+0000 7ff71712bb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc51d837f0 con 0x55dc51d6e8e0 2023-11-28T13:02:41.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff708d3e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6c200 0x55dc51d43c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff708d3e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6c200 0x55dc51d43c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33512/0 (socket says 172.21.15.6:33512) 2023-11-28T13:02:41.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff708d3e700 1 -- 172.21.15.6:0/4215709683 learned_addr learned my addr 172.21.15.6:0/4215709683 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff708d3e700 1 -- 172.21.15.6:0/4215709683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6e8e0 msgr2=0x55dc51d42200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff708d3e700 1 --2- 172.21.15.6:0/4215709683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6e8e0 0x55dc51d42200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff708d3e700 1 -- 172.21.15.6:0/4215709683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6f1f0 msgr2=0x55dc51d625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff708d3e700 1 --2- 172.21.15.6:0/4215709683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6f1f0 0x55dc51d625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff708d3e700 1 -- 172.21.15.6:0/4215709683 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc51c5d210 con 0x55dc51d6c200 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff708d3e700 1 --2- 172.21.15.6:0/4215709683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6c200 0x55dc51d43c30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8007950 tx=0x7ff6f8007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd37db08473e7fd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff6eeffd700 1 -- 172.21.15.6:0/4215709683 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8005e30 con 0x55dc51d6c200 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff6eeffd700 1 -- 172.21.15.6:0/4215709683 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f8019040 con 0x55dc51d6c200 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.124+0000 7ff6eeffd700 1 -- 172.21.15.6:0/4215709683 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f802fa60 con 0x55dc51d6c200 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.125+0000 7ff71712bb80 1 -- 172.21.15.6:0/4215709683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6c200 msgr2=0x55dc51d43c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.125+0000 7ff71712bb80 1 --2- 172.21.15.6:0/4215709683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6c200 0x55dc51d43c30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8007950 tx=0x7ff6f8007000 comp rx=0 tx=0).stop 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.125+0000 7ff71712bb80 1 -- 172.21.15.6:0/4215709683 shutdown_connections 2023-11-28T13:02:41.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.125+0000 7ff71712bb80 1 --2- 172.21.15.6:0/4215709683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6f1f0 0x55dc51d625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.125+0000 7ff71712bb80 1 --2- 172.21.15.6:0/4215709683 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6c200 0x55dc51d43c30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.125+0000 7ff71712bb80 1 --2- 172.21.15.6:0/4215709683 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6e8e0 0x55dc51d42200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.125+0000 7ff71712bb80 1 -- 172.21.15.6:0/4215709683 >> 172.21.15.6:0/4215709683 conn(0x55dc51d41e30 msgr2=0x55dc51d63ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.125+0000 7ff71712bb80 1 -- 172.21.15.6:0/4215709683 shutdown_connections 2023-11-28T13:02:41.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.125+0000 7ff71712bb80 1 -- 172.21.15.6:0/4215709683 wait complete. 2023-11-28T13:02:41.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.125+0000 7ff71712bb80 1 -- start start 2023-11-28T13:02:41.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.126+0000 7ff71712bb80 1 Processor -- start 2023-11-28T13:02:41.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.126+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6c200 0x55dc51d674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51de35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff71712bb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6f1f0 0x55dc51d8d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff71712bb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc51d82640 con 0x55dc51d6e8e0 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff71712bb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc51de66f0 con 0x55dc51d6f1f0 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff71712bb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc51d3dc20 con 0x55dc51d6c200 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff71712bb80 10 client.0 _ll_register_callbacks cb 0x55dc51b3c310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff703fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51de35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff703fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51de35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33520/0 (socket says 172.21.15.6:33520) 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff703fff700 1 -- 172.21.15.6:0/2607584673 learned_addr learned my addr 172.21.15.6:0/2607584673 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff708d3e700 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6c200 0x55dc51d674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff70953f700 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6f1f0 0x55dc51d8d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff703fff700 1 -- 172.21.15.6:0/2607584673 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6c200 msgr2=0x55dc51d674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff703fff700 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6c200 0x55dc51d674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff703fff700 1 -- 172.21.15.6:0/2607584673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6f1f0 msgr2=0x55dc51d8d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff703fff700 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6f1f0 0x55dc51d8d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff703fff700 1 -- 172.21.15.6:0/2607584673 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc51b186b0 con 0x55dc51d6e8e0 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff70953f700 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6f1f0 0x55dc51d8d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff703fff700 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51de35c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0056170 tx=0x7ff6f0058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92a35889f9b5fba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff6e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff6e37fe700 1 -- 172.21.15.6:0/2607584673 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f006d120 con 0x55dc51d6e8e0 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff6e37fe700 1 -- 172.21.15.6:0/2607584673 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff6f005e2d0 con 0x55dc51d6e8e0 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff71712bb80 1 -- 172.21.15.6:0/2607584673 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc51c54230 con 0x55dc51d6e8e0 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff6e37fe700 1 -- 172.21.15.6:0/2607584673 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6f0055fb0 con 0x55dc51d6e8e0 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff71712bb80 10 client.6231 fetch_fsmap learned FSMap version 24 2023-11-28T13:02:41.146 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.127+0000 7ff71712bb80 1 -- 172.21.15.6:0/2607584673 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55dc51aeb2d0 con 0x55dc51d6e8e0 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff6e37fe700 1 -- 172.21.15.6:0/2607584673 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc51c54230 con 0x55dc51d6e8e0 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff6e37fe700 1 -- 172.21.15.6:0/2607584673 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff6f0052030 con 0x55dc51d6e8e0 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff71712bb80 10 client.6231 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff71712bb80 10 client.6231 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff71712bb80 20 client.6231 populate_metadata read hostname 'smithi006' 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff71712bb80 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55dc51d3d520 0x55dc51d8d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff71712bb80 4 client.6231 mds_command: new command op to 6003 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff71712bb80 1 -- 172.21.15.6:0/2607584673 --> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55dc51c3e240 con 0x55dc51d3d520 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff708d3e700 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55dc51d3d520 0x55dc51d8d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff6e37fe700 10 client.6231 ms_handle_connect on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:41.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.128+0000 7ff708d3e700 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55dc51d3d520 0x55dc51d8d590 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f2d641fbd9109d server_cookie=5597c690687cb5aa in_seq=0 out_seq=0 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff6e37fe700 1 -- 172.21.15.6:0/2607584673 <== mds.0 v2:172.21.15.6:6835/451011578 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dc51c3e240 con 0x55dc51d3d520 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff6e37fe700 10 client.6231 handle_command_reply: tid=0 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 client.6231 shutdown 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 -- 172.21.15.6:0/2607584673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 msgr2=0x55dc51de35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51de35c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0056170 tx=0x7ff6f0058000 comp rx=0 tx=0).stop 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 -- 172.21.15.6:0/2607584673 shutdown_connections 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dc51d6f1f0 0x55dc51d8d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55dc51d6c200 0x55dc51d674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x55dc51d3d520 0x55dc51d8d590 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 --2- 172.21.15.6:0/2607584673 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55dc51d6e8e0 0x55dc51de35c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 -- 172.21.15.6:0/2607584673 >> 172.21.15.6:0/2607584673 conn(0x55dc51d41e30 msgr2=0x55dc51d63b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff6e37fe700 0 client.6231 ms_handle_reset on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:41.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 -- 172.21.15.6:0/2607584673 shutdown_connections 2023-11-28T13:02:41.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.129+0000 7ff71712bb80 1 -- 172.21.15.6:0/2607584673 wait complete. 2023-11-28T13:02:41.149 INFO:tasks.ceph.mds.a.smithi006.stderr:2023-11-28T13:02:41.130+0000 7fae10058700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6180=0-4194304@1} caps={6180=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6180 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x557f306a5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T13:02:41.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff71712bb80 20 client.6231 trim_cache size 0 max 16384 2023-11-28T13:02:41.150 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:02:41.150 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-28T13:02:41.150 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "5d9e4fbd-0fda-45c1-976e-db47a494ee20", 2023-11-28T13:02:41.150 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-28T13:02:41.150 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:02:41.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 -- 172.21.15.6:0/131558282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6e80682f0 msgr2=0x7ff6e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6e80682f0 0x7ff6e806a7a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8005910 tx=0x7ff6f8009040 comp rx=0 tx=0).stop 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 -- 172.21.15.6:0/131558282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff70415c740 msgr2=0x7ff7040b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff70415c740 0x7ff7040b5ce0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000ebe0 tx=0x7ff6f0009e60 comp rx=0 tx=0).stop 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 -- 172.21.15.6:0/131558282 shutdown_connections 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff704160f60 0x7ff7040b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6e80682f0 0x7ff6e806a7a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff70406c640 0x7ff704158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 --2- 172.21.15.6:0/131558282 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff70415c740 0x7ff7040b5ce0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 -- 172.21.15.6:0/131558282 >> 172.21.15.6:0/131558282 conn(0x7ff7040d2e70 msgr2=0x7ff7040b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 -- 172.21.15.6:0/131558282 shutdown_connections 2023-11-28T13:02:41.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.130+0000 7ff70afa2700 1 -- 172.21.15.6:0/131558282 wait complete. 2023-11-28T13:02:41.153 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T13:02:41.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.279+0000 7f2c3af29700 1 Processor -- start 2023-11-28T13:02:41.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.279+0000 7f2c3af29700 1 -- start start 2023-11-28T13:02:41.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.279+0000 7f2c3af29700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 0x7f2c340caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.279+0000 7f2c3af29700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c340ce550 con 0x7f2c340b1630 2023-11-28T13:02:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.279+0000 7f2c3af29700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c340ce830 con 0x7f2c340cb3e0 2023-11-28T13:02:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.279+0000 7f2c3af29700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c340cfa60 con 0x7f2c340cf5f0 2023-11-28T13:02:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c38cc5700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f2c340b1630 0x7f2c340b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49078/0 (socket says 172.21.15.6:49078) 2023-11-28T13:02:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c38cc5700 1 -- 172.21.15.6:0/2813369213 learned_addr learned my addr 172.21.15.6:0/2813369213 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c33fff700 1 --2- 172.21.15.6:0/2813369213 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 0x7f2c340caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c337fe700 1 -- 172.21.15.6:0/2813369213 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282959886 0 0) 0x7f2c340ce550 con 0x7f2c340b1630 2023-11-28T13:02:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c337fe700 1 -- 172.21.15.6:0/2813369213 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c240035b0 con 0x7f2c340b1630 2023-11-28T13:02:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c33fff700 1 -- 172.21.15.6:0/2813369213 >> v1:172.21.15.61:6789/0 conn(0x7f2c340cb3e0 legacy=0x7f2c3406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c33fff700 1 -- 172.21.15.6:0/2813369213 >> v1:172.21.15.6:6789/0 conn(0x7f2c340b1630 legacy=0x7f2c340b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c337fe700 1 -- 172.21.15.6:0/2813369213 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400545197 0 0) 0x7f2c340ce830 con 0x7f2c340cb3e0 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c33fff700 1 -- 172.21.15.6:0/2813369213 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c340ceb10 con 0x7f2c340cf5f0 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c33fff700 1 --2- 172.21.15.6:0/2813369213 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 0x7f2c340caea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009870 tx=0x7f2c20009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70b0dbf113e85c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c337fe700 1 -- 172.21.15.6:0/2813369213 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011830 con 0x7f2c340cf5f0 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.280+0000 7f2c337fe700 1 -- 172.21.15.6:0/2813369213 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20011990 con 0x7f2c340cf5f0 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.281+0000 7f2c337fe700 1 -- 172.21.15.6:0/2813369213 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011d20 con 0x7f2c340cf5f0 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.281+0000 7f2c3af29700 1 -- 172.21.15.6:0/2813369213 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 msgr2=0x7f2c340caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.281+0000 7f2c3af29700 1 --2- 172.21.15.6:0/2813369213 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 0x7f2c340caea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009870 tx=0x7f2c20009b80 comp rx=0 tx=0).stop 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.281+0000 7f2c3af29700 1 -- 172.21.15.6:0/2813369213 shutdown_connections 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.281+0000 7f2c3af29700 1 --2- 172.21.15.6:0/2813369213 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 0x7f2c340caea0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.281+0000 7f2c3af29700 1 -- 172.21.15.6:0/2813369213 >> 172.21.15.6:0/2813369213 conn(0x7f2c340cc630 msgr2=0x7f2c340cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.281+0000 7f2c3af29700 1 -- 172.21.15.6:0/2813369213 shutdown_connections 2023-11-28T13:02:41.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.281+0000 7f2c3af29700 1 -- 172.21.15.6:0/2813369213 wait complete. 2023-11-28T13:02:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.283+0000 7f2c3af29700 1 Processor -- start 2023-11-28T13:02:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.283+0000 7f2c3af29700 1 -- start start 2023-11-28T13:02:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c3af29700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c340b1630 0x7f2c34153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c3af29700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c340cb3e0 0x7f2c340b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c3af29700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 0x7f2c340b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c3af29700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c340c4c30 con 0x7f2c340cb3e0 2023-11-28T13:02:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c3af29700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c340c48e0 con 0x7f2c340b1630 2023-11-28T13:02:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c3af29700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c340c7940 con 0x7f2c340cf5f0 2023-11-28T13:02:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c33fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c340cb3e0 0x7f2c340b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c33fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c340cb3e0 0x7f2c340b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33534/0 (socket says 172.21.15.6:33534) 2023-11-28T13:02:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c33fff700 1 -- 172.21.15.6:0/3334541553 learned_addr learned my addr 172.21.15.6:0/3334541553 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c340b1630 0x7f2c34153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c394c6700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 0x7f2c340b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c33fff700 1 -- 172.21.15.6:0/3334541553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 msgr2=0x7f2c340b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c33fff700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 0x7f2c340b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c33fff700 1 -- 172.21.15.6:0/3334541553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c340b1630 msgr2=0x7f2c34153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c33fff700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c340b1630 0x7f2c34153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c33fff700 1 -- 172.21.15.6:0/3334541553 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c200094a0 con 0x7f2c340cb3e0 2023-11-28T13:02:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c340b1630 0x7f2c34153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c33fff700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c340cb3e0 0x7f2c340b44e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009bc0 tx=0x7f2c20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.284+0000 7f2c317fa700 1 -- 172.21.15.6:0/3334541553 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2001a910 con 0x7f2c340cb3e0 2023-11-28T13:02:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.285+0000 7f2c317fa700 1 -- 172.21.15.6:0/3334541553 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2001aa70 con 0x7f2c340cb3e0 2023-11-28T13:02:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.285+0000 7f2c317fa700 1 -- 172.21.15.6:0/3334541553 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2001ae20 con 0x7f2c340cb3e0 2023-11-28T13:02:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.285+0000 7f2c3af29700 1 -- 172.21.15.6:0/3334541553 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c340cf0d0 con 0x7f2c340cb3e0 2023-11-28T13:02:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.285+0000 7f2c3af29700 1 -- 172.21.15.6:0/3334541553 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c340cedf0 con 0x7f2c340cb3e0 2023-11-28T13:02:41.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.286+0000 7f2c317fa700 1 -- 172.21.15.6:0/3334541553 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2c2000f050 con 0x7f2c340cb3e0 2023-11-28T13:02:41.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.286+0000 7f2c317fa700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c180682f0 0x7f2c1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.286+0000 7f2c317fa700 1 -- 172.21.15.6:0/3334541553 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2c20086db0 con 0x7f2c340cb3e0 2023-11-28T13:02:41.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.287+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c180682f0 0x7f2c1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.287+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c180682f0 0x7f2c1806a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2c28005910 tx=0x7f2c28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.288+0000 7f2c470b2b80 1 Processor -- start 2023-11-28T13:02:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.288+0000 7f2c470b2b80 1 -- start start 2023-11-28T13:02:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401beaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c12180 0x562401c14c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c15170 0x562401c307c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401ad4c10 con 0x562401c14860 2023-11-28T13:02:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401b433a0 con 0x562401c12180 2023-11-28T13:02:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x562401b3c8d0 con 0x562401c15170 2023-11-28T13:02:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c38cc5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401beaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c38cc5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401beaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33540/0 (socket says 172.21.15.6:33540) 2023-11-28T13:02:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c38cc5700 1 -- 172.21.15.6:0/3642093272 learned_addr learned my addr 172.21.15.6:0/3642093272 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c394c6700 1 --2- 172.21.15.6:0/3642093272 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c15170 0x562401c307c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.289+0000 7f2c33fff700 1 --2- 172.21.15.6:0/3642093272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c12180 0x562401c14c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.290+0000 7f2c38cc5700 1 -- 172.21.15.6:0/3642093272 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c15170 msgr2=0x562401c307c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.290+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/3642093272 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c15170 0x562401c307c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.290+0000 7f2c38cc5700 1 -- 172.21.15.6:0/3642093272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c12180 msgr2=0x562401c14c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.290+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/3642093272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c12180 0x562401c14c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.290+0000 7f2c38cc5700 1 -- 172.21.15.6:0/3642093272 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624019be630 con 0x562401c14860 2023-11-28T13:02:41.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.290+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/3642093272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401beaa70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2c28005a70 tx=0x7f2c28011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3634e4353c1212de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.290+0000 7f2c1e7fc700 1 -- 172.21.15.6:0/3642093272 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28007560 con 0x562401c14860 2023-11-28T13:02:41.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.290+0000 7f2c1e7fc700 1 -- 172.21.15.6:0/3642093272 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c280076c0 con 0x562401c14860 2023-11-28T13:02:41.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.290+0000 7f2c1e7fc700 1 -- 172.21.15.6:0/3642093272 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800e820 con 0x562401c14860 2023-11-28T13:02:41.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.291+0000 7f2c470b2b80 1 -- 172.21.15.6:0/3642093272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 msgr2=0x562401beaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.291+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/3642093272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401beaa70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2c28005a70 tx=0x7f2c28011000 comp rx=0 tx=0).stop 2023-11-28T13:02:41.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.291+0000 7f2c470b2b80 1 -- 172.21.15.6:0/3642093272 shutdown_connections 2023-11-28T13:02:41.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.291+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/3642093272 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c15170 0x562401c307c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.291+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/3642093272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401beaa70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.291+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/3642093272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c12180 0x562401c14c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.291+0000 7f2c470b2b80 1 -- 172.21.15.6:0/3642093272 >> 172.21.15.6:0/3642093272 conn(0x562401be7db0 msgr2=0x562401be8740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.291+0000 7f2c470b2b80 1 -- 172.21.15.6:0/3642093272 shutdown_connections 2023-11-28T13:02:41.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.291+0000 7f2c470b2b80 1 -- 172.21.15.6:0/3642093272 wait complete. 2023-11-28T13:02:41.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.291+0000 7f2c470b2b80 1 -- start start 2023-11-28T13:02:41.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c470b2b80 1 Processor -- start 2023-11-28T13:02:41.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 0x562401c265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c15170 0x562401c16310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624019c0920 con 0x562401c14860 2023-11-28T13:02:41.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401c27440 con 0x562401c15170 2023-11-28T13:02:41.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x562401c062a0 con 0x562401c12180 2023-11-28T13:02:41.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c470b2b80 10 client.0 _ll_register_callbacks cb 0x5624019e2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:41.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c33fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c33fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33548/0 (socket says 172.21.15.6:33548) 2023-11-28T13:02:41.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.292+0000 7f2c33fff700 1 -- 172.21.15.6:0/2500181289 learned_addr learned my addr 172.21.15.6:0/2500181289 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c394c6700 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c15170 0x562401c16310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 0x562401c265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c33fff700 1 -- 172.21.15.6:0/2500181289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 msgr2=0x562401c265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c33fff700 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 0x562401c265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c33fff700 1 -- 172.21.15.6:0/2500181289 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c15170 msgr2=0x562401c16310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c33fff700 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c15170 0x562401c16310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c33fff700 1 -- 172.21.15.6:0/2500181289 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2800f040 con 0x562401c14860 2023-11-28T13:02:41.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c394c6700 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c15170 0x562401c16310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:41.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 0x562401c265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:41.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c33fff700 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26b10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2c20000c00 tx=0x7f2c20056dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abc649421ef7ab1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c137fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:02:41.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c137fe700 1 -- 172.21.15.6:0/2500181289 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2006e3c0 con 0x562401c14860 2023-11-28T13:02:41.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c137fe700 1 -- 172.21.15.6:0/2500181289 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2c200560f0 con 0x562401c14860 2023-11-28T13:02:41.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c470b2b80 1 -- 172.21.15.6:0/2500181289 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562401b0a190 con 0x562401c14860 2023-11-28T13:02:41.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c137fe700 1 -- 172.21.15.6:0/2500181289 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c200562b0 con 0x562401c14860 2023-11-28T13:02:41.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c470b2b80 10 client.6243 fetch_fsmap learned FSMap version 24 2023-11-28T13:02:41.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c470b2b80 1 -- 172.21.15.6:0/2500181289 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56240196ade0 con 0x562401c14860 2023-11-28T13:02:41.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c137fe700 1 -- 172.21.15.6:0/2500181289 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562401b0a190 con 0x562401c14860 2023-11-28T13:02:41.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c137fe700 1 -- 172.21.15.6:0/2500181289 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2c20005a20 con 0x562401c14860 2023-11-28T13:02:41.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c470b2b80 10 client.6243 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T13:02:41.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c470b2b80 10 client.6243 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-28T13:02:41.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c470b2b80 20 client.6243 populate_metadata read hostname 'smithi006' 2023-11-28T13:02:41.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.293+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x562401c04db0 0x562401c16d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.294+0000 7f2c470b2b80 4 client.6243 mds_command: new command op to 6003 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:02:41.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.294+0000 7f2c470b2b80 1 -- 172.21.15.6:0/2500181289 --> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624019c0f60 con 0x562401c04db0 2023-11-28T13:02:41.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.294+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x562401c04db0 0x562401c16d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.294+0000 7f2c137fe700 10 client.6243 ms_handle_connect on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:41.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.294+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x562401c04db0 0x562401c16d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db4e8059906257d0 server_cookie=ea77e7a8a237f2d9 in_seq=0 out_seq=0 2023-11-28T13:02:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.294+0000 7f2c137fe700 1 -- 172.21.15.6:0/2500181289 <== mds.0 v2:172.21.15.6:6835/451011578 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5624019c0f60 con 0x562401c04db0 2023-11-28T13:02:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.294+0000 7f2c137fe700 10 client.6243 handle_command_reply: tid=0 2023-11-28T13:02:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.294+0000 7f2c470b2b80 1 client.6243 shutdown 2023-11-28T13:02:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 1 -- 172.21.15.6:0/2500181289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 msgr2=0x562401c26b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26b10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2c20000c00 tx=0x7f2c20056dd0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 1 -- 172.21.15.6:0/2500181289 shutdown_connections 2023-11-28T13:02:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c15170 0x562401c16310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 0x562401c265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x562401c04db0 0x562401c16d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/2500181289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26b10 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c137fe700 0 client.6243 ms_handle_reset on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:41.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 1 -- 172.21.15.6:0/2500181289 >> 172.21.15.6:0/2500181289 conn(0x562401be7db0 msgr2=0x562401be3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 1 -- 172.21.15.6:0/2500181289 shutdown_connections 2023-11-28T13:02:41.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 1 -- 172.21.15.6:0/2500181289 wait complete. 2023-11-28T13:02:41.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.295+0000 7f2c470b2b80 20 client.6243 trim_cache size 0 max 16384 2023-11-28T13:02:41.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.311+0000 7f2c470b2b80 1 Processor -- start 2023-11-28T13:02:41.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.312+0000 7f2c470b2b80 1 -- start start 2023-11-28T13:02:41.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.312+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c12180 0x562401c265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.312+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.312+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c15170 0x562401c024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.312+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401bdfa40 con 0x562401c14860 2023-11-28T13:02:41.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.312+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401c16670 con 0x562401c12180 2023-11-28T13:02:41.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.312+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x562401c05ff0 con 0x562401c15170 2023-11-28T13:02:41.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.312+0000 7f2c33fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.323 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.312+0000 7f2c33fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33556/0 (socket says 172.21.15.6:33556) 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.312+0000 7f2c33fff700 1 -- 172.21.15.6:0/1857112222 learned_addr learned my addr 172.21.15.6:0/1857112222 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/1857112222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c12180 0x562401c265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c394c6700 1 --2- 172.21.15.6:0/1857112222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c15170 0x562401c024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c33fff700 1 -- 172.21.15.6:0/1857112222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c15170 msgr2=0x562401c024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c33fff700 1 --2- 172.21.15.6:0/1857112222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c15170 0x562401c024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c33fff700 1 -- 172.21.15.6:0/1857112222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c12180 msgr2=0x562401c265b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c33fff700 1 --2- 172.21.15.6:0/1857112222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c12180 0x562401c265b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c33fff700 1 -- 172.21.15.6:0/1857112222 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562401b03190 con 0x562401c14860 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c33fff700 1 --2- 172.21.15.6:0/1857112222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26af0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2c20014040 tx=0x7f2c2002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee49860980db61ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c1effd700 1 -- 172.21.15.6:0/1857112222 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2006e1c0 con 0x562401c14860 2023-11-28T13:02:41.324 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c1effd700 1 -- 172.21.15.6:0/1857112222 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2006e320 con 0x562401c14860 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c1effd700 1 -- 172.21.15.6:0/1857112222 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200562f0 con 0x562401c14860 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1857112222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 msgr2=0x562401c26af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/1857112222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26af0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2c20014040 tx=0x7f2c2002e000 comp rx=0 tx=0).stop 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1857112222 shutdown_connections 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/1857112222 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c15170 0x562401c024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/1857112222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c12180 0x562401c265b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/1857112222 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c14860 0x562401c26af0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.313+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1857112222 >> 172.21.15.6:0/1857112222 conn(0x562401be7db0 msgr2=0x562401be6150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.314+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1857112222 shutdown_connections 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.314+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1857112222 wait complete. 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.314+0000 7f2c470b2b80 1 -- start start 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.314+0000 7f2c470b2b80 1 Processor -- start 2023-11-28T13:02:41.325 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 0x562401c25a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c14860 0x562401c26430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c470b2b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c15170 0x562401c05420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401c99490 con 0x562401c15170 2023-11-28T13:02:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562401c0a090 con 0x562401c14860 2023-11-28T13:02:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c470b2b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x562401c7dea0 con 0x562401c12180 2023-11-28T13:02:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c470b2b80 10 client.0 _ll_register_callbacks cb 0x5624019e2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:02:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c394c6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c15170 0x562401c05420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c394c6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c15170 0x562401c05420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33564/0 (socket says 172.21.15.6:33564) 2023-11-28T13:02:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c394c6700 1 -- 172.21.15.6:0/1122198176 learned_addr learned my addr 172.21.15.6:0/1122198176 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:41.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 0x562401c25a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c33fff700 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c14860 0x562401c26430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c394c6700 1 -- 172.21.15.6:0/1122198176 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 msgr2=0x562401c25a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c394c6700 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 0x562401c25a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c394c6700 1 -- 172.21.15.6:0/1122198176 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c14860 msgr2=0x562401c26430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c394c6700 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c14860 0x562401c26430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.315+0000 7f2c394c6700 1 -- 172.21.15.6:0/1122198176 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56240196ade0 con 0x562401c15170 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c394c6700 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c15170 0x562401c05420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0012e0 tx=0x7f2c2c005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb459c2cc1326a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c137fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c137fe700 1 -- 172.21.15.6:0/1122198176 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c0058a0 con 0x562401c15170 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c137fe700 1 -- 172.21.15.6:0/1122198176 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2c2c016630 con 0x562401c15170 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1122198176 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562401afa1b0 con 0x562401c15170 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c137fe700 1 -- 172.21.15.6:0/1122198176 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c2c0167f0 con 0x562401c15170 2023-11-28T13:02:41.327 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c470b2b80 10 client.6249 fetch_fsmap learned FSMap version 24 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1122198176 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x562401991250 con 0x562401c15170 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c137fe700 1 -- 172.21.15.6:0/1122198176 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562401afa1b0 con 0x562401c15170 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c137fe700 1 -- 172.21.15.6:0/1122198176 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 24) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2c2c016950 con 0x562401c15170 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c470b2b80 10 client.6249 fetch_fsmap finished waiting for FSMap version 24 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c470b2b80 10 client.6249 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c470b2b80 20 client.6249 populate_metadata read hostname 'smithi006' 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x562401c7b060 0x562401c05960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c470b2b80 4 client.6249 mds_command: new command op to 6003 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1122198176 --> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562401ae41c0 con 0x562401c7b060 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.316+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x562401c7b060 0x562401c05960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c137fe700 10 client.6249 ms_handle_connect on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c38cc5700 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x562401c7b060 0x562401c05960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e0ad69d5b2276b server_cookie=5dd8ac4b00cd958a in_seq=0 out_seq=0 2023-11-28T13:02:41.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c137fe700 1 -- 172.21.15.6:0/1122198176 <== mds.0 v2:172.21.15.6:6835/451011578 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562401ae41c0 con 0x562401c7b060 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c137fe700 10 client.6249 handle_command_reply: tid=0 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 client.6249 shutdown 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1122198176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c15170 msgr2=0x562401c05420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c15170 0x562401c05420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0012e0 tx=0x7f2c2c005330 comp rx=0 tx=0).stop 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1122198176 shutdown_connections 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.6:6835/451011578,v1:172.21.15.6:6837/451011578] conn(0x562401c7b060 0x562401c05960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562401c15170 0x562401c05420 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c137fe700 0 client.6249 ms_handle_reset on v2:172.21.15.6:6835/451011578 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562401c12180 0x562401c25a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 --2- 172.21.15.6:0/1122198176 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562401c14860 0x562401c26430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1122198176 >> 172.21.15.6:0/1122198176 conn(0x562401be7db0 msgr2=0x562401c117a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1122198176 shutdown_connections 2023-11-28T13:02:41.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 1 -- 172.21.15.6:0/1122198176 wait complete. 2023-11-28T13:02:41.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.317+0000 7f2c470b2b80 20 client.6249 trim_cache size 0 max 16384 2023-11-28T13:02:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:02:41.330 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-28T13:02:41.330 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-28T13:02:41.330 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:02:41.331 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 -- 172.21.15.6:0/3334541553 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c180682f0 msgr2=0x7f2c1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c180682f0 0x7f2c1806a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2c28005910 tx=0x7f2c28009040 comp rx=0 tx=0).stop 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 -- 172.21.15.6:0/3334541553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c340cb3e0 msgr2=0x7f2c340b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c340cb3e0 0x7f2c340b44e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009bc0 tx=0x7f2c20004750 comp rx=0 tx=0).stop 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 -- 172.21.15.6:0/3334541553 shutdown_connections 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f2c180682f0 0x7f2c1806a7a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f2c340cf5f0 0x7f2c340b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2c340b1630 0x7f2c34153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 --2- 172.21.15.6:0/3334541553 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2c340cb3e0 0x7f2c340b44e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 -- 172.21.15.6:0/3334541553 >> 172.21.15.6:0/3334541553 conn(0x7f2c340cc630 msgr2=0x7f2c340b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 -- 172.21.15.6:0/3334541553 shutdown_connections 2023-11-28T13:02:41.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.318+0000 7f2c3af29700 1 -- 172.21.15.6:0/3334541553 wait complete. 2023-11-28T13:02:41.333 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T13:02:41.434 DEBUG:tasks.ceph_test_case:No log hits yet, waiting... 2023-11-28T13:02:41.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:41.967+0000 7f17867fc700 1 -- 172.21.15.6:0/2501487955 <== mon.0 v2:172.21.15.6:3300/0 7 ==== log(6 entries from seq 1 at 2023-11-28T13:02:41.130063+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f178c05fc10 con 0x7f179006ac60 2023-11-28T13:02:41.969 INFO:teuthology.orchestra.run.smithi006.stdout: cluster: 2023-11-28T13:02:41.969 INFO:teuthology.orchestra.run.smithi006.stdout: id: dc643718-e974-4d8a-b829-ff6ced554687 2023-11-28T13:02:41.969 INFO:teuthology.orchestra.run.smithi006.stdout: health: HEALTH_OK 2023-11-28T13:02:41.969 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:41.969 INFO:teuthology.orchestra.run.smithi006.stdout: services: 2023-11-28T13:02:41.970 INFO:teuthology.orchestra.run.smithi006.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-28T13:02:41.970 INFO:teuthology.orchestra.run.smithi006.stdout: mgr: y(active, since 2m), standbys: x 2023-11-28T13:02:41.970 INFO:teuthology.orchestra.run.smithi006.stdout: mds: 1/1 daemons up, 3 standby 2023-11-28T13:02:41.970 INFO:teuthology.orchestra.run.smithi006.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2023-11-28T13:02:41.970 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:41.970 INFO:teuthology.orchestra.run.smithi006.stdout: data: 2023-11-28T13:02:41.970 INFO:teuthology.orchestra.run.smithi006.stdout: volumes: 1/1 healthy 2023-11-28T13:02:41.970 INFO:teuthology.orchestra.run.smithi006.stdout: pools: 4 pools, 105 pgs 2023-11-28T13:02:41.970 INFO:teuthology.orchestra.run.smithi006.stdout: objects: 23 objects, 579 KiB 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout: usage: 74 MiB used, 720 GiB / 720 GiB avail 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout: pgs: 105 active+clean 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout: io: 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout: client: 1.8 KiB/s wr, 0 op/s rd, 5 op/s wr 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout:2023-11-28T13:02:41.130063+0000 mds.a [INF] scrub queued for path: / 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout:2023-11-28T13:02:41.130067+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout:2023-11-28T13:02:41.130072+0000 mds.a [INF] scrub summary: active paths [/] 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout:2023-11-28T13:02:41.131095+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-28T13:02:41.971 INFO:teuthology.orchestra.run.smithi006.stdout:2023-11-28T13:02:41.131106+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-28T13:02:41.972 INFO:teuthology.orchestra.run.smithi006.stdout:2023-11-28T13:02:41.131434+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2023-11-28T13:02:42.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:42.966+0000 7f17867fc700 1 -- 172.21.15.6:0/2501487955 <== mon.0 v2:172.21.15.6:3300/0 8 ==== log(1 entries from seq 7 at 2023-11-28T13:02:41.226209+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f178c087a20 con 0x7f179006ac60 2023-11-28T13:02:42.968 INFO:teuthology.orchestra.run.smithi006.stdout:2023-11-28T13:02:41.226209+0000 mds.a [INF] scrub summary: idle 2023-11-28T13:02:56.446 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:02:56.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.623+0000 7f08709d5700 1 Processor -- start 2023-11-28T13:02:56.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.623+0000 7f08709d5700 1 -- start start 2023-11-28T13:02:56.632 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.624+0000 7f08709d5700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f086c158a30 0x7f086c06aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:56.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.624+0000 7f08709d5700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086c06ed80 con 0x7f086c15c680 2023-11-28T13:02:56.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.624+0000 7f08709d5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086c06f060 con 0x7f086c0ce150 2023-11-28T13:02:56.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.624+0000 7f08709d5700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f086c0cd950 con 0x7f086c158a30 2023-11-28T13:02:56.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.624+0000 7f086a59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f086c158a30 0x7f086c06aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:56.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086ad9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f086c15c680 0x7f086c15cab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49124/0 (socket says 172.21.15.6:49124) 2023-11-28T13:02:56.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086a59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f086c158a30 0x7f086c06aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33568/0 (socket says 172.21.15.6:33568) 2023-11-28T13:02:56.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086ad9d700 1 -- 172.21.15.6:0/3255769471 learned_addr learned my addr 172.21.15.6:0/3255769471 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:56.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086a59c700 1 -- 172.21.15.6:0/3255769471 >> v1:172.21.15.61:6789/0 conn(0x7f086c0ce150 legacy=0x7f086c0ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:56.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086a59c700 1 -- 172.21.15.6:0/3255769471 >> v1:172.21.15.6:6789/0 conn(0x7f086c15c680 legacy=0x7f086c15cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:56.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086a59c700 1 -- 172.21.15.6:0/3255769471 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c0cdc30 con 0x7f086c158a30 2023-11-28T13:02:56.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086959a700 1 -- 172.21.15.6:0/3255769471 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643652922 0 0) 0x7f086c06ed80 con 0x7f086c15c680 2023-11-28T13:02:56.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086a59c700 1 --2- 172.21.15.6:0/3255769471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 0x7f086c06aa70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0860009840 tx=0x7f0860009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc047f174672884 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:56.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086959a700 1 -- 172.21.15.6:0/3255769471 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860004230 con 0x7f086c158a30 2023-11-28T13:02:56.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086959a700 1 -- 172.21.15.6:0/3255769471 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0860004390 con 0x7f086c158a30 2023-11-28T13:02:56.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.625+0000 7f086959a700 1 -- 172.21.15.6:0/3255769471 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600036a0 con 0x7f086c158a30 2023-11-28T13:02:56.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.626+0000 7f08709d5700 1 -- 172.21.15.6:0/3255769471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 msgr2=0x7f086c06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:56.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.626+0000 7f08709d5700 1 --2- 172.21.15.6:0/3255769471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 0x7f086c06aa70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0860009840 tx=0x7f0860009b50 comp rx=0 tx=0).stop 2023-11-28T13:02:56.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.626+0000 7f08709d5700 1 -- 172.21.15.6:0/3255769471 shutdown_connections 2023-11-28T13:02:56.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.626+0000 7f08709d5700 1 --2- 172.21.15.6:0/3255769471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 0x7f086c06aa70 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.626+0000 7f08709d5700 1 -- 172.21.15.6:0/3255769471 >> 172.21.15.6:0/3255769471 conn(0x7f086c0b0dd0 msgr2=0x7f086c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:56.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.626+0000 7f08709d5700 1 -- 172.21.15.6:0/3255769471 shutdown_connections 2023-11-28T13:02:56.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.626+0000 7f08709d5700 1 -- 172.21.15.6:0/3255769471 wait complete. 2023-11-28T13:02:56.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.627+0000 7f08709d5700 1 Processor -- start 2023-11-28T13:02:56.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.627+0000 7f08709d5700 1 -- start start 2023-11-28T13:02:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f08709d5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f086c0ce150 0x7f086c0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f08709d5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 0x7f086c0c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f08709d5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f086c15c680 0x7f086c0b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f08709d5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c0b2110 con 0x7f086c158a30 2023-11-28T13:02:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f08709d5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c0c41d0 con 0x7f086c15c680 2023-11-28T13:02:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f08709d5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086c0abea0 con 0x7f086c0ce150 2023-11-28T13:02:56.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f0869d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 0x7f086c0c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:56.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f0869d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 0x7f086c0c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33574/0 (socket says 172.21.15.6:33574) 2023-11-28T13:02:56.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f0869d9b700 1 -- 172.21.15.6:0/2277286208 learned_addr learned my addr 172.21.15.6:0/2277286208 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:56.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f086a59c700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f086c0ce150 0x7f086c0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:56.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.628+0000 7f086ad9d700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f086c15c680 0x7f086c0b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:56.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f0869d9b700 1 -- 172.21.15.6:0/2277286208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f086c0ce150 msgr2=0x7f086c0ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:56.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f0869d9b700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f086c0ce150 0x7f086c0ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f0869d9b700 1 -- 172.21.15.6:0/2277286208 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f086c15c680 msgr2=0x7f086c0b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:56.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f0869d9b700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f086c15c680 0x7f086c0b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f0869d9b700 1 -- 172.21.15.6:0/2277286208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08600094a0 con 0x7f086c158a30 2023-11-28T13:02:56.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f086a59c700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f086c0ce150 0x7f086c0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:56.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f0869d9b700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 0x7f086c0c3c90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f085c00c4a0 tx=0x7f085c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:56.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f085affd700 1 -- 172.21.15.6:0/2277286208 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c0054e0 con 0x7f086c158a30 2023-11-28T13:02:56.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f085affd700 1 -- 172.21.15.6:0/2277286208 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c003680 con 0x7f086c158a30 2023-11-28T13:02:56.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f085affd700 1 -- 172.21.15.6:0/2277286208 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c003a30 con 0x7f086c158a30 2023-11-28T13:02:56.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f08709d5700 1 -- 172.21.15.6:0/2277286208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c06e9c0 con 0x7f086c158a30 2023-11-28T13:02:56.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.629+0000 7f08709d5700 1 -- 172.21.15.6:0/2277286208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086c06f340 con 0x7f086c158a30 2023-11-28T13:02:56.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.630+0000 7f08709d5700 1 -- 172.21.15.6:0/2277286208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086c06eb50 con 0x7f086c158a30 2023-11-28T13:02:56.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.630+0000 7f085affd700 1 -- 172.21.15.6:0/2277286208 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f085c010070 con 0x7f086c158a30 2023-11-28T13:02:56.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.631+0000 7f085affd700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f084c0682f0 0x7f084c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:56.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.631+0000 7f085affd700 1 -- 172.21.15.6:0/2277286208 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f085c084d80 con 0x7f086c158a30 2023-11-28T13:02:56.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.631+0000 7f086a59c700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f084c0682f0 0x7f084c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:56.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.631+0000 7f086a59c700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f084c0682f0 0x7f084c06a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0860000c00 tx=0x7f086001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:56.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.635+0000 7f085affd700 1 -- 172.21.15.6:0/2277286208 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f085c0513b0 con 0x7f086c158a30 2023-11-28T13:02:56.835 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:02:56.837 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:02:56.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.874+0000 7f08709d5700 1 -- 172.21.15.6:0/2277286208 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f086c0675a0 con 0x7f086c158a30 2023-11-28T13:02:56.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.875+0000 7f085affd700 1 -- 172.21.15.6:0/2277286208 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4636 (secure 0 0 0) 0x7f085c0549d0 con 0x7f086c158a30 2023-11-28T13:02:56.876 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:56.876 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4004512137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4004512137},{"type":"v1","addr":"172.21.15.61:6833","nonce":4004512137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/1313775424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":1313775424},{"type":"v1","addr":"172.21.15.61:6835","nonce":1313775424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/233781200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":233781200},{"type":"v1","addr":"172.21.15.6:6836","nonce":233781200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:31.562404+0000","modified":"2023-11-28T13:02:34.939442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/451011578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":451011578},{"type":"v1","addr":"172.21.15.6:6837","nonce":451011578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:02:56.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 -- 172.21.15.6:0/2277286208 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f084c0682f0 msgr2=0x7f084c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:56.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f084c0682f0 0x7f084c06a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0860000c00 tx=0x7f086001c040 comp rx=0 tx=0).stop 2023-11-28T13:02:56.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 -- 172.21.15.6:0/2277286208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 msgr2=0x7f086c0c3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:56.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 0x7f086c0c3c90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f085c00c4a0 tx=0x7f085c00c7b0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 -- 172.21.15.6:0/2277286208 shutdown_connections 2023-11-28T13:02:56.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f086c15c680 0x7f086c0b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f084c0682f0 0x7f084c06a7a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f086c0ce150 0x7f086c0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 --2- 172.21.15.6:0/2277286208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f086c158a30 0x7f086c0c3c90 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:56.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 -- 172.21.15.6:0/2277286208 >> 172.21.15.6:0/2277286208 conn(0x7f086c0b0dd0 msgr2=0x7f086c06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:56.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 -- 172.21.15.6:0/2277286208 shutdown_connections 2023-11-28T13:02:56.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:56.877+0000 7f08709d5700 1 -- 172.21.15.6:0/2277286208 wait complete. 2023-11-28T13:02:56.880 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2023-11-28T13:02:56.891 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-28T13:02:57.097 INFO:teuthology.orchestra.run.smithi006.stdout:{"message":"","return_code":0} 2023-11-28T13:02:57.099 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:02:57.100 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T13:02:57.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.146+0000 7f8040d9a080 1 Processor -- start 2023-11-28T13:02:57.148 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.146+0000 7f8040d9a080 1 -- start start 2023-11-28T13:02:57.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f8040d9a080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e0fba45b0 0x562e0fc20050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f8040d9a080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba4dc0 0x562e0fc378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f8040d9a080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562e0fc217f0 0x562e0fb8bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f8040d9a080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0fa9afe0 con 0x562e0fba4dc0 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f8040d9a080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0fb60050 con 0x562e0fba45b0 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f8040d9a080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e0fc37df0 con 0x562e0fc217f0 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f803669c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba4dc0 0x562e0fc378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f803669c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba4dc0 0x562e0fc378b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33590/0 (socket says 172.21.15.6:33590) 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f803669c700 1 -- 172.21.15.6:0/2534939687 learned_addr learned my addr 172.21.15.6:0/2534939687 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f803669c700 1 -- 172.21.15.6:0/2534939687 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562e0fc217f0 msgr2=0x562e0fb8bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f803669c700 1 --2- 172.21.15.6:0/2534939687 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562e0fc217f0 0x562e0fb8bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f803669c700 1 -- 172.21.15.6:0/2534939687 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e0fba45b0 msgr2=0x562e0fc20050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.147+0000 7f803669c700 1 --2- 172.21.15.6:0/2534939687 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e0fba45b0 0x562e0fc20050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f803669c700 1 -- 172.21.15.6:0/2534939687 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028005de0 con 0x562e0fba4dc0 2023-11-28T13:02:57.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f803669c700 1 --2- 172.21.15.6:0/2534939687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba4dc0 0x562e0fc378b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f802800c010 tx=0x7f802800d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43aaa9f7644fd267 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f803569a700 1 -- 172.21.15.6:0/2534939687 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028008c20 con 0x562e0fba4dc0 2023-11-28T13:02:57.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f803569a700 1 -- 172.21.15.6:0/2534939687 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8028008da0 con 0x562e0fba4dc0 2023-11-28T13:02:57.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f803569a700 1 -- 172.21.15.6:0/2534939687 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802801c670 con 0x562e0fba4dc0 2023-11-28T13:02:57.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f8040d9a080 1 -- 172.21.15.6:0/2534939687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba4dc0 msgr2=0x562e0fc378b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f8040d9a080 1 --2- 172.21.15.6:0/2534939687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba4dc0 0x562e0fc378b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f802800c010 tx=0x7f802800d000 comp rx=0 tx=0).stop 2023-11-28T13:02:57.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f8040d9a080 1 -- 172.21.15.6:0/2534939687 shutdown_connections 2023-11-28T13:02:57.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f8040d9a080 1 --2- 172.21.15.6:0/2534939687 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562e0fc217f0 0x562e0fb8bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f8040d9a080 1 --2- 172.21.15.6:0/2534939687 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e0fba45b0 0x562e0fc20050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f8040d9a080 1 --2- 172.21.15.6:0/2534939687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba4dc0 0x562e0fc378b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f8040d9a080 1 -- 172.21.15.6:0/2534939687 >> 172.21.15.6:0/2534939687 conn(0x562e0fba77e0 msgr2=0x562e0fb60cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:57.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f8040d9a080 1 -- 172.21.15.6:0/2534939687 shutdown_connections 2023-11-28T13:02:57.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.148+0000 7f8040d9a080 1 -- 172.21.15.6:0/2534939687 wait complete. 2023-11-28T13:02:57.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.149+0000 7f8040d9a080 1 Processor -- start 2023-11-28T13:02:57.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.149+0000 7f8040d9a080 1 -- start start 2023-11-28T13:02:57.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.150+0000 7f8040d9a080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba45b0 0x562e0fc1c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.150+0000 7f8040d9a080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e0fba4dc0 0x562e0fb60430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.150+0000 7f8040d9a080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562e0fc217f0 0x562e0fb60970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.150+0000 7f8040d9a080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0fc18240 con 0x562e0fba45b0 2023-11-28T13:02:57.158 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.150+0000 7f8040d9a080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0fc37a70 con 0x562e0fba4dc0 2023-11-28T13:02:57.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.150+0000 7f8040d9a080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e0fc1c640 con 0x562e0fc217f0 2023-11-28T13:02:57.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.150+0000 7f8036e9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba45b0 0x562e0fc1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.150+0000 7f8036e9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba45b0 0x562e0fc1c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33594/0 (socket says 172.21.15.6:33594) 2023-11-28T13:02:57.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.150+0000 7f8036e9d700 1 -- 172.21.15.6:0/873982365 learned_addr learned my addr 172.21.15.6:0/873982365 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:57.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f8036e9d700 1 -- 172.21.15.6:0/873982365 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562e0fc217f0 msgr2=0x562e0fb60970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f803669c700 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e0fba4dc0 0x562e0fb60430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f8036e9d700 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562e0fc217f0 0x562e0fb60970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f8036e9d700 1 -- 172.21.15.6:0/873982365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e0fba4dc0 msgr2=0x562e0fb60430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f8036e9d700 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e0fba4dc0 0x562e0fb60430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f8036e9d700 1 -- 172.21.15.6:0/873982365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e0facd4f0 con 0x562e0fba45b0 2023-11-28T13:02:57.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f8036e9d700 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba45b0 0x562e0fc1c100 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f802c00af30 tx=0x7f802c008650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f801ffff700 1 -- 172.21.15.6:0/873982365 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c00c620 con 0x562e0fba45b0 2023-11-28T13:02:57.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f801ffff700 1 -- 172.21.15.6:0/873982365 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c00c7a0 con 0x562e0fba45b0 2023-11-28T13:02:57.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f8040d9a080 1 -- 172.21.15.6:0/873982365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562e0fab3120 con 0x562e0fba45b0 2023-11-28T13:02:57.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f801ffff700 1 -- 172.21.15.6:0/873982365 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c00cb50 con 0x562e0fba45b0 2023-11-28T13:02:57.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.151+0000 7f8040d9a080 1 -- 172.21.15.6:0/873982365 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e0faa6830 con 0x562e0fba45b0 2023-11-28T13:02:57.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.152+0000 7f801ffff700 1 -- 172.21.15.6:0/873982365 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f802c00ccb0 con 0x562e0fba45b0 2023-11-28T13:02:57.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.153+0000 7f801ffff700 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8024066c10 0x7f80240690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.153+0000 7f801ffff700 1 -- 172.21.15.6:0/873982365 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f802c084a80 con 0x562e0fba45b0 2023-11-28T13:02:57.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.153+0000 7f803669c700 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8024066c10 0x7f80240690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.153+0000 7f8040d9a080 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x562e0fb8ba90 0x562e0fcd19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.153+0000 7f8040d9a080 1 -- 172.21.15.6:0/873982365 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54) v8 -- 0x562e0fb8b560 con 0x562e0fb8ba90 2023-11-28T13:02:57.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.153+0000 7f803769e700 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x562e0fb8ba90 0x562e0fcd19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.153+0000 7f803669c700 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8024066c10 0x7f80240690c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8028023000 tx=0x7f8028008000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f803769e700 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x562e0fb8ba90 0x562e0fcd19f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f803769e700 1 -- 172.21.15.6:0/873982365 <== osd.5 v2:172.21.15.61:6808/103780 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f803000b4e0 con 0x562e0fb8ba90 2023-11-28T13:02:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f8040d9a080 1 -- 172.21.15.6:0/873982365 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x562e0fb8ba90 msgr2=0x562e0fcd19f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f8040d9a080 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x562e0fb8ba90 0x562e0fcd19f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f8040d9a080 1 -- 172.21.15.6:0/873982365 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8024066c10 msgr2=0x7f80240690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f8040d9a080 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8024066c10 0x7f80240690c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8028023000 tx=0x7f8028008000 comp rx=0 tx=0).stop 2023-11-28T13:02:57.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f8040d9a080 1 -- 172.21.15.6:0/873982365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba45b0 msgr2=0x562e0fc1c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f8040d9a080 1 --2- 172.21.15.6:0/873982365 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562e0fba45b0 0x562e0fc1c100 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f802c00af30 tx=0x7f802c008650 comp rx=0 tx=0).stop 2023-11-28T13:02:57.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f803769e700 1 -- 172.21.15.6:0/873982365 reap_dead start 2023-11-28T13:02:57.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f8040d9a080 1 -- 172.21.15.6:0/873982365 shutdown_connections 2023-11-28T13:02:57.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f8040d9a080 1 -- 172.21.15.6:0/873982365 >> 172.21.15.6:0/873982365 conn(0x562e0fba77e0 msgr2=0x562e0fc35e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:57.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f8040d9a080 1 -- 172.21.15.6:0/873982365 shutdown_connections 2023-11-28T13:02:57.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.154+0000 7f8040d9a080 1 -- 172.21.15.6:0/873982365 wait complete. 2023-11-28T13:02:57.168 INFO:teuthology.orchestra.run.smithi006.stdout:Yalphaparent_a 2023-11-28T13:02:57.169 DEBUG:teuthology.orchestra.run.smithi006:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T13:02:57.225 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:02:57.225 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511627777, 2023-11-28T13:02:57.225 INFO:teuthology.orchestra.run.smithi006.stdout: "ancestors": [ 2023-11-28T13:02:57.226 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:57.226 INFO:teuthology.orchestra.run.smithi006.stdout: "dirino": 1099511627776, 2023-11-28T13:02:57.226 INFO:teuthology.orchestra.run.smithi006.stdout: "dname": "alpha", 2023-11-28T13:02:57.226 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 4 2023-11-28T13:02:57.226 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:02:57.226 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:02:57.227 INFO:teuthology.orchestra.run.smithi006.stdout: "dirino": 1, 2023-11-28T13:02:57.227 INFO:teuthology.orchestra.run.smithi006.stdout: "dname": "parent_a", 2023-11-28T13:02:57.227 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 7 2023-11-28T13:02:57.227 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:02:57.227 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-28T13:02:57.227 INFO:teuthology.orchestra.run.smithi006.stdout: "pool": 9, 2023-11-28T13:02:57.228 INFO:teuthology.orchestra.run.smithi006.stdout: "old_pools": [] 2023-11-28T13:02:57.228 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:02:57.228 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:57.229 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:02:57.229 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:02:57.310 DEBUG:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:02:57.358 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:02:57.358 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:02:57.359 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:57.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:02:57.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.539+0000 7f0347034700 1 Processor -- start 2023-11-28T13:02:57.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.539+0000 7f0347034700 1 -- start start 2023-11-28T13:02:57.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.540+0000 7f0347034700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f034015c990 0x7f03400cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.540+0000 7f0347034700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f03400cc310 0x7f03400cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.540+0000 7f0347034700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03400ce340 con 0x7f0340161090 2023-11-28T13:02:57.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.540+0000 7f0347034700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03400cebe0 con 0x7f03400cc310 2023-11-28T13:02:57.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.540+0000 7f0347034700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03400cccf0 con 0x7f034015c990 2023-11-28T13:02:57.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.540+0000 7f03455d1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f03400cc310 0x7f03400cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.540+0000 7f0344dd0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f034015c990 0x7f03400cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.540+0000 7f033ffff700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f0340161090 0x7f03400ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:39186/0 (socket says 172.21.15.6:39186) 2023-11-28T13:02:57.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.540+0000 7f033ffff700 1 -- 172.21.15.6:0/3985921446 learned_addr learned my addr 172.21.15.6:0/3985921446 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:57.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.540+0000 7f03455d1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f03400cc310 0x7f03400cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:37326/0 (socket says 172.21.15.6:37326) 2023-11-28T13:02:57.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f03455d1700 1 -- 172.21.15.6:0/3985921446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f034015c990 msgr2=0x7f03400cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:57.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f03455d1700 1 --2- 172.21.15.6:0/3985921446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f034015c990 0x7f03400cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f03455d1700 1 -- 172.21.15.6:0/3985921446 >> v1:172.21.15.61:6789/0 conn(0x7f0340161090 legacy=0x7f03400ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f03455d1700 1 -- 172.21.15.6:0/3985921446 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03400cb3f0 con 0x7f03400cc310 2023-11-28T13:02:57.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f033f7fe700 1 -- 172.21.15.6:0/3985921446 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844268898 0 0) 0x7f03400ce340 con 0x7f0340161090 2023-11-28T13:02:57.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f0344dd0700 1 --2- 172.21.15.6:0/3985921446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f034015c990 0x7f03400cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:02:57.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f03455d1700 1 --2- 172.21.15.6:0/3985921446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03400cc310 0x7f03400cc7b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0338009d90 tx=0x7f033800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f256d718fd2fc2f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f033f7fe700 1 -- 172.21.15.6:0/3985921446 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033800e070 con 0x7f03400cc310 2023-11-28T13:02:57.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f033f7fe700 1 -- 172.21.15.6:0/3985921446 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033800bb20 con 0x7f03400cc310 2023-11-28T13:02:57.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f033f7fe700 1 -- 172.21.15.6:0/3985921446 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338012430 con 0x7f03400cc310 2023-11-28T13:02:57.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f0347034700 1 -- 172.21.15.6:0/3985921446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03400cc310 msgr2=0x7f03400cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f0347034700 1 --2- 172.21.15.6:0/3985921446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03400cc310 0x7f03400cc7b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0338009d90 tx=0x7f033800b330 comp rx=0 tx=0).stop 2023-11-28T13:02:57.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f0347034700 1 -- 172.21.15.6:0/3985921446 shutdown_connections 2023-11-28T13:02:57.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f0347034700 1 --2- 172.21.15.6:0/3985921446 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f034015c990 0x7f03400cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f0347034700 1 --2- 172.21.15.6:0/3985921446 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03400cc310 0x7f03400cc7b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.541+0000 7f0347034700 1 -- 172.21.15.6:0/3985921446 >> 172.21.15.6:0/3985921446 conn(0x7f03400b1650 msgr2=0x7f03400b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:57.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.542+0000 7f0347034700 1 -- 172.21.15.6:0/3985921446 shutdown_connections 2023-11-28T13:02:57.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.542+0000 7f0347034700 1 -- 172.21.15.6:0/3985921446 wait complete. 2023-11-28T13:02:57.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.543+0000 7f0347034700 1 Processor -- start 2023-11-28T13:02:57.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.543+0000 7f0347034700 1 -- start start 2023-11-28T13:02:57.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f0347034700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03400cc310 0x7f0340150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f0347034700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f034015c990 0x7f03400b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f0347034700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0340161090 0x7f03400acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f0347034700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03400b2a20 con 0x7f034015c990 2023-11-28T13:02:57.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f0347034700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03400ae020 con 0x7f03400cc310 2023-11-28T13:02:57.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f0347034700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03400ae300 con 0x7f0340161090 2023-11-28T13:02:57.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f034015c990 0x7f03400b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f034015c990 0x7f03400b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33612/0 (socket says 172.21.15.6:33612) 2023-11-28T13:02:57.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033ffff700 1 -- 172.21.15.6:0/294772614 learned_addr learned my addr 172.21.15.6:0/294772614 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:57.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f0344dd0700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03400cc310 0x7f0340150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f03455d1700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0340161090 0x7f03400acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033ffff700 1 -- 172.21.15.6:0/294772614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0340161090 msgr2=0x7f03400acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033ffff700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0340161090 0x7f03400acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033ffff700 1 -- 172.21.15.6:0/294772614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03400cc310 msgr2=0x7f0340150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033ffff700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03400cc310 0x7f0340150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033ffff700 1 -- 172.21.15.6:0/294772614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03340094a0 con 0x7f034015c990 2023-11-28T13:02:57.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033ffff700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f034015c990 0x7f03400b52d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f032c00c540 tx=0x7f032c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033d7fa700 1 -- 172.21.15.6:0/294772614 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c004c80 con 0x7f034015c990 2023-11-28T13:02:57.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f033d7fa700 1 -- 172.21.15.6:0/294772614 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c004de0 con 0x7f034015c990 2023-11-28T13:02:57.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.544+0000 7f0347034700 1 -- 172.21.15.6:0/294772614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0338009a70 con 0x7f034015c990 2023-11-28T13:02:57.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.545+0000 7f033d7fa700 1 -- 172.21.15.6:0/294772614 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c007310 con 0x7f034015c990 2023-11-28T13:02:57.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.545+0000 7f0347034700 1 -- 172.21.15.6:0/294772614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03400ce870 con 0x7f034015c990 2023-11-28T13:02:57.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.546+0000 7f0347034700 1 -- 172.21.15.6:0/294772614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03400ce590 con 0x7f034015c990 2023-11-28T13:02:57.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.546+0000 7f033d7fa700 1 -- 172.21.15.6:0/294772614 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f032c01a020 con 0x7f034015c990 2023-11-28T13:02:57.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.546+0000 7f033d7fa700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03240682f0 0x7f032406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.546+0000 7f033d7fa700 1 -- 172.21.15.6:0/294772614 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f032c085980 con 0x7f034015c990 2023-11-28T13:02:57.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.546+0000 7f0344dd0700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03240682f0 0x7f032406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.547+0000 7f0344dd0700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03240682f0 0x7f032406a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0334009cd0 tx=0x7f0334009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.551+0000 7f033d7fa700 1 -- 172.21.15.6:0/294772614 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f032c0520a0 con 0x7f034015c990 2023-11-28T13:02:57.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.749+0000 7f0347034700 1 -- 172.21.15.6:0/294772614 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03400cb080 con 0x7f034015c990 2023-11-28T13:02:57.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.750+0000 7f033d7fa700 1 -- 172.21.15.6:0/294772614 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18026 (secure 0 0 0) 0x7f032c0556c0 con 0x7f034015c990 2023-11-28T13:02:57.751 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:02:57.751 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":54,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:32.564104+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:29.490442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:30.534532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:02:57.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 -- 172.21.15.6:0/294772614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03240682f0 msgr2=0x7f032406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03240682f0 0x7f032406a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0334009cd0 tx=0x7f0334009210 comp rx=0 tx=0).stop 2023-11-28T13:02:57.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 -- 172.21.15.6:0/294772614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f034015c990 msgr2=0x7f03400b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f034015c990 0x7f03400b52d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f032c00c540 tx=0x7f032c00c850 comp rx=0 tx=0).stop 2023-11-28T13:02:57.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 -- 172.21.15.6:0/294772614 shutdown_connections 2023-11-28T13:02:57.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f03240682f0 0x7f032406a7a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0340161090 0x7f03400acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f03400cc310 0x7f0340150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 --2- 172.21.15.6:0/294772614 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f034015c990 0x7f03400b52d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 -- 172.21.15.6:0/294772614 >> 172.21.15.6:0/294772614 conn(0x7f03400b1650 msgr2=0x7f03400b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:57.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 -- 172.21.15.6:0/294772614 shutdown_connections 2023-11-28T13:02:57.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.752+0000 7f0347034700 1 -- 172.21.15.6:0/294772614 wait complete. 2023-11-28T13:02:57.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:02:29.490442+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:02:30.534532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:02:57.766 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:02:57.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.892+0000 7f0446021700 1 Processor -- start 2023-11-28T13:02:57.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.892+0000 7f0446021700 1 -- start start 2023-11-28T13:02:57.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f0446021700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440165670 0x7f0440167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f0446021700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04400ce550 con 0x7f04400b1630 2023-11-28T13:02:57.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f0446021700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04400ce830 con 0x7f04400cf5f0 2023-11-28T13:02:57.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f0446021700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04400ceb10 con 0x7f0440165670 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043f7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f04400b1630 0x7f04400b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49170/0 (socket says 172.21.15.6:49170) 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043f7fe700 1 -- 172.21.15.6:0/1364452916 learned_addr learned my addr 172.21.15.6:0/1364452916 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043ffff700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440165670 0x7f0440167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001353078 0 0) 0x7f04400ce550 con 0x7f04400b1630 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f042c0035b0 con 0x7f04400b1630 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172729923 0 0) 0x7f04400ce830 con 0x7f04400cf5f0 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04400ce550 con 0x7f04400cf5f0 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3416756184 0 0) 0x7f042c0035b0 con 0x7f04400b1630 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440165670 msgr2=0x7f0440167a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440165670 0x7f0440167a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 >> v1:172.21.15.61:6789/0 conn(0x7f04400cf5f0 legacy=0x7f04400caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04400cedf0 con 0x7f04400b1630 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f0428001670 con 0x7f04400b1630 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.893+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 >> v1:172.21.15.6:6789/0 conn(0x7f04400b1630 legacy=0x7f04400b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043e7fc700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f042c003bc0 0x7f042c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043e7fc700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440165670 0x7f042c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043e7fc700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f042c004d90 0x7f042c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04400ceb10 con 0x7f042c004d90 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c00bac0 con 0x7f042c003bc0 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c00bc50 con 0x7f0440165670 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043ffff700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f042c004d90 0x7f042c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043f7fe700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f042c003bc0 0x7f042c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043effd700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440165670 0x7f042c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043ffff700 1 -- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440165670 msgr2=0x7f042c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043ffff700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440165670 0x7f042c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043ffff700 1 -- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f042c003bc0 msgr2=0x7f042c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043ffff700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f042c003bc0 0x7f042c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043ffff700 1 -- 172.21.15.6:0/1364452916 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c0035b0 con 0x7f042c004d90 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043f7fe700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f042c003bc0 0x7f042c003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043effd700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440165670 0x7f042c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043ffff700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f042c004d90 0x7f042c005200 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f04400af5c0 tx=0x7f042c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed248bc44023fba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043800ebd0 con 0x7f042c004d90 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043800f040 con 0x7f042c004d90 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.894+0000 7f043e7fc700 1 -- 172.21.15.6:0/1364452916 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438014480 con 0x7f042c004d90 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.895+0000 7f0446021700 1 -- 172.21.15.6:0/1364452916 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f042c004d90 msgr2=0x7f042c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.895+0000 7f0446021700 1 --2- 172.21.15.6:0/1364452916 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f042c004d90 0x7f042c005200 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f04400af5c0 tx=0x7f042c005880 comp rx=0 tx=0).stop 2023-11-28T13:02:57.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.895+0000 7f043ffff700 1 -- 172.21.15.6:0/1364452916 reap_dead start 2023-11-28T13:02:57.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.895+0000 7f0446021700 1 -- 172.21.15.6:0/1364452916 shutdown_connections 2023-11-28T13:02:57.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.895+0000 7f0446021700 1 -- 172.21.15.6:0/1364452916 >> 172.21.15.6:0/1364452916 conn(0x7f04400cc630 msgr2=0x7f04400cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:57.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.895+0000 7f0446021700 1 -- 172.21.15.6:0/1364452916 shutdown_connections 2023-11-28T13:02:57.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.895+0000 7f0446021700 1 -- 172.21.15.6:0/1364452916 wait complete. 2023-11-28T13:02:57.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.896+0000 7f0446021700 1 Processor -- start 2023-11-28T13:02:57.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.896+0000 7f0446021700 1 -- start start 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f0446021700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f044014efb0 0x7f04400b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f0446021700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440153cc0 0x7f04401548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f0446021700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04400b4380 0x7f0440154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f0446021700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04400b32e0 con 0x7f04400b4380 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f0446021700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04400b35c0 con 0x7f044014efb0 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f0446021700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04400b38a0 con 0x7f0440153cc0 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04400b4380 0x7f0440154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043ffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04400b4380 0x7f0440154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33634/0 (socket says 172.21.15.6:33634) 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043ffff700 1 -- 172.21.15.6:0/1413214726 learned_addr learned my addr 172.21.15.6:0/1413214726 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043f7fe700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f044014efb0 0x7f04400b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043effd700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440153cc0 0x7f04401548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043ffff700 1 -- 172.21.15.6:0/1413214726 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440153cc0 msgr2=0x7f04401548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043ffff700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440153cc0 0x7f04401548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043ffff700 1 -- 172.21.15.6:0/1413214726 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f044014efb0 msgr2=0x7f04400b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043ffff700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f044014efb0 0x7f04400b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043ffff700 1 -- 172.21.15.6:0/1413214726 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04400cb5b0 con 0x7f04400b4380 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043effd700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440153cc0 0x7f04401548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.897+0000 7f043ffff700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04400b4380 0x7f0440154e00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0438016040 tx=0x7f0438009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.898+0000 7f0437fff700 1 -- 172.21.15.6:0/1413214726 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438007000 con 0x7f04400b4380 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.898+0000 7f0437fff700 1 -- 172.21.15.6:0/1413214726 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0438003e50 con 0x7f04400b4380 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.898+0000 7f0437fff700 1 -- 172.21.15.6:0/1413214726 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438014aa0 con 0x7f04400b4380 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.898+0000 7f0446021700 1 -- 172.21.15.6:0/1413214726 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04400cf0d0 con 0x7f04400b4380 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.898+0000 7f0446021700 1 -- 172.21.15.6:0/1413214726 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04400ce140 con 0x7f04400b4380 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.899+0000 7f0446021700 1 -- 172.21.15.6:0/1413214726 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04400ce830 con 0x7f04400b4380 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.899+0000 7f0437fff700 1 -- 172.21.15.6:0/1413214726 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0438014c00 con 0x7f04400b4380 2023-11-28T13:02:57.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.899+0000 7f0437fff700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04200682f0 0x7f042006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:57.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.900+0000 7f0437fff700 1 -- 172.21.15.6:0/1413214726 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0438087520 con 0x7f04400b4380 2023-11-28T13:02:57.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.900+0000 7f043f7fe700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04200682f0 0x7f042006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:57.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.900+0000 7f043f7fe700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04200682f0 0x7f042006a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0428001eb0 tx=0x7f0428007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:57.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:57.904+0000 7f0437fff700 1 -- 172.21.15.6:0/1413214726 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0438053c40 con 0x7f04400b4380 2023-11-28T13:02:58.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.098+0000 7f0446021700 1 -- 172.21.15.6:0/1413214726 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f04400739f0 con 0x7f04400b4380 2023-11-28T13:02:58.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.098+0000 7f0437fff700 1 -- 172.21.15.6:0/1413214726 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0438057260 con 0x7f04400b4380 2023-11-28T13:02:58.099 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:02:58.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 -- 172.21.15.6:0/1413214726 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04200682f0 msgr2=0x7f042006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04200682f0 0x7f042006a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0428001eb0 tx=0x7f0428007040 comp rx=0 tx=0).stop 2023-11-28T13:02:58.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 -- 172.21.15.6:0/1413214726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04400b4380 msgr2=0x7f0440154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04400b4380 0x7f0440154e00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0438016040 tx=0x7f0438009750 comp rx=0 tx=0).stop 2023-11-28T13:02:58.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 -- 172.21.15.6:0/1413214726 shutdown_connections 2023-11-28T13:02:58.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04400b4380 0x7f0440154e00 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f044014efb0 0x7f04400b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f04200682f0 0x7f042006a7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 --2- 172.21.15.6:0/1413214726 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0440153cc0 0x7f04401548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 -- 172.21.15.6:0/1413214726 >> 172.21.15.6:0/1413214726 conn(0x7f04400cc630 msgr2=0x7f04400cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 -- 172.21.15.6:0/1413214726 shutdown_connections 2023-11-28T13:02:58.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.100+0000 7f0446021700 1 -- 172.21.15.6:0/1413214726 wait complete. 2023-11-28T13:02:58.113 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:02:58.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.239+0000 7f4436c77700 1 Processor -- start 2023-11-28T13:02:58.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.239+0000 7f4436c77700 1 -- start start 2023-11-28T13:02:58.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.240+0000 7f4436c77700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f443015fc00 0x7f443006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.240+0000 7f4436c77700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f44300b2d90 0x7f44300cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.240+0000 7f4436c77700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443006f940 con 0x7f443006f3e0 2023-11-28T13:02:58.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.240+0000 7f4436c77700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443006fc20 con 0x7f44300b2d90 2023-11-28T13:02:58.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.240+0000 7f4436c77700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44300cb3f0 con 0x7f443015fc00 2023-11-28T13:02:58.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.240+0000 7f4435214700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f44300b2d90 0x7f44300cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.240+0000 7f442ffff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f443006f3e0 0x7f44300b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49190/0 (socket says 172.21.15.6:49190) 2023-11-28T13:02:58.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.240+0000 7f442ffff700 1 -- 172.21.15.6:0/2816069261 learned_addr learned my addr 172.21.15.6:0/2816069261 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:58.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4434a13700 1 --2- 172.21.15.6:0/2816069261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f443015fc00 0x7f443006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f442f7fe700 1 -- 172.21.15.6:0/2816069261 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797280371 0 0) 0x7f443006f940 con 0x7f443006f3e0 2023-11-28T13:02:58.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f442f7fe700 1 -- 172.21.15.6:0/2816069261 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44200035b0 con 0x7f443006f3e0 2023-11-28T13:02:58.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4435214700 1 -- 172.21.15.6:0/2816069261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f443015fc00 msgr2=0x7f443006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:58.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4435214700 1 --2- 172.21.15.6:0/2816069261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f443015fc00 0x7f443006eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4435214700 1 -- 172.21.15.6:0/2816069261 >> v1:172.21.15.6:6789/0 conn(0x7f443006f3e0 legacy=0x7f44300b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4435214700 1 -- 172.21.15.6:0/2816069261 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300cb6d0 con 0x7f44300b2d90 2023-11-28T13:02:58.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4435214700 1 --2- 172.21.15.6:0/2816069261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44300b2d90 0x7f44300cbbf0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4428009d90 tx=0x7f442800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9b05260272a110 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f442f7fe700 1 -- 172.21.15.6:0/2816069261 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442800e070 con 0x7f44300b2d90 2023-11-28T13:02:58.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f442f7fe700 1 -- 172.21.15.6:0/2816069261 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442800bb20 con 0x7f44300b2d90 2023-11-28T13:02:58.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f442f7fe700 1 -- 172.21.15.6:0/2816069261 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428012430 con 0x7f44300b2d90 2023-11-28T13:02:58.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4436c77700 1 -- 172.21.15.6:0/2816069261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44300b2d90 msgr2=0x7f44300cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4436c77700 1 --2- 172.21.15.6:0/2816069261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44300b2d90 0x7f44300cbbf0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4428009d90 tx=0x7f442800b330 comp rx=0 tx=0).stop 2023-11-28T13:02:58.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4436c77700 1 -- 172.21.15.6:0/2816069261 shutdown_connections 2023-11-28T13:02:58.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4436c77700 1 --2- 172.21.15.6:0/2816069261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44300b2d90 0x7f44300cbbf0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.241+0000 7f4436c77700 1 --2- 172.21.15.6:0/2816069261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f443015fc00 0x7f443006eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.242+0000 7f4436c77700 1 -- 172.21.15.6:0/2816069261 >> 172.21.15.6:0/2816069261 conn(0x7f44300cd680 msgr2=0x7f44300cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.242+0000 7f4436c77700 1 -- 172.21.15.6:0/2816069261 shutdown_connections 2023-11-28T13:02:58.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.242+0000 7f4436c77700 1 -- 172.21.15.6:0/2816069261 wait complete. 2023-11-28T13:02:58.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.243+0000 7f4436c77700 1 Processor -- start 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.243+0000 7f4436c77700 1 -- start start 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.244+0000 7f4436c77700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f443006f3e0 0x7f4430146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.244+0000 7f4436c77700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44300b2d90 0x7f44300b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.244+0000 7f4436c77700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f443015fc00 0x7f44300ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.244+0000 7f4436c77700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44300b4a80 con 0x7f443015fc00 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.244+0000 7f4436c77700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4430146c50 con 0x7f44300b2d90 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.244+0000 7f4436c77700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44300b4cd0 con 0x7f443006f3e0 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.244+0000 7f4435214700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f443015fc00 0x7f44300ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.244+0000 7f4435214700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f443015fc00 0x7f44300ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33648/0 (socket says 172.21.15.6:33648) 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.244+0000 7f4435214700 1 -- 172.21.15.6:0/1605892965 learned_addr learned my addr 172.21.15.6:0/1605892965 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:58.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f442ffff700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44300b2d90 0x7f44300b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f4434a13700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f443006f3e0 0x7f4430146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f4435214700 1 -- 172.21.15.6:0/1605892965 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f443006f3e0 msgr2=0x7f4430146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f4435214700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f443006f3e0 0x7f4430146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f4435214700 1 -- 172.21.15.6:0/1605892965 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44300b2d90 msgr2=0x7f44300b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f4435214700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44300b2d90 0x7f44300b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f4435214700 1 -- 172.21.15.6:0/1605892965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4428009a70 con 0x7f443015fc00 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f442ffff700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44300b2d90 0x7f44300b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f4435214700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f443015fc00 0x7f44300ca7d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4428013000 tx=0x7f4428004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f442d7fa700 1 -- 172.21.15.6:0/1605892965 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428012430 con 0x7f443015fc00 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f442d7fa700 1 -- 172.21.15.6:0/1605892965 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4428020070 con 0x7f443015fc00 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f4436c77700 1 -- 172.21.15.6:0/1605892965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44300ce340 con 0x7f443015fc00 2023-11-28T13:02:58.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f442d7fa700 1 -- 172.21.15.6:0/1605892965 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428003ea0 con 0x7f443015fc00 2023-11-28T13:02:58.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.245+0000 7f4436c77700 1 -- 172.21.15.6:0/1605892965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443015ffd0 con 0x7f443015fc00 2023-11-28T13:02:58.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.246+0000 7f4436c77700 1 -- 172.21.15.6:0/1605892965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44300cc130 con 0x7f443015fc00 2023-11-28T13:02:58.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.247+0000 7f442d7fa700 1 -- 172.21.15.6:0/1605892965 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4428004280 con 0x7f443015fc00 2023-11-28T13:02:58.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.247+0000 7f442d7fa700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44140682f0 0x7f441406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.247+0000 7f442d7fa700 1 -- 172.21.15.6:0/1605892965 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4428022070 con 0x7f443015fc00 2023-11-28T13:02:58.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.247+0000 7f4434a13700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44140682f0 0x7f441406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.247+0000 7f4434a13700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44140682f0 0x7f441406a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f44300b67c0 tx=0x7f4424008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.252+0000 7f442d7fa700 1 -- 172.21.15.6:0/1605892965 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4428054990 con 0x7f443015fc00 2023-11-28T13:02:58.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.444+0000 7f4436c77700 1 -- 172.21.15.6:0/1605892965 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f44300739f0 con 0x7f443015fc00 2023-11-28T13:02:58.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.445+0000 7f442d7fa700 1 -- 172.21.15.6:0/1605892965 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4428057fb0 con 0x7f443015fc00 2023-11-28T13:02:58.446 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:02:58.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 -- 172.21.15.6:0/1605892965 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44140682f0 msgr2=0x7f441406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44140682f0 0x7f441406a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f44300b67c0 tx=0x7f4424008040 comp rx=0 tx=0).stop 2023-11-28T13:02:58.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 -- 172.21.15.6:0/1605892965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f443015fc00 msgr2=0x7f44300ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f443015fc00 0x7f44300ca7d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4428013000 tx=0x7f4428004ad0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 -- 172.21.15.6:0/1605892965 shutdown_connections 2023-11-28T13:02:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f443015fc00 0x7f44300ca7d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44140682f0 0x7f441406a7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f443006f3e0 0x7f4430146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 --2- 172.21.15.6:0/1605892965 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44300b2d90 0x7f44300b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 -- 172.21.15.6:0/1605892965 >> 172.21.15.6:0/1605892965 conn(0x7f44300cd680 msgr2=0x7f44300b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 -- 172.21.15.6:0/1605892965 shutdown_connections 2023-11-28T13:02:58.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.447+0000 7f4436c77700 1 -- 172.21.15.6:0/1605892965 wait complete. 2023-11-28T13:02:58.460 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:02:58.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.586+0000 7fb2c413e700 1 Processor -- start 2023-11-28T13:02:58.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.586+0000 7fb2c413e700 1 -- start start 2023-11-28T13:02:58.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.587+0000 7fb2c413e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb2bc15a640 0x7fb2bc0c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.587+0000 7fb2c413e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc06b900 0x7fb2bc0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.587+0000 7fb2c413e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0ae730 con 0x7fb2bc0cd140 2023-11-28T13:02:58.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.587+0000 7fb2c413e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc0cb210 con 0x7fb2bc15a640 2023-11-28T13:02:58.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.587+0000 7fb2c413e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc06e9e0 con 0x7fb2bc06b900 2023-11-28T13:02:58.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.587+0000 7fb2c1eda700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb2bc15a640 0x7fb2bc0c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.587+0000 7fb2c1eda700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb2bc15a640 0x7fb2bc0c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33652/0 (socket says 172.21.15.6:33652) 2023-11-28T13:02:58.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.587+0000 7fb2c1eda700 1 -- 172.21.15.6:0/1322906058 learned_addr learned my addr 172.21.15.6:0/1322906058 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:58.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.588+0000 7fb2c26db700 1 --2- 172.21.15.6:0/1322906058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc06b900 0x7fb2bc0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.588+0000 7fb2c1eda700 1 -- 172.21.15.6:0/1322906058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc06b900 msgr2=0x7fb2bc0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:58.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.588+0000 7fb2c1eda700 1 --2- 172.21.15.6:0/1322906058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc06b900 0x7fb2bc0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.588+0000 7fb2c1eda700 1 -- 172.21.15.6:0/1322906058 >> v1:172.21.15.61:6789/0 conn(0x7fb2bc0cd140 legacy=0x7fb2bc06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.588+0000 7fb2c1eda700 1 -- 172.21.15.6:0/1322906058 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0ae980 con 0x7fb2bc15a640 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.588+0000 7fb2c26db700 1 --2- 172.21.15.6:0/1322906058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc06b900 0x7fb2bc0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.588+0000 7fb2c1eda700 1 --2- 172.21.15.6:0/1322906058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc15a640 0x7fb2bc0c3700 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0098f0 tx=0x7fb2ac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a020237d3b75ea92 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.588+0000 7fb2c0ed8700 1 -- 172.21.15.6:0/1322906058 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac00b8a0 con 0x7fb2bc15a640 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.588+0000 7fb2c0ed8700 1 -- 172.21.15.6:0/1322906058 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac00ba00 con 0x7fb2bc15a640 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.588+0000 7fb2c0ed8700 1 -- 172.21.15.6:0/1322906058 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac00bd90 con 0x7fb2bc15a640 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.589+0000 7fb2c413e700 1 -- 172.21.15.6:0/1322906058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc15a640 msgr2=0x7fb2bc0c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.589+0000 7fb2c413e700 1 --2- 172.21.15.6:0/1322906058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc15a640 0x7fb2bc0c3700 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0098f0 tx=0x7fb2ac009c00 comp rx=0 tx=0).stop 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.589+0000 7fb2c413e700 1 -- 172.21.15.6:0/1322906058 shutdown_connections 2023-11-28T13:02:58.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.589+0000 7fb2c413e700 1 --2- 172.21.15.6:0/1322906058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc06b900 0x7fb2bc0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.589+0000 7fb2c413e700 1 --2- 172.21.15.6:0/1322906058 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc15a640 0x7fb2bc0c3700 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.589+0000 7fb2c413e700 1 -- 172.21.15.6:0/1322906058 >> 172.21.15.6:0/1322906058 conn(0x7fb2bc0cc630 msgr2=0x7fb2bc0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.589+0000 7fb2c413e700 1 -- 172.21.15.6:0/1322906058 shutdown_connections 2023-11-28T13:02:58.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.589+0000 7fb2c413e700 1 -- 172.21.15.6:0/1322906058 wait complete. 2023-11-28T13:02:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.590+0000 7fb2c413e700 1 Processor -- start 2023-11-28T13:02:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.590+0000 7fb2c413e700 1 -- start start 2023-11-28T13:02:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c413e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2bc06b900 0x7fb2bc152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c413e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc0cd140 0x7fb2bc153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c413e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc15a640 0x7fb2bc1f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c413e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc15d8a0 con 0x7fb2bc0cd140 2023-11-28T13:02:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c413e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc153a60 con 0x7fb2bc06b900 2023-11-28T13:02:58.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c413e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc153d40 con 0x7fb2bc15a640 2023-11-28T13:02:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c16d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc0cd140 0x7fb2bc153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c16d9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc0cd140 0x7fb2bc153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33660/0 (socket says 172.21.15.6:33660) 2023-11-28T13:02:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c16d9700 1 -- 172.21.15.6:0/1310774217 learned_addr learned my addr 172.21.15.6:0/1310774217 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c1eda700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2bc06b900 0x7fb2bc152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c26db700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc15a640 0x7fb2bc1f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c16d9700 1 -- 172.21.15.6:0/1310774217 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc15a640 msgr2=0x7fb2bc1f98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c16d9700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc15a640 0x7fb2bc1f98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c16d9700 1 -- 172.21.15.6:0/1310774217 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2bc06b900 msgr2=0x7fb2bc152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c16d9700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2bc06b900 0x7fb2bc152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c16d9700 1 -- 172.21.15.6:0/1310774217 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac009550 con 0x7fb2bc0cd140 2023-11-28T13:02:58.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c26db700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc15a640 0x7fb2bc1f98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:58.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.591+0000 7fb2c16d9700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc0cd140 0x7fb2bc153430 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800c4e0 tx=0x7fb2b800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.592+0000 7fb2b27fc700 1 -- 172.21.15.6:0/1310774217 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80052d0 con 0x7fb2bc0cd140 2023-11-28T13:02:58.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.592+0000 7fb2b27fc700 1 -- 172.21.15.6:0/1310774217 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b8005430 con 0x7fb2bc0cd140 2023-11-28T13:02:58.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.592+0000 7fb2b27fc700 1 -- 172.21.15.6:0/1310774217 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80038c0 con 0x7fb2bc0cd140 2023-11-28T13:02:58.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.592+0000 7fb2c413e700 1 -- 172.21.15.6:0/1310774217 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc0aef40 con 0x7fb2bc0cd140 2023-11-28T13:02:58.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.592+0000 7fb2c413e700 1 -- 172.21.15.6:0/1310774217 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc0aec60 con 0x7fb2bc0cd140 2023-11-28T13:02:58.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.593+0000 7fb2c413e700 1 -- 172.21.15.6:0/1310774217 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc0af220 con 0x7fb2bc0cd140 2023-11-28T13:02:58.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.593+0000 7fb2b27fc700 1 -- 172.21.15.6:0/1310774217 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb2b8019030 con 0x7fb2bc0cd140 2023-11-28T13:02:58.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.593+0000 7fb2b27fc700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb2a00682f0 0x7fb2a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.594+0000 7fb2b27fc700 1 -- 172.21.15.6:0/1310774217 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb2b8085100 con 0x7fb2bc0cd140 2023-11-28T13:02:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.594+0000 7fb2c1eda700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb2a00682f0 0x7fb2a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.594+0000 7fb2c1eda700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb2a00682f0 0x7fb2a006a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009c40 tx=0x7fb2ac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.598+0000 7fb2b27fc700 1 -- 172.21.15.6:0/1310774217 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb2b8051820 con 0x7fb2bc0cd140 2023-11-28T13:02:58.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.789+0000 7fb2c413e700 1 -- 172.21.15.6:0/1310774217 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2bc154200 con 0x7fb2bc0cd140 2023-11-28T13:02:58.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.789+0000 7fb2b27fc700 1 -- 172.21.15.6:0/1310774217 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2b8054e40 con 0x7fb2bc0cd140 2023-11-28T13:02:58.790 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 -- 172.21.15.6:0/1310774217 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb2a00682f0 msgr2=0x7fb2a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb2a00682f0 0x7fb2a006a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009c40 tx=0x7fb2ac008030 comp rx=0 tx=0).stop 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 -- 172.21.15.6:0/1310774217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc0cd140 msgr2=0x7fb2bc153430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc0cd140 0x7fb2bc153430 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800c4e0 tx=0x7fb2b800c7f0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 -- 172.21.15.6:0/1310774217 shutdown_connections 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb2a00682f0 0x7fb2a006a7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb2bc15a640 0x7fb2bc1f98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2bc06b900 0x7fb2bc152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 --2- 172.21.15.6:0/1310774217 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2bc0cd140 0x7fb2bc153430 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 -- 172.21.15.6:0/1310774217 >> 172.21.15.6:0/1310774217 conn(0x7fb2bc0cc630 msgr2=0x7fb2bc0b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 -- 172.21.15.6:0/1310774217 shutdown_connections 2023-11-28T13:02:58.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.791+0000 7fb2c413e700 1 -- 172.21.15.6:0/1310774217 wait complete. 2023-11-28T13:02:58.804 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:02:58.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.931+0000 7fac69fcb700 1 Processor -- start 2023-11-28T13:02:58.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.931+0000 7fac69fcb700 1 -- start start 2023-11-28T13:02:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac69fcb700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fac640b1630 0x7fac640cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac69fcb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6406afc0 0x7fac6406b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac69fcb700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac640ce550 con 0x7fac640b1a00 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac69fcb700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac640ce830 con 0x7fac640b1630 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac69fcb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac640ceb10 con 0x7fac6406afc0 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac637fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fac640b1630 0x7fac640cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac62ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fac640b1a00 0x7fac640caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49220/0 (socket says 172.21.15.6:49220) 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac62ffd700 1 -- 172.21.15.6:0/4182240811 learned_addr learned my addr 172.21.15.6:0/4182240811 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac637fe700 1 -- 172.21.15.6:0/4182240811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6406afc0 msgr2=0x7fac6406b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac637fe700 1 --2- 172.21.15.6:0/4182240811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6406afc0 0x7fac6406b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac637fe700 1 -- 172.21.15.6:0/4182240811 >> v1:172.21.15.6:6789/0 conn(0x7fac640b1a00 legacy=0x7fac640caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac637fe700 1 -- 172.21.15.6:0/4182240811 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac640cedf0 con 0x7fac640b1630 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac627fc700 1 -- 172.21.15.6:0/4182240811 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254513106 0 0) 0x7fac640ce550 con 0x7fac640b1a00 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac637fe700 1 --2- 172.21.15.6:0/4182240811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac640b1630 0x7fac640cf5f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fac54009870 tx=0x7fac54009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62c1fc73d0814d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac627fc700 1 -- 172.21.15.6:0/4182240811 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54011830 con 0x7fac640b1630 2023-11-28T13:02:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac627fc700 1 -- 172.21.15.6:0/4182240811 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac54011990 con 0x7fac640b1630 2023-11-28T13:02:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.932+0000 7fac627fc700 1 -- 172.21.15.6:0/4182240811 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54011d20 con 0x7fac640b1630 2023-11-28T13:02:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.933+0000 7fac69fcb700 1 -- 172.21.15.6:0/4182240811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac640b1630 msgr2=0x7fac640cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.933+0000 7fac69fcb700 1 --2- 172.21.15.6:0/4182240811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac640b1630 0x7fac640cf5f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fac54009870 tx=0x7fac54009b80 comp rx=0 tx=0).stop 2023-11-28T13:02:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.933+0000 7fac69fcb700 1 -- 172.21.15.6:0/4182240811 shutdown_connections 2023-11-28T13:02:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.933+0000 7fac69fcb700 1 --2- 172.21.15.6:0/4182240811 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6406afc0 0x7fac6406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.933+0000 7fac69fcb700 1 --2- 172.21.15.6:0/4182240811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac640b1630 0x7fac640cf5f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.933+0000 7fac69fcb700 1 -- 172.21.15.6:0/4182240811 >> 172.21.15.6:0/4182240811 conn(0x7fac640cc630 msgr2=0x7fac640cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.933+0000 7fac69fcb700 1 -- 172.21.15.6:0/4182240811 shutdown_connections 2023-11-28T13:02:58.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.933+0000 7fac69fcb700 1 -- 172.21.15.6:0/4182240811 wait complete. 2023-11-28T13:02:58.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.934+0000 7fac69fcb700 1 Processor -- start 2023-11-28T13:02:58.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.934+0000 7fac69fcb700 1 -- start start 2023-11-28T13:02:58.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.935+0000 7fac69fcb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac6406afc0 0x7fac6415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.935+0000 7fac69fcb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac640b1630 0x7fac640b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.935+0000 7fac69fcb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac640b1a00 0x7fac6415e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.935+0000 7fac69fcb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640b5af0 con 0x7fac6406afc0 2023-11-28T13:02:58.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.935+0000 7fac69fcb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640b5550 con 0x7fac640b1a00 2023-11-28T13:02:58.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.935+0000 7fac69fcb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac640b5810 con 0x7fac640b1630 2023-11-28T13:02:58.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.935+0000 7fac637fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac6406afc0 0x7fac6415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.935+0000 7fac637fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac6406afc0 0x7fac6415b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33672/0 (socket says 172.21.15.6:33672) 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.935+0000 7fac637fe700 1 -- 172.21.15.6:0/389153077 learned_addr learned my addr 172.21.15.6:0/389153077 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac62ffd700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac640b1630 0x7fac640b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac63fff700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac640b1a00 0x7fac6415e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac637fe700 1 -- 172.21.15.6:0/389153077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac640b1630 msgr2=0x7fac640b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac637fe700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac640b1630 0x7fac640b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac637fe700 1 -- 172.21.15.6:0/389153077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac640b1a00 msgr2=0x7fac6415e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac637fe700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac640b1a00 0x7fac6415e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac637fe700 1 -- 172.21.15.6:0/389153077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac540094a0 con 0x7fac6406afc0 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac62ffd700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac640b1630 0x7fac640b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac637fe700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac6406afc0 0x7fac6415b3b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fac54009bc0 tx=0x7fac54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac4bfff700 1 -- 172.21.15.6:0/389153077 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540036a0 con 0x7fac6406afc0 2023-11-28T13:02:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac4bfff700 1 -- 172.21.15.6:0/389153077 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5401f070 con 0x7fac6406afc0 2023-11-28T13:02:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac69fcb700 1 -- 172.21.15.6:0/389153077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac640b1ea0 con 0x7fac6406afc0 2023-11-28T13:02:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac4bfff700 1 -- 172.21.15.6:0/389153077 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54003a20 con 0x7fac6406afc0 2023-11-28T13:02:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.936+0000 7fac69fcb700 1 -- 172.21.15.6:0/389153077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac640cf0d0 con 0x7fac6406afc0 2023-11-28T13:02:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.937+0000 7fac69fcb700 1 -- 172.21.15.6:0/389153077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac640cb5b0 con 0x7fac6406afc0 2023-11-28T13:02:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.938+0000 7fac4bfff700 1 -- 172.21.15.6:0/389153077 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fac5400f050 con 0x7fac6406afc0 2023-11-28T13:02:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.938+0000 7fac4bfff700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac500682f0 0x7fac5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.938+0000 7fac4bfff700 1 -- 172.21.15.6:0/389153077 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fac54021070 con 0x7fac6406afc0 2023-11-28T13:02:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.938+0000 7fac62ffd700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac500682f0 0x7fac5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:58.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.938+0000 7fac62ffd700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac500682f0 0x7fac5006a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac640ad8c0 tx=0x7fac58007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:58.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:58.942+0000 7fac4bfff700 1 -- 172.21.15.6:0/389153077 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fac54053e10 con 0x7fac6406afc0 2023-11-28T13:02:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.138+0000 7fac69fcb700 1 -- 172.21.15.6:0/389153077 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac640739f0 con 0x7fac6406afc0 2023-11-28T13:02:59.140 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.139+0000 7fac4bfff700 1 -- 172.21.15.6:0/389153077 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac54015020 con 0x7fac6406afc0 2023-11-28T13:02:59.140 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:02:59.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 -- 172.21.15.6:0/389153077 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac500682f0 msgr2=0x7fac5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac500682f0 0x7fac5006a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac640ad8c0 tx=0x7fac58007d30 comp rx=0 tx=0).stop 2023-11-28T13:02:59.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 -- 172.21.15.6:0/389153077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac6406afc0 msgr2=0x7fac6415b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac6406afc0 0x7fac6415b3b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fac54009bc0 tx=0x7fac54004750 comp rx=0 tx=0).stop 2023-11-28T13:02:59.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 -- 172.21.15.6:0/389153077 shutdown_connections 2023-11-28T13:02:59.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac640b1a00 0x7fac6415e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac6406afc0 0x7fac6415b3b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac500682f0 0x7fac5006a7a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 --2- 172.21.15.6:0/389153077 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac640b1630 0x7fac640b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 -- 172.21.15.6:0/389153077 >> 172.21.15.6:0/389153077 conn(0x7fac640cc630 msgr2=0x7fac640ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:59.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 -- 172.21.15.6:0/389153077 shutdown_connections 2023-11-28T13:02:59.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.141+0000 7fac69fcb700 1 -- 172.21.15.6:0/389153077 wait complete. 2023-11-28T13:02:59.154 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:02:59.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.278+0000 7fd88be8d700 1 Processor -- start 2023-11-28T13:02:59.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.278+0000 7fd88be8d700 1 -- start start 2023-11-28T13:02:59.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88be8d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd8840cb230 0x7fd8840ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88be8d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840b2660 0x7fd8840b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88be8d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8840cfb90 con 0x7fd88415a6c0 2023-11-28T13:02:59.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88be8d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8840b3020 con 0x7fd8840cb230 2023-11-28T13:02:59.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88be8d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840cb680 con 0x7fd8840b2660 2023-11-28T13:02:59.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88a42a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840b2660 0x7fd8840b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88a42a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840b2660 0x7fd8840b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:51620/0 (socket says 172.21.15.6:51620) 2023-11-28T13:02:59.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd889428700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd8840cb230 0x7fd8840ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88a42a700 1 -- 172.21.15.6:0/517671459 learned_addr learned my addr 172.21.15.6:0/517671459 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:59.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88a42a700 1 -- 172.21.15.6:0/517671459 >> v2:172.21.15.61:3300/0 conn(0x7fd8840cb230 msgr2=0x7fd8840ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:02:59.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88a42a700 1 --2- 172.21.15.6:0/517671459 >> v2:172.21.15.61:3300/0 conn(0x7fd8840cb230 0x7fd8840ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88a42a700 1 -- 172.21.15.6:0/517671459 >> v1:172.21.15.61:6789/0 conn(0x7fd88415a6c0 legacy=0x7fd8840d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.279+0000 7fd88a42a700 1 -- 172.21.15.6:0/517671459 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8840cb930 con 0x7fd8840b2660 2023-11-28T13:02:59.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd88a42a700 1 --2- 172.21.15.6:0/517671459 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840b2660 0x7fd8840b2ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009e50 tx=0x7fd87c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281d9f59411b472c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd888c27700 1 -- 172.21.15.6:0/517671459 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00e070 con 0x7fd8840b2660 2023-11-28T13:02:59.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd888c27700 1 -- 172.21.15.6:0/517671459 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c00b910 con 0x7fd8840b2660 2023-11-28T13:02:59.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd888c27700 1 -- 172.21.15.6:0/517671459 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00bcc0 con 0x7fd8840b2660 2023-11-28T13:02:59.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd88be8d700 1 -- 172.21.15.6:0/517671459 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840b2660 msgr2=0x7fd8840b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd88be8d700 1 --2- 172.21.15.6:0/517671459 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840b2660 0x7fd8840b2ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009e50 tx=0x7fd87c00b330 comp rx=0 tx=0).stop 2023-11-28T13:02:59.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd88be8d700 1 -- 172.21.15.6:0/517671459 shutdown_connections 2023-11-28T13:02:59.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd88be8d700 1 --2- 172.21.15.6:0/517671459 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840b2660 0x7fd8840b2ae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd88be8d700 1 --2- 172.21.15.6:0/517671459 >> v2:172.21.15.61:3300/0 conn(0x7fd8840cb230 0x7fd8840ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd88be8d700 1 -- 172.21.15.6:0/517671459 >> 172.21.15.6:0/517671459 conn(0x7fd8840cd660 msgr2=0x7fd8840cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:59.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.280+0000 7fd88be8d700 1 -- 172.21.15.6:0/517671459 shutdown_connections 2023-11-28T13:02:59.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.281+0000 7fd88be8d700 1 -- 172.21.15.6:0/517671459 wait complete. 2023-11-28T13:02:59.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.282+0000 7fd88be8d700 1 Processor -- start 2023-11-28T13:02:59.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.282+0000 7fd88be8d700 1 -- start start 2023-11-28T13:02:59.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.282+0000 7fd88be8d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8840b2660 0x7fd884153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.282+0000 7fd88be8d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840cb230 0x7fd8841584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.282+0000 7fd88be8d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd88415a6c0 0x7fd884158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.282+0000 7fd88be8d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840c5340 con 0x7fd8840b2660 2023-11-28T13:02:59.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.282+0000 7fd88be8d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840c2e80 con 0x7fd88415a6c0 2023-11-28T13:02:59.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.282+0000 7fd88be8d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8840c7550 con 0x7fd8840cb230 2023-11-28T13:02:59.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd889c29700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8840b2660 0x7fd884153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd889c29700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8840b2660 0x7fd884153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33686/0 (socket says 172.21.15.6:33686) 2023-11-28T13:02:59.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd889c29700 1 -- 172.21.15.6:0/1746468169 learned_addr learned my addr 172.21.15.6:0/1746468169 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:02:59.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd889428700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840cb230 0x7fd8841584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd88a42a700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd88415a6c0 0x7fd884158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd889c29700 1 -- 172.21.15.6:0/1746468169 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840cb230 msgr2=0x7fd8841584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd889c29700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840cb230 0x7fd8841584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd889c29700 1 -- 172.21.15.6:0/1746468169 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd88415a6c0 msgr2=0x7fd884158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd889c29700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd88415a6c0 0x7fd884158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd889c29700 1 -- 172.21.15.6:0/1746468169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87c009b30 con 0x7fd8840b2660 2023-11-28T13:02:59.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd889c29700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8840b2660 0x7fd884153e70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c4e0 tx=0x7fd87400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd87a7fc700 1 -- 172.21.15.6:0/1746468169 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8740052d0 con 0x7fd8840b2660 2023-11-28T13:02:59.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd87a7fc700 1 -- 172.21.15.6:0/1746468169 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd874005430 con 0x7fd8840b2660 2023-11-28T13:02:59.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd88be8d700 1 -- 172.21.15.6:0/1746468169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8840cf540 con 0x7fd8840b2660 2023-11-28T13:02:59.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd87a7fc700 1 -- 172.21.15.6:0/1746468169 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8740038c0 con 0x7fd8840b2660 2023-11-28T13:02:59.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.283+0000 7fd88be8d700 1 -- 172.21.15.6:0/1746468169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8840cf260 con 0x7fd8840b2660 2023-11-28T13:02:59.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.284+0000 7fd88be8d700 1 -- 172.21.15.6:0/1746468169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8840cf820 con 0x7fd8840b2660 2023-11-28T13:02:59.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.285+0000 7fd87a7fc700 1 -- 172.21.15.6:0/1746468169 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd874019030 con 0x7fd8840b2660 2023-11-28T13:02:59.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.285+0000 7fd87a7fc700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd8680682f0 0x7fd86806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:02:59.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.285+0000 7fd87a7fc700 1 -- 172.21.15.6:0/1746468169 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd874085030 con 0x7fd8840b2660 2023-11-28T13:02:59.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.285+0000 7fd889428700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd8680682f0 0x7fd86806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:02:59.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.286+0000 7fd889428700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd8680682f0 0x7fd86806a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd8841e6cf0 tx=0x7fd880009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:02:59.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.289+0000 7fd87a7fc700 1 -- 172.21.15.6:0/1746468169 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd874051750 con 0x7fd8840b2660 2023-11-28T13:02:59.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.499+0000 7fd88be8d700 1 -- 172.21.15.6:0/1746468169 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd884061e80 con 0x7fd8840b2660 2023-11-28T13:02:59.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.499+0000 7fd87a7fc700 1 -- 172.21.15.6:0/1746468169 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd874017080 con 0x7fd8840b2660 2023-11-28T13:02:59.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 -- 172.21.15.6:0/1746468169 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd8680682f0 msgr2=0x7fd86806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd8680682f0 0x7fd86806a7a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd8841e6cf0 tx=0x7fd880009210 comp rx=0 tx=0).stop 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 -- 172.21.15.6:0/1746468169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8840b2660 msgr2=0x7fd884153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8840b2660 0x7fd884153e70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c4e0 tx=0x7fd87400c7f0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 -- 172.21.15.6:0/1746468169 shutdown_connections 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd88415a6c0 0x7fd884158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd8840b2660 0x7fd884153e70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd8680682f0 0x7fd86806a7a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 --2- 172.21.15.6:0/1746468169 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd8840cb230 0x7fd8841584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 -- 172.21.15.6:0/1746468169 >> 172.21.15.6:0/1746468169 conn(0x7fd8840cd660 msgr2=0x7fd8840b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 -- 172.21.15.6:0/1746468169 shutdown_connections 2023-11-28T13:02:59.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:02:59.501+0000 7fd88be8d700 1 -- 172.21.15.6:0/1746468169 wait complete. 2023-11-28T13:02:59.504 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:02:59.514 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2023-11-28T13:02:59.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:59.515 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T13:02:59.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[126073]: fuse finished with error 0 and tester_r 0 2023-11-28T13:02:59.555 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:02:59.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:59.555 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:02:59.692 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:02:59.693 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:59.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:02:59.715 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:02:59.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:59.715 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T13:02:59.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:02:59.775 INFO:teuthology.orchestra.run.smithi061.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T13:02:59.775 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:02:59.775 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:59.793 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3446 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:02:59.793 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:02:59.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:59.794 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T13:02:59.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:02:59.851 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T13:02:59.851 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:02:59.851 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:02:59.873 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:02:59.873 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:02:59.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:02:59.873 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T13:02:59.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:02:59.935 INFO:teuthology.orchestra.run.smithi061.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T13:02:59.936 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:00.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.061+0000 7f3a82f39700 1 Processor -- start 2023-11-28T13:03:00.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.061+0000 7f3a82f39700 1 -- start start 2023-11-28T13:03:00.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.062+0000 7f3a82f39700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3a7c15c990 0x7f3a7c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.062+0000 7f3a82f39700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 0x7f3a7c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.062+0000 7f3a82f39700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c0ce340 con 0x7f3a7c0caeb0 2023-11-28T13:03:00.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.062+0000 7f3a82f39700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7c0cebe0 con 0x7f3a7c15c990 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.062+0000 7f3a82f39700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0cb3f0 con 0x7f3a7c161090 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.062+0000 7f3a80cd5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3a7c15c990 0x7f3a7c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.062+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 0x7f3a7c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.062+0000 7f3a814d6700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f3a7c0caeb0 0x7f3a7c0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:39284/0 (socket says 172.21.15.6:39284) 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.062+0000 7f3a814d6700 1 -- 172.21.15.6:0/1910270994 learned_addr learned my addr 172.21.15.6:0/1910270994 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.063+0000 7f3a7bfff700 1 -- 172.21.15.6:0/1910270994 >> v2:172.21.15.61:3300/0 conn(0x7f3a7c15c990 msgr2=0x7f3a7c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.063+0000 7f3a7bfff700 1 --2- 172.21.15.6:0/1910270994 >> v2:172.21.15.61:3300/0 conn(0x7f3a7c15c990 0x7f3a7c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.063+0000 7f3a7bfff700 1 -- 172.21.15.6:0/1910270994 >> v1:172.21.15.61:6789/0 conn(0x7f3a7c0caeb0 legacy=0x7f3a7c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.063+0000 7f3a7bfff700 1 -- 172.21.15.6:0/1910270994 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c0ce590 con 0x7f3a7c161090 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.063+0000 7f3a7bfff700 1 --2- 172.21.15.6:0/1910270994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 0x7f3a7c0ca970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009840 tx=0x7f3a68009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3090ab71918d586c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.063+0000 7f3a7b7fe700 1 -- 172.21.15.6:0/1910270994 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68011830 con 0x7f3a7c161090 2023-11-28T13:03:00.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.063+0000 7f3a7b7fe700 1 -- 172.21.15.6:0/1910270994 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a68011990 con 0x7f3a7c161090 2023-11-28T13:03:00.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.063+0000 7f3a7b7fe700 1 -- 172.21.15.6:0/1910270994 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68011d20 con 0x7f3a7c161090 2023-11-28T13:03:00.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.064+0000 7f3a82f39700 1 -- 172.21.15.6:0/1910270994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 msgr2=0x7f3a7c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.064+0000 7f3a82f39700 1 --2- 172.21.15.6:0/1910270994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 0x7f3a7c0ca970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009840 tx=0x7f3a68009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:00.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.064+0000 7f3a82f39700 1 -- 172.21.15.6:0/1910270994 shutdown_connections 2023-11-28T13:03:00.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.064+0000 7f3a82f39700 1 --2- 172.21.15.6:0/1910270994 >> v2:172.21.15.61:3300/0 conn(0x7f3a7c15c990 0x7f3a7c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.064+0000 7f3a82f39700 1 --2- 172.21.15.6:0/1910270994 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 0x7f3a7c0ca970 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.064+0000 7f3a82f39700 1 -- 172.21.15.6:0/1910270994 >> 172.21.15.6:0/1910270994 conn(0x7f3a7c0b1650 msgr2=0x7f3a7c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.064+0000 7f3a82f39700 1 -- 172.21.15.6:0/1910270994 shutdown_connections 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.064+0000 7f3a82f39700 1 -- 172.21.15.6:0/1910270994 wait complete. 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.065+0000 7f3a82f39700 1 Processor -- start 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.065+0000 7f3a82f39700 1 -- start start 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a82f39700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a7c0caeb0 0x7f3a7c0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a82f39700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c990 0x7f3a7c0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a82f39700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 0x7f3a7c1465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a82f39700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c159d90 con 0x7f3a7c15c990 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a82f39700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c15dcf0 con 0x7f3a7c0caeb0 2023-11-28T13:03:00.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a82f39700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c159ab0 con 0x7f3a7c161090 2023-11-28T13:03:00.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c990 0x7f3a7c0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c990 0x7f3a7c0b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33706/0 (socket says 172.21.15.6:33706) 2023-11-28T13:03:00.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a7bfff700 1 -- 172.21.15.6:0/2936822304 learned_addr learned my addr 172.21.15.6:0/2936822304 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:00.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a814d6700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 0x7f3a7c1465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a80cd5700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a7c0caeb0 0x7f3a7c0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a7bfff700 1 -- 172.21.15.6:0/2936822304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 msgr2=0x7f3a7c1465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a7bfff700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 0x7f3a7c1465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a7bfff700 1 -- 172.21.15.6:0/2936822304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a7c0caeb0 msgr2=0x7f3a7c0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a7bfff700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a7c0caeb0 0x7f3a7c0b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a7bfff700 1 -- 172.21.15.6:0/2936822304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680094a0 con 0x7f3a7c15c990 2023-11-28T13:03:00.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.066+0000 7f3a7bfff700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c990 0x7f3a7c0b2e90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009fd0 tx=0x7f3a68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.067+0000 7f3a797fa700 1 -- 172.21.15.6:0/2936822304 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a680036a0 con 0x7f3a7c15c990 2023-11-28T13:03:00.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.067+0000 7f3a797fa700 1 -- 172.21.15.6:0/2936822304 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6801e070 con 0x7f3a7c15c990 2023-11-28T13:03:00.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.067+0000 7f3a82f39700 1 -- 172.21.15.6:0/2936822304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c0cf5b0 con 0x7f3a7c15c990 2023-11-28T13:03:00.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.067+0000 7f3a797fa700 1 -- 172.21.15.6:0/2936822304 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68003a80 con 0x7f3a7c15c990 2023-11-28T13:03:00.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.067+0000 7f3a82f39700 1 -- 172.21.15.6:0/2936822304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c0ce870 con 0x7f3a7c15c990 2023-11-28T13:03:00.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.068+0000 7f3a82f39700 1 -- 172.21.15.6:0/2936822304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c0cf740 con 0x7f3a7c15c990 2023-11-28T13:03:00.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.068+0000 7f3a797fa700 1 -- 172.21.15.6:0/2936822304 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3a6800f050 con 0x7f3a7c15c990 2023-11-28T13:03:00.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.068+0000 7f3a797fa700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a600682f0 0x7f3a6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.069+0000 7f3a797fa700 1 -- 172.21.15.6:0/2936822304 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3a68020070 con 0x7f3a7c15c990 2023-11-28T13:03:00.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.069+0000 7f3a80cd5700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a600682f0 0x7f3a6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.069+0000 7f3a80cd5700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a600682f0 0x7f3a6006a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009be0 tx=0x7f3a70009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.073+0000 7f3a797fa700 1 -- 172.21.15.6:0/2936822304 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3a680b85e0 con 0x7f3a7c15c990 2023-11-28T13:03:00.312 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.311+0000 7f3a82f39700 1 -- 172.21.15.6:0/2936822304 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a7c0675a0 con 0x7f3a7c15c990 2023-11-28T13:03:00.313 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.312+0000 7f3a797fa700 1 -- 172.21.15.6:0/2936822304 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4636 (secure 0 0 0) 0x7f3a6802a090 con 0x7f3a7c15c990 2023-11-28T13:03:00.313 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:00.314 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4004512137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4004512137},{"type":"v1","addr":"172.21.15.61:6833","nonce":4004512137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/1313775424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":1313775424},{"type":"v1","addr":"172.21.15.61:6835","nonce":1313775424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/233781200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":233781200},{"type":"v1","addr":"172.21.15.6:6836","nonce":233781200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:31.562404+0000","modified":"2023-11-28T13:02:34.939442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/451011578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":451011578},{"type":"v1","addr":"172.21.15.6:6837","nonce":451011578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:03:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 -- 172.21.15.6:0/2936822304 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a600682f0 msgr2=0x7f3a6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a600682f0 0x7f3a6006a7a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009be0 tx=0x7f3a70009210 comp rx=0 tx=0).stop 2023-11-28T13:03:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 -- 172.21.15.6:0/2936822304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c990 msgr2=0x7f3a7c0b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c990 0x7f3a7c0b2e90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009fd0 tx=0x7f3a68004750 comp rx=0 tx=0).stop 2023-11-28T13:03:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 -- 172.21.15.6:0/2936822304 shutdown_connections 2023-11-28T13:03:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a600682f0 0x7f3a6006a7a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.315 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a7c161090 0x7f3a7c1465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a7c0caeb0 0x7f3a7c0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 --2- 172.21.15.6:0/2936822304 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a7c15c990 0x7f3a7c0b2e90 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 -- 172.21.15.6:0/2936822304 >> 172.21.15.6:0/2936822304 conn(0x7f3a7c0b1650 msgr2=0x7f3a7c0c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:00.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 -- 172.21.15.6:0/2936822304 shutdown_connections 2023-11-28T13:03:00.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.314+0000 7f3a82f39700 1 -- 172.21.15.6:0/2936822304 wait complete. 2023-11-28T13:03:00.316 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2023-11-28T13:03:00.327 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:03:00.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.452+0000 7f18f636b700 1 Processor -- start 2023-11-28T13:03:00.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.452+0000 7f18f636b700 1 -- start start 2023-11-28T13:03:00.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.452+0000 7f18f636b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 0x7f18f00ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.452+0000 7f18f636b700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f18f00caeb0 0x7f18f00cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.452+0000 7f18f636b700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f00ce340 con 0x7f18f015c990 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.452+0000 7f18f636b700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f00cebe0 con 0x7f18f00caeb0 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.452+0000 7f18f636b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f00cf780 con 0x7f18f0161090 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18effff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f18f015c990 0x7f18f00cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49264/0 (socket says 172.21.15.6:49264) 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18effff700 1 -- 172.21.15.6:0/3805858058 learned_addr learned my addr 172.21.15.6:0/3805858058 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18f4908700 1 --2- 172.21.15.6:0/3805858058 >> v2:172.21.15.61:3300/0 conn(0x7f18f00caeb0 0x7f18f00cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18ef7fe700 1 --2- 172.21.15.6:0/3805858058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 0x7f18f00ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18eeffd700 1 -- 172.21.15.6:0/3805858058 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755729853 0 0) 0x7f18f00ce340 con 0x7f18f015c990 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18eeffd700 1 -- 172.21.15.6:0/3805858058 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d80035b0 con 0x7f18f015c990 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18f4908700 1 -- 172.21.15.6:0/3805858058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 msgr2=0x7f18f00ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18f4908700 1 --2- 172.21.15.6:0/3805858058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 0x7f18f00ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18f4908700 1 -- 172.21.15.6:0/3805858058 >> v1:172.21.15.6:6789/0 conn(0x7f18f015c990 legacy=0x7f18f00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18f4908700 1 -- 172.21.15.6:0/3805858058 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f00cb3f0 con 0x7f18f00caeb0 2023-11-28T13:03:00.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18ef7fe700 1 --2- 172.21.15.6:0/3805858058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 0x7f18f00ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.453+0000 7f18f4908700 1 --2- 172.21.15.6:0/3805858058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18f00caeb0 0x7f18f00cc310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009d90 tx=0x7f18e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8de8fe093fa2fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18eeffd700 1 -- 172.21.15.6:0/3805858058 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e800e070 con 0x7f18f00caeb0 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18eeffd700 1 -- 172.21.15.6:0/3805858058 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e800bb20 con 0x7f18f00caeb0 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18eeffd700 1 -- 172.21.15.6:0/3805858058 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8012430 con 0x7f18f00caeb0 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18f636b700 1 -- 172.21.15.6:0/3805858058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18f00caeb0 msgr2=0x7f18f00cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18f636b700 1 --2- 172.21.15.6:0/3805858058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18f00caeb0 0x7f18f00cc310 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009d90 tx=0x7f18e800b330 comp rx=0 tx=0).stop 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18f636b700 1 -- 172.21.15.6:0/3805858058 shutdown_connections 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18f636b700 1 --2- 172.21.15.6:0/3805858058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18f00caeb0 0x7f18f00cc310 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18f636b700 1 --2- 172.21.15.6:0/3805858058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 0x7f18f00ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18f636b700 1 -- 172.21.15.6:0/3805858058 >> 172.21.15.6:0/3805858058 conn(0x7f18f00b1650 msgr2=0x7f18f00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18f636b700 1 -- 172.21.15.6:0/3805858058 shutdown_connections 2023-11-28T13:03:00.457 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.454+0000 7f18f636b700 1 -- 172.21.15.6:0/3805858058 wait complete. 2023-11-28T13:03:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.455+0000 7f18f636b700 1 Processor -- start 2023-11-28T13:03:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.456+0000 7f18f636b700 1 -- start start 2023-11-28T13:03:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.456+0000 7f18f636b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18f00caeb0 0x7f18f015e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.456+0000 7f18f636b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18f015c990 0x7f18f00c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.456+0000 7f18f636b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 0x7f18f015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.456+0000 7f18f636b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f015d2a0 con 0x7f18f00caeb0 2023-11-28T13:03:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.456+0000 7f18f636b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f015d580 con 0x7f18f015c990 2023-11-28T13:03:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.456+0000 7f18f636b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f015d860 con 0x7f18f0161090 2023-11-28T13:03:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.456+0000 7f18effff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18f00caeb0 0x7f18f015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.456+0000 7f18effff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18f00caeb0 0x7f18f015e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33716/0 (socket says 172.21.15.6:33716) 2023-11-28T13:03:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.456+0000 7f18effff700 1 -- 172.21.15.6:0/1882994174 learned_addr learned my addr 172.21.15.6:0/1882994174 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18f4908700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 0x7f18f015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18ef7fe700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18f015c990 0x7f18f00c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18effff700 1 -- 172.21.15.6:0/1882994174 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 msgr2=0x7f18f015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18effff700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 0x7f18f015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18effff700 1 -- 172.21.15.6:0/1882994174 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18f015c990 msgr2=0x7f18f00c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18effff700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18f015c990 0x7f18f00c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18effff700 1 -- 172.21.15.6:0/1882994174 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e40094a0 con 0x7f18f00caeb0 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18ef7fe700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18f015c990 0x7f18f00c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18effff700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18f00caeb0 0x7f18f015e340 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f18e000c4d0 tx=0x7f18e000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18ecff9700 1 -- 172.21.15.6:0/1882994174 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0005d70 con 0x7f18f00caeb0 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18ecff9700 1 -- 172.21.15.6:0/1882994174 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e00070d0 con 0x7f18f00caeb0 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18f636b700 1 -- 172.21.15.6:0/1882994174 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e8009a70 con 0x7f18f00caeb0 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18ecff9700 1 -- 172.21.15.6:0/1882994174 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0007440 con 0x7f18f00caeb0 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.457+0000 7f18f636b700 1 -- 172.21.15.6:0/1882994174 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f00ce870 con 0x7f18f00caeb0 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.458+0000 7f18f636b700 1 -- 172.21.15.6:0/1882994174 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f00ce590 con 0x7f18f00caeb0 2023-11-28T13:03:00.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.459+0000 7f18ecff9700 1 -- 172.21.15.6:0/1882994174 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f18e0019030 con 0x7f18f00caeb0 2023-11-28T13:03:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.459+0000 7f18ecff9700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18dc0682f0 0x7f18dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.459+0000 7f18ecff9700 1 -- 172.21.15.6:0/1882994174 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f18e0084fe0 con 0x7f18f00caeb0 2023-11-28T13:03:00.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.459+0000 7f18ef7fe700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18dc0682f0 0x7f18dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.459+0000 7f18ef7fe700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18dc0682f0 0x7f18dc06a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f18f015f020 tx=0x7f18e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.468 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.463+0000 7f18ecff9700 1 -- 172.21.15.6:0/1882994174 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f18e004b5e0 con 0x7f18f00caeb0 2023-11-28T13:03:00.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.659+0000 7f18f636b700 1 -- 172.21.15.6:0/1882994174 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18f00cc850 con 0x7f18f00caeb0 2023-11-28T13:03:00.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.660+0000 7f18ecff9700 1 -- 172.21.15.6:0/1882994174 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18026 (secure 0 0 0) 0x7f18e0017020 con 0x7f18f00caeb0 2023-11-28T13:03:00.662 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:00.662 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":54,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:32.564104+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:29.490442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:30.534532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:03:00.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 -- 172.21.15.6:0/1882994174 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18dc0682f0 msgr2=0x7f18dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18dc0682f0 0x7f18dc06a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f18f015f020 tx=0x7f18e4008040 comp rx=0 tx=0).stop 2023-11-28T13:03:00.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 -- 172.21.15.6:0/1882994174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18f00caeb0 msgr2=0x7f18f015e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18f00caeb0 0x7f18f015e340 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f18e000c4d0 tx=0x7f18e000c7e0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 -- 172.21.15.6:0/1882994174 shutdown_connections 2023-11-28T13:03:00.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18dc0682f0 0x7f18dc06a7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18f0161090 0x7f18f015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18f00caeb0 0x7f18f015e340 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 --2- 172.21.15.6:0/1882994174 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18f015c990 0x7f18f00c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 -- 172.21.15.6:0/1882994174 >> 172.21.15.6:0/1882994174 conn(0x7f18f00b1650 msgr2=0x7f18f00c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:00.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 -- 172.21.15.6:0/1882994174 shutdown_connections 2023-11-28T13:03:00.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.662+0000 7f18f636b700 1 -- 172.21.15.6:0/1882994174 wait complete. 2023-11-28T13:03:00.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:02:29.490442+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:02:30.534532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:03:00.677 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:03:00.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.803+0000 7f44fecbe700 1 Processor -- start 2023-11-28T13:03:00.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.803+0000 7f44fecbe700 1 -- start start 2023-11-28T13:03:00.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fecbe700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f44f80cf5f0 0x7f44f80caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fecbe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f44f80cb3e0 0x7f44f806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fecbe700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f80ce550 con 0x7f44f80b1630 2023-11-28T13:03:00.808 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fecbe700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f80ce830 con 0x7f44f80cb3e0 2023-11-28T13:03:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fecbe700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f80ceb10 con 0x7f44f80cf5f0 2023-11-28T13:03:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fd25b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f44f80cb3e0 0x7f44f806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fd25b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f44f80cb3e0 0x7f44f806afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33728/0 (socket says 172.21.15.6:33728) 2023-11-28T13:03:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fd25b700 1 -- 172.21.15.6:0/2859273063 learned_addr learned my addr 172.21.15.6:0/2859273063 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44f7fff700 1 --2- 172.21.15.6:0/2859273063 >> v2:172.21.15.61:3300/0 conn(0x7f44f80cf5f0 0x7f44f80caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fd25b700 1 -- 172.21.15.6:0/2859273063 >> v2:172.21.15.61:3300/0 conn(0x7f44f80cf5f0 msgr2=0x7f44f80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fd25b700 1 --2- 172.21.15.6:0/2859273063 >> v2:172.21.15.61:3300/0 conn(0x7f44f80cf5f0 0x7f44f80caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fd25b700 1 -- 172.21.15.6:0/2859273063 >> v1:172.21.15.61:6789/0 conn(0x7f44f80b1630 legacy=0x7f44f80b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.809 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fd25b700 1 -- 172.21.15.6:0/2859273063 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f80cedf0 con 0x7f44f80cb3e0 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.804+0000 7f44fd25b700 1 --2- 172.21.15.6:0/2859273063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cb3e0 0x7f44f806afc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009e50 tx=0x7f44f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cf0d0e0b894fb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44f77fe700 1 -- 172.21.15.6:0/2859273063 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f000e070 con 0x7f44f80cb3e0 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44f77fe700 1 -- 172.21.15.6:0/2859273063 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f000b8d0 con 0x7f44f80cb3e0 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44f77fe700 1 -- 172.21.15.6:0/2859273063 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f000bc80 con 0x7f44f80cb3e0 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44fecbe700 1 -- 172.21.15.6:0/2859273063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cb3e0 msgr2=0x7f44f806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44fecbe700 1 --2- 172.21.15.6:0/2859273063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cb3e0 0x7f44f806afc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009e50 tx=0x7f44f000b330 comp rx=0 tx=0).stop 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44fecbe700 1 -- 172.21.15.6:0/2859273063 shutdown_connections 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44fecbe700 1 --2- 172.21.15.6:0/2859273063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cb3e0 0x7f44f806afc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44fecbe700 1 --2- 172.21.15.6:0/2859273063 >> v2:172.21.15.61:3300/0 conn(0x7f44f80cf5f0 0x7f44f80caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44fecbe700 1 -- 172.21.15.6:0/2859273063 >> 172.21.15.6:0/2859273063 conn(0x7f44f80cc630 msgr2=0x7f44f80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44fecbe700 1 -- 172.21.15.6:0/2859273063 shutdown_connections 2023-11-28T13:03:00.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.805+0000 7f44fecbe700 1 -- 172.21.15.6:0/2859273063 wait complete. 2023-11-28T13:03:00.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.806+0000 7f44fecbe700 1 Processor -- start 2023-11-28T13:03:00.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.806+0000 7f44fecbe700 1 -- start start 2023-11-28T13:03:00.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.807+0000 7f44fecbe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44f80b1630 0x7f44f815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.807+0000 7f44fecbe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44f80cb3e0 0x7f44f81f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.807+0000 7f44fecbe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cf5f0 0x7f44f80c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.807+0000 7f44fecbe700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f815b060 con 0x7f44f80cf5f0 2023-11-28T13:03:00.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.807+0000 7f44fecbe700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80acb10 con 0x7f44f80b1630 2023-11-28T13:03:00.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.807+0000 7f44fecbe700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f80acdf0 con 0x7f44f80cb3e0 2023-11-28T13:03:00.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.807+0000 7f44fd25b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cf5f0 0x7f44f80c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.807+0000 7f44fd25b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cf5f0 0x7f44f80c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33734/0 (socket says 172.21.15.6:33734) 2023-11-28T13:03:00.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.807+0000 7f44fd25b700 1 -- 172.21.15.6:0/1599614555 learned_addr learned my addr 172.21.15.6:0/1599614555 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:00.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.807+0000 7f44f7fff700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44f80cb3e0 0x7f44f81f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44fca5a700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44f80b1630 0x7f44f815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44fd25b700 1 -- 172.21.15.6:0/1599614555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44f80cb3e0 msgr2=0x7f44f81f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44fd25b700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44f80cb3e0 0x7f44f81f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.814 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44fd25b700 1 -- 172.21.15.6:0/1599614555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44f80b1630 msgr2=0x7f44f815ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44fd25b700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44f80b1630 0x7f44f815ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44fd25b700 1 -- 172.21.15.6:0/1599614555 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f0009b30 con 0x7f44f80cf5f0 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44fd25b700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cf5f0 0x7f44f80c8ed0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f44f8066a20 tx=0x7f44f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44f57fa700 1 -- 172.21.15.6:0/1599614555 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f00092e0 con 0x7f44f80cf5f0 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44f57fa700 1 -- 172.21.15.6:0/1599614555 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f00099b0 con 0x7f44f80cf5f0 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44fecbe700 1 -- 172.21.15.6:0/1599614555 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f80ce140 con 0x7f44f80cf5f0 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44f57fa700 1 -- 172.21.15.6:0/1599614555 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f001c680 con 0x7f44f80cf5f0 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.808+0000 7f44fecbe700 1 -- 172.21.15.6:0/1599614555 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f80cf0d0 con 0x7f44f80cf5f0 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.809+0000 7f44fecbe700 1 -- 172.21.15.6:0/1599614555 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f80ce2d0 con 0x7f44f80cf5f0 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.809+0000 7f44f57fa700 1 -- 172.21.15.6:0/1599614555 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f44f002c070 con 0x7f44f80cf5f0 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.810+0000 7f44f57fa700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44dc0682f0 0x7f44dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:00.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.810+0000 7f44f57fa700 1 -- 172.21.15.6:0/1599614555 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f44f0087ad0 con 0x7f44f80cf5f0 2023-11-28T13:03:00.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.810+0000 7f44fca5a700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44dc0682f0 0x7f44dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:00.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.810+0000 7f44fca5a700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44dc0682f0 0x7f44dc06a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f44f80ac6c0 tx=0x7f44ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:00.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:00.814+0000 7f44f57fa700 1 -- 172.21.15.6:0/1599614555 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f44f0054170 con 0x7f44f80cf5f0 2023-11-28T13:03:01.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.006+0000 7f44fecbe700 1 -- 172.21.15.6:0/1599614555 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f44f80739f0 con 0x7f44f80cf5f0 2023-11-28T13:03:01.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.006+0000 7f44f57fa700 1 -- 172.21.15.6:0/1599614555 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f44f0057790 con 0x7f44f80cf5f0 2023-11-28T13:03:01.007 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:03:01.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 -- 172.21.15.6:0/1599614555 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44dc0682f0 msgr2=0x7f44dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44dc0682f0 0x7f44dc06a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f44f80ac6c0 tx=0x7f44ec009040 comp rx=0 tx=0).stop 2023-11-28T13:03:01.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 -- 172.21.15.6:0/1599614555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cf5f0 msgr2=0x7f44f80c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cf5f0 0x7f44f80c8ed0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f44f8066a20 tx=0x7f44f0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 -- 172.21.15.6:0/1599614555 shutdown_connections 2023-11-28T13:03:01.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44f80cf5f0 0x7f44f80c8ed0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f44f80b1630 0x7f44f815ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44dc0682f0 0x7f44dc06a7a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 --2- 172.21.15.6:0/1599614555 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f44f80cb3e0 0x7f44f81f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 -- 172.21.15.6:0/1599614555 >> 172.21.15.6:0/1599614555 conn(0x7f44f80cc630 msgr2=0x7f44f80cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:01.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 -- 172.21.15.6:0/1599614555 shutdown_connections 2023-11-28T13:03:01.010 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.008+0000 7f44fecbe700 1 -- 172.21.15.6:0/1599614555 wait complete. 2023-11-28T13:03:01.024 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:03:01.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.151+0000 7f4f8e0e7700 1 Processor -- start 2023-11-28T13:03:01.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.152+0000 7f4f8e0e7700 1 -- start start 2023-11-28T13:03:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.152+0000 7f4f8e0e7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4f880b2d90 0x7f4f880ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.152+0000 7f4f8e0e7700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f880cd2a0 con 0x7f4f880b62d0 2023-11-28T13:03:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.152+0000 7f4f8e0e7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f880d0400 con 0x7f4f880cb630 2023-11-28T13:03:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.152+0000 7f4f8e0e7700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f880d06e0 con 0x7f4f880b2d90 2023-11-28T13:03:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.152+0000 7f4f87fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4f880b2d90 0x7f4f880ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f877fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f4f880b62d0 0x7f4f8806f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49290/0 (socket says 172.21.15.6:49290) 2023-11-28T13:03:01.155 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f87fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4f880b2d90 0x7f4f880ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33742/0 (socket says 172.21.15.6:33742) 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f877fe700 1 -- 172.21.15.6:0/2233049627 learned_addr learned my addr 172.21.15.6:0/2233049627 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f87fff700 1 -- 172.21.15.6:0/2233049627 >> v1:172.21.15.61:6789/0 conn(0x7f4f880cb630 legacy=0x7f4f880b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f87fff700 1 -- 172.21.15.6:0/2233049627 >> v1:172.21.15.6:6789/0 conn(0x7f4f880b62d0 legacy=0x7f4f8806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f87fff700 1 -- 172.21.15.6:0/2233049627 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f880cffb0 con 0x7f4f880b2d90 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f867fc700 1 -- 172.21.15.6:0/2233049627 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306302576 0 0) 0x7f4f880cd2a0 con 0x7f4f880b62d0 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f87fff700 1 --2- 172.21.15.6:0/2233049627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880b2d90 0x7f4f880ce7d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009d90 tx=0x7f4f8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167d006c7759fd95 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f867fc700 1 -- 172.21.15.6:0/2233049627 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000e070 con 0x7f4f880b2d90 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f867fc700 1 -- 172.21.15.6:0/2233049627 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8000bbd0 con 0x7f4f880b2d90 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.153+0000 7f4f867fc700 1 -- 172.21.15.6:0/2233049627 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80012430 con 0x7f4f880b2d90 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.154+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/2233049627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880b2d90 msgr2=0x7f4f880ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.154+0000 7f4f8e0e7700 1 --2- 172.21.15.6:0/2233049627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880b2d90 0x7f4f880ce7d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009d90 tx=0x7f4f8000b330 comp rx=0 tx=0).stop 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.154+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/2233049627 shutdown_connections 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.154+0000 7f4f8e0e7700 1 --2- 172.21.15.6:0/2233049627 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880b2d90 0x7f4f880ce7d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.156 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.154+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/2233049627 >> 172.21.15.6:0/2233049627 conn(0x7f4f880cd680 msgr2=0x7f4f880cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.154+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/2233049627 shutdown_connections 2023-11-28T13:03:01.157 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.154+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/2233049627 wait complete. 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.155+0000 7f4f8e0e7700 1 Processor -- start 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.155+0000 7f4f8e0e7700 1 -- start start 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f8e0e7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f880b2d90 0x7f4f880c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f8e0e7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f880b62d0 0x7f4f880c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f8e0e7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880cb630 0x7f4f881ef680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f8e0e7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f880af580 con 0x7f4f880cb630 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f8e0e7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f880aecc0 con 0x7f4f880b2d90 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f8e0e7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f880aef30 con 0x7f4f880b62d0 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f87fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880cb630 0x7f4f881ef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f87fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880cb630 0x7f4f881ef680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33748/0 (socket says 172.21.15.6:33748) 2023-11-28T13:03:01.159 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f87fff700 1 -- 172.21.15.6:0/3334513693 learned_addr learned my addr 172.21.15.6:0/3334513693 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f86ffd700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f880b62d0 0x7f4f880c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f877fe700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f880b2d90 0x7f4f880c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f87fff700 1 -- 172.21.15.6:0/3334513693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f880b62d0 msgr2=0x7f4f880c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f87fff700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f880b62d0 0x7f4f880c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f87fff700 1 -- 172.21.15.6:0/3334513693 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f880b2d90 msgr2=0x7f4f880c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f87fff700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f880b2d90 0x7f4f880c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f87fff700 1 -- 172.21.15.6:0/3334513693 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80009a70 con 0x7f4f880cb630 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.156+0000 7f4f877fe700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f880b2d90 0x7f4f880c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.157+0000 7f4f87fff700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880cb630 0x7f4f881ef680 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4f80013000 tx=0x7f4f80004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.157+0000 7f4f7ffff700 1 -- 172.21.15.6:0/3334513693 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8001b810 con 0x7f4f880cb630 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.157+0000 7f4f7ffff700 1 -- 172.21.15.6:0/3334513693 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f80020070 con 0x7f4f880cb630 2023-11-28T13:03:01.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.157+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/3334513693 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f880cce50 con 0x7f4f880cb630 2023-11-28T13:03:01.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.157+0000 7f4f7ffff700 1 -- 172.21.15.6:0/3334513693 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8001bbf0 con 0x7f4f880cb630 2023-11-28T13:03:01.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.157+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/3334513693 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f880b66a0 con 0x7f4f880cb630 2023-11-28T13:03:01.161 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.158+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/3334513693 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f880ccfe0 con 0x7f4f880cb630 2023-11-28T13:03:01.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.158+0000 7f4f7ffff700 1 -- 172.21.15.6:0/3334513693 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4f8001bd50 con 0x7f4f880cb630 2023-11-28T13:03:01.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.159+0000 7f4f7ffff700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f680682f0 0x7f4f6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.159+0000 7f4f877fe700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f680682f0 0x7f4f6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.159+0000 7f4f7ffff700 1 -- 172.21.15.6:0/3334513693 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4f80022070 con 0x7f4f880cb630 2023-11-28T13:03:01.162 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.159+0000 7f4f877fe700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f680682f0 0x7f4f6806a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4f70000f60 tx=0x7f4f70007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:01.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.163+0000 7f4f7ffff700 1 -- 172.21.15.6:0/3334513693 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4f80054dc0 con 0x7f4f880cb630 2023-11-28T13:03:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.357+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/3334513693 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f880739f0 con 0x7f4f880cb630 2023-11-28T13:03:01.358 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.357+0000 7f4f7ffff700 1 -- 172.21.15.6:0/3334513693 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f800583e0 con 0x7f4f880cb630 2023-11-28T13:03:01.359 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:03:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/3334513693 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f680682f0 msgr2=0x7f4f6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f680682f0 0x7f4f6806a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4f70000f60 tx=0x7f4f70007ce0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/3334513693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880cb630 msgr2=0x7f4f881ef680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880cb630 0x7f4f881ef680 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4f80013000 tx=0x7f4f80004ad0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/3334513693 shutdown_connections 2023-11-28T13:03:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4f880cb630 0x7f4f881ef680 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4f880b2d90 0x7f4f880c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4f680682f0 0x7f4f6806a7a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 --2- 172.21.15.6:0/3334513693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4f880b62d0 0x7f4f880c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/3334513693 >> 172.21.15.6:0/3334513693 conn(0x7f4f880cd680 msgr2=0x7f4f880cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/3334513693 shutdown_connections 2023-11-28T13:03:01.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.359+0000 7f4f8e0e7700 1 -- 172.21.15.6:0/3334513693 wait complete. 2023-11-28T13:03:01.373 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:03:01.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.499+0000 7f07ddf9f700 1 Processor -- start 2023-11-28T13:03:01.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.499+0000 7f07ddf9f700 1 -- start start 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07ddf9f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80cb630 0x7f07d80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07ddf9f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f07d80b2bc0 0x7f07d80ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07ddf9f700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d80cd2a0 con 0x7f07d80b62d0 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07ddf9f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d80cba80 con 0x7f07d80b2bc0 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07ddf9f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80b3040 con 0x7f07d80cb630 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d77fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f07d80b62d0 0x7f07d806f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49304/0 (socket says 172.21.15.6:49304) 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d77fe700 1 -- 172.21.15.6:0/2954947160 learned_addr learned my addr 172.21.15.6:0/2954947160 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d6ffd700 1 --2- 172.21.15.6:0/2954947160 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80cb630 0x7f07d80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d7fff700 1 --2- 172.21.15.6:0/2954947160 >> v2:172.21.15.61:3300/0 conn(0x7f07d80b2bc0 0x7f07d80ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d67fc700 1 -- 172.21.15.6:0/2954947160 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613375890 0 0) 0x7f07d80cd2a0 con 0x7f07d80b62d0 2023-11-28T13:03:01.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d67fc700 1 -- 172.21.15.6:0/2954947160 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c40035b0 con 0x7f07d80b62d0 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d6ffd700 1 -- 172.21.15.6:0/2954947160 >> v2:172.21.15.61:3300/0 conn(0x7f07d80b2bc0 msgr2=0x7f07d80ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d6ffd700 1 --2- 172.21.15.6:0/2954947160 >> v2:172.21.15.61:3300/0 conn(0x7f07d80b2bc0 0x7f07d80ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d6ffd700 1 -- 172.21.15.6:0/2954947160 >> v1:172.21.15.6:6789/0 conn(0x7f07d80b62d0 legacy=0x7f07d806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d67fc700 1 -- 172.21.15.6:0/2954947160 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369634222 0 0) 0x7f07c40035b0 con 0x7f07d80b62d0 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.500+0000 7f07d6ffd700 1 -- 172.21.15.6:0/2954947160 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d80d0400 con 0x7f07d80cb630 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07d6ffd700 1 --2- 172.21.15.6:0/2954947160 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80cb630 0x7f07d80b2680 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009840 tx=0x7f07c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fad36fdab37fcb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07d67fc700 1 -- 172.21.15.6:0/2954947160 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8001670 con 0x7f07d80cb630 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07d67fc700 1 -- 172.21.15.6:0/2954947160 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c0004030 con 0x7f07d80cb630 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07d67fc700 1 -- 172.21.15.6:0/2954947160 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c00043c0 con 0x7f07d80cb630 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2954947160 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80cb630 msgr2=0x7f07d80b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07ddf9f700 1 --2- 172.21.15.6:0/2954947160 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80cb630 0x7f07d80b2680 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009840 tx=0x7f07c0009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:01.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2954947160 shutdown_connections 2023-11-28T13:03:01.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07ddf9f700 1 --2- 172.21.15.6:0/2954947160 >> v2:172.21.15.61:3300/0 conn(0x7f07d80b2bc0 0x7f07d80ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07ddf9f700 1 --2- 172.21.15.6:0/2954947160 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80cb630 0x7f07d80b2680 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2954947160 >> 172.21.15.6:0/2954947160 conn(0x7f07d80cd680 msgr2=0x7f07d80cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:01.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2954947160 shutdown_connections 2023-11-28T13:03:01.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.501+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2954947160 wait complete. 2023-11-28T13:03:01.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.503+0000 7f07ddf9f700 1 Processor -- start 2023-11-28T13:03:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.503+0000 7f07ddf9f700 1 -- start start 2023-11-28T13:03:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07ddf9f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80b2bc0 0x7f07d81e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07ddf9f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07d80b62d0 0x7f07d81ef590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07ddf9f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d80cb630 0x7f07d80b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07ddf9f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8146990 con 0x7f07d80cb630 2023-11-28T13:03:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07ddf9f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8146c70 con 0x7f07d80b62d0 2023-11-28T13:03:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07ddf9f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8146f50 con 0x7f07d80b2bc0 2023-11-28T13:03:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d80cb630 0x7f07d80b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d80cb630 0x7f07d80b2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33762/0 (socket says 172.21.15.6:33762) 2023-11-28T13:03:01.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d7fff700 1 -- 172.21.15.6:0/2149773383 learned_addr learned my addr 172.21.15.6:0/2149773383 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d77fe700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80b2bc0 0x7f07d81e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d6ffd700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07d80b62d0 0x7f07d81ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d7fff700 1 -- 172.21.15.6:0/2149773383 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80b2bc0 msgr2=0x7f07d81e3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d7fff700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80b2bc0 0x7f07d81e3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d7fff700 1 -- 172.21.15.6:0/2149773383 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07d80b62d0 msgr2=0x7f07d81ef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.509 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d7fff700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07d80b62d0 0x7f07d81ef590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d7fff700 1 -- 172.21.15.6:0/2149773383 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c00094a0 con 0x7f07d80cb630 2023-11-28T13:03:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.504+0000 7f07d7fff700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d80cb630 0x7f07d80b2020 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f07d000efc0 tx=0x7f07d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.505+0000 7f07cffff700 1 -- 172.21.15.6:0/2149773383 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d000cd40 con 0x7f07d80cb630 2023-11-28T13:03:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.505+0000 7f07cffff700 1 -- 172.21.15.6:0/2149773383 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d000f040 con 0x7f07d80cb630 2023-11-28T13:03:01.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.505+0000 7f07cffff700 1 -- 172.21.15.6:0/2149773383 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d00146c0 con 0x7f07d80cb630 2023-11-28T13:03:01.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.505+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2149773383 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d80cffb0 con 0x7f07d80cb630 2023-11-28T13:03:01.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.505+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2149773383 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d80d06e0 con 0x7f07d80cb630 2023-11-28T13:03:01.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.506+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2149773383 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d80b66a0 con 0x7f07d80cb630 2023-11-28T13:03:01.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.506+0000 7f07cffff700 1 -- 172.21.15.6:0/2149773383 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f07d0014820 con 0x7f07d80cb630 2023-11-28T13:03:01.511 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.506+0000 7f07cffff700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07b80682f0 0x7f07b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.507+0000 7f07cffff700 1 -- 172.21.15.6:0/2149773383 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f07d00869b0 con 0x7f07d80cb630 2023-11-28T13:03:01.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.507+0000 7f07d77fe700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07b80682f0 0x7f07b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.507+0000 7f07d77fe700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07b80682f0 0x7f07b806a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f07c8001eb0 tx=0x7f07c8001ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:01.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.511+0000 7f07cffff700 1 -- 172.21.15.6:0/2149773383 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f07d0052f60 con 0x7f07d80cb630 2023-11-28T13:03:01.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.706+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2149773383 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07d80739f0 con 0x7f07d80cb630 2023-11-28T13:03:01.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.707+0000 7f07cffff700 1 -- 172.21.15.6:0/2149773383 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07d000cea0 con 0x7f07d80cb630 2023-11-28T13:03:01.708 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:03:01.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2149773383 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07b80682f0 msgr2=0x7f07b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07b80682f0 0x7f07b806a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f07c8001eb0 tx=0x7f07c8001ce0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2149773383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d80cb630 msgr2=0x7f07d80b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d80cb630 0x7f07d80b2020 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f07d000efc0 tx=0x7f07d000c3a0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2149773383 shutdown_connections 2023-11-28T13:03:01.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07d80cb630 0x7f07d80b2020 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07b80682f0 0x7f07b806a7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07d80b2bc0 0x7f07d81e3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 --2- 172.21.15.6:0/2149773383 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07d80b62d0 0x7f07d81ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2149773383 >> 172.21.15.6:0/2149773383 conn(0x7f07d80cd680 msgr2=0x7f07d80b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:01.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2149773383 shutdown_connections 2023-11-28T13:03:01.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.709+0000 7f07ddf9f700 1 -- 172.21.15.6:0/2149773383 wait complete. 2023-11-28T13:03:01.723 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:03:01.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.849+0000 7f7668272700 1 Processor -- start 2023-11-28T13:03:01.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.849+0000 7f7668272700 1 -- start start 2023-11-28T13:03:01.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f7668272700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f76600b1d40 0x7f76600cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f7668272700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f766015dbf0 0x7f76600ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f7668272700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76600ceff0 con 0x7f7660159500 2023-11-28T13:03:01.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f7668272700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f766015e090 con 0x7f76600b1d40 2023-11-28T13:03:01.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f7668272700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76600cede0 con 0x7f766015dbf0 2023-11-28T13:03:01.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f766580d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f76600b1d40 0x7f76600cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f766580d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f76600b1d40 0x7f76600cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33768/0 (socket says 172.21.15.6:33768) 2023-11-28T13:03:01.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f766580d700 1 -- 172.21.15.6:0/905827082 learned_addr learned my addr 172.21.15.6:0/905827082 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:01.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f766680f700 1 --2- 172.21.15.6:0/905827082 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f766015dbf0 0x7f76600ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f766580d700 1 -- 172.21.15.6:0/905827082 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f766015dbf0 msgr2=0x7f76600ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:01.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f766580d700 1 --2- 172.21.15.6:0/905827082 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f766015dbf0 0x7f76600ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f766580d700 1 -- 172.21.15.6:0/905827082 >> v1:172.21.15.61:6789/0 conn(0x7f7660159500 legacy=0x7f76600b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:01.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.850+0000 7f766580d700 1 -- 172.21.15.6:0/905827082 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76600cf2d0 con 0x7f76600b1d40 2023-11-28T13:03:01.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f766580d700 1 --2- 172.21.15.6:0/905827082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76600b1d40 0x7f76600cd140 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f765c00bb90 tx=0x7f765c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873e077655c1e060 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:01.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f766500c700 1 -- 172.21.15.6:0/905827082 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c00dd10 con 0x7f76600b1d40 2023-11-28T13:03:01.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f766500c700 1 -- 172.21.15.6:0/905827082 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765c00de70 con 0x7f76600b1d40 2023-11-28T13:03:01.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f766500c700 1 -- 172.21.15.6:0/905827082 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765c004230 con 0x7f76600b1d40 2023-11-28T13:03:01.855 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f7668272700 1 -- 172.21.15.6:0/905827082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76600b1d40 msgr2=0x7f76600cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f7668272700 1 --2- 172.21.15.6:0/905827082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76600b1d40 0x7f76600cd140 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f765c00bb90 tx=0x7f765c00bea0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f7668272700 1 -- 172.21.15.6:0/905827082 shutdown_connections 2023-11-28T13:03:01.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f7668272700 1 --2- 172.21.15.6:0/905827082 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f766015dbf0 0x7f76600ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f7668272700 1 --2- 172.21.15.6:0/905827082 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76600b1d40 0x7f76600cd140 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f7668272700 1 -- 172.21.15.6:0/905827082 >> 172.21.15.6:0/905827082 conn(0x7f76600cc630 msgr2=0x7f76600cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:01.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f7668272700 1 -- 172.21.15.6:0/905827082 shutdown_connections 2023-11-28T13:03:01.856 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.851+0000 7f7668272700 1 -- 172.21.15.6:0/905827082 wait complete. 2023-11-28T13:03:01.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.853+0000 7f7668272700 1 Processor -- start 2023-11-28T13:03:01.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.853+0000 7f7668272700 1 -- start start 2023-11-28T13:03:01.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.853+0000 7f7668272700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76600b1d40 0x7f76601519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.853+0000 7f7668272700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7660159500 0x7f76600c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.853+0000 7f7668272700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f766015dbf0 0x7f7660157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.853+0000 7f7668272700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76600c4200 con 0x7f766015dbf0 2023-11-28T13:03:01.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.853+0000 7f7668272700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76600c5bd0 con 0x7f76600b1d40 2023-11-28T13:03:01.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.853+0000 7f7668272700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76600c5eb0 con 0x7f7660159500 2023-11-28T13:03:01.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766680f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f766015dbf0 0x7f7660157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766680f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f766015dbf0 0x7f7660157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33776/0 (socket says 172.21.15.6:33776) 2023-11-28T13:03:01.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766680f700 1 -- 172.21.15.6:0/1318479481 learned_addr learned my addr 172.21.15.6:0/1318479481 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:01.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766600e700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76600b1d40 0x7f76601519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766580d700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7660159500 0x7f76600c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766680f700 1 -- 172.21.15.6:0/1318479481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7660159500 msgr2=0x7f76600c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766680f700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7660159500 0x7f76600c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766680f700 1 -- 172.21.15.6:0/1318479481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76600b1d40 msgr2=0x7f76601519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:01.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766680f700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76600b1d40 0x7f76601519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:01.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766680f700 1 -- 172.21.15.6:0/1318479481 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c00b7f0 con 0x7f766015dbf0 2023-11-28T13:03:01.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f766680f700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f766015dbf0 0x7f7660157180 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7658007e40 tx=0x7f765800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:01.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f7656ffd700 1 -- 172.21.15.6:0/1318479481 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765800eea0 con 0x7f766015dbf0 2023-11-28T13:03:01.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f7656ffd700 1 -- 172.21.15.6:0/1318479481 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7658014420 con 0x7f766015dbf0 2023-11-28T13:03:01.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f7656ffd700 1 -- 172.21.15.6:0/1318479481 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76580147b0 con 0x7f766015dbf0 2023-11-28T13:03:01.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f7668272700 1 -- 172.21.15.6:0/1318479481 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766015d3f0 con 0x7f766015dbf0 2023-11-28T13:03:01.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.854+0000 7f7668272700 1 -- 172.21.15.6:0/1318479481 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766015d110 con 0x7f766015dbf0 2023-11-28T13:03:01.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.855+0000 7f7668272700 1 -- 172.21.15.6:0/1318479481 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766015d6d0 con 0x7f766015dbf0 2023-11-28T13:03:01.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.856+0000 7f7656ffd700 1 -- 172.21.15.6:0/1318479481 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7658012050 con 0x7f766015dbf0 2023-11-28T13:03:01.862 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.856+0000 7f7656ffd700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76440682f0 0x7f764406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:01.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.856+0000 7f7656ffd700 1 -- 172.21.15.6:0/1318479481 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f76580864a0 con 0x7f766015dbf0 2023-11-28T13:03:01.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.856+0000 7f766600e700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76440682f0 0x7f764406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:01.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.857+0000 7f766600e700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76440682f0 0x7f764406a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7650004f30 tx=0x7f7650009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:01.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:01.860+0000 7f7656ffd700 1 -- 172.21.15.6:0/1318479481 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7658052bc0 con 0x7f766015dbf0 2023-11-28T13:03:02.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.051+0000 7f7668272700 1 -- 172.21.15.6:0/1318479481 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76600c63a0 con 0x7f766015dbf0 2023-11-28T13:03:02.053 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.052+0000 7f7656ffd700 1 -- 172.21.15.6:0/1318479481 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f76580561e0 con 0x7f766015dbf0 2023-11-28T13:03:02.053 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:03:02.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 -- 172.21.15.6:0/1318479481 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76440682f0 msgr2=0x7f764406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76440682f0 0x7f764406a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7650004f30 tx=0x7f7650009040 comp rx=0 tx=0).stop 2023-11-28T13:03:02.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 -- 172.21.15.6:0/1318479481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f766015dbf0 msgr2=0x7f7660157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f766015dbf0 0x7f7660157180 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7658007e40 tx=0x7f765800e3c0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 -- 172.21.15.6:0/1318479481 shutdown_connections 2023-11-28T13:03:02.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f766015dbf0 0x7f7660157180 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76600b1d40 0x7f76601519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76440682f0 0x7f764406a7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 --2- 172.21.15.6:0/1318479481 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7660159500 0x7f76600c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 -- 172.21.15.6:0/1318479481 >> 172.21.15.6:0/1318479481 conn(0x7f76600cc630 msgr2=0x7f766006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:02.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 -- 172.21.15.6:0/1318479481 shutdown_connections 2023-11-28T13:03:02.056 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.054+0000 7f7668272700 1 -- 172.21.15.6:0/1318479481 wait complete. 2023-11-28T13:03:02.067 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:02.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.191+0000 7f14b37f2700 1 Processor -- start 2023-11-28T13:03:02.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.191+0000 7f14b37f2700 1 -- start start 2023-11-28T13:03:02.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b37f2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14ac15c990 0x7f14ac0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b37f2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f14ac161090 0x7f14ac0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b37f2700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ac0ce340 con 0x7f14ac0caeb0 2023-11-28T13:03:02.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b37f2700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac0cebe0 con 0x7f14ac161090 2023-11-28T13:03:02.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b37f2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac0cb3f0 con 0x7f14ac15c990 2023-11-28T13:03:02.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b0d8d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f14ac161090 0x7f14ac0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b0d8d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f14ac161090 0x7f14ac0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33780/0 (socket says 172.21.15.6:33780) 2023-11-28T13:03:02.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b0d8d700 1 -- 172.21.15.6:0/3027121657 learned_addr learned my addr 172.21.15.6:0/3027121657 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:02.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b1d8f700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f14ac0caeb0 0x7f14ac0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49334/0 (socket says 172.21.15.6:49334) 2023-11-28T13:03:02.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b0d8d700 1 -- 172.21.15.6:0/3027121657 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14ac15c990 msgr2=0x7f14ac0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:02.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b0d8d700 1 --2- 172.21.15.6:0/3027121657 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14ac15c990 0x7f14ac0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b0d8d700 1 -- 172.21.15.6:0/3027121657 >> v1:172.21.15.6:6789/0 conn(0x7f14ac0caeb0 legacy=0x7f14ac0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14a3fff700 1 -- 172.21.15.6:0/3027121657 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066652804 0 0) 0x7f14ac0ce340 con 0x7f14ac0caeb0 2023-11-28T13:03:02.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b0d8d700 1 -- 172.21.15.6:0/3027121657 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac0ce590 con 0x7f14ac161090 2023-11-28T13:03:02.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14b0d8d700 1 --2- 172.21.15.6:0/3027121657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac161090 0x7f14ac0ca970 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f149c0098f0 tx=0x7f149c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c11aadfa78e7062 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:02.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.192+0000 7f14a3fff700 1 -- 172.21.15.6:0/3027121657 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00b8a0 con 0x7f14ac161090 2023-11-28T13:03:02.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.193+0000 7f14a3fff700 1 -- 172.21.15.6:0/3027121657 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149c00ba00 con 0x7f14ac161090 2023-11-28T13:03:02.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.193+0000 7f14a3fff700 1 -- 172.21.15.6:0/3027121657 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149c00bd90 con 0x7f14ac161090 2023-11-28T13:03:02.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.193+0000 7f14b37f2700 1 -- 172.21.15.6:0/3027121657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac161090 msgr2=0x7f14ac0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.193+0000 7f14b37f2700 1 --2- 172.21.15.6:0/3027121657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac161090 0x7f14ac0ca970 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f149c0098f0 tx=0x7f149c009c00 comp rx=0 tx=0).stop 2023-11-28T13:03:02.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.193+0000 7f14b37f2700 1 -- 172.21.15.6:0/3027121657 shutdown_connections 2023-11-28T13:03:02.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.193+0000 7f14b37f2700 1 --2- 172.21.15.6:0/3027121657 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14ac15c990 0x7f14ac0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.193+0000 7f14b37f2700 1 --2- 172.21.15.6:0/3027121657 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac161090 0x7f14ac0ca970 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.193+0000 7f14b37f2700 1 -- 172.21.15.6:0/3027121657 >> 172.21.15.6:0/3027121657 conn(0x7f14ac0b1650 msgr2=0x7f14ac0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:02.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.193+0000 7f14b37f2700 1 -- 172.21.15.6:0/3027121657 shutdown_connections 2023-11-28T13:03:02.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.193+0000 7f14b37f2700 1 -- 172.21.15.6:0/3027121657 wait complete. 2023-11-28T13:03:02.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.195+0000 7f14b37f2700 1 Processor -- start 2023-11-28T13:03:02.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.195+0000 7f14b37f2700 1 -- start start 2023-11-28T13:03:02.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b37f2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac0caeb0 0x7f14ac0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b37f2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14ac15c990 0x7f14ac15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b37f2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14ac161090 0x7f14ac1f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b37f2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac0b4430 con 0x7f14ac0caeb0 2023-11-28T13:03:02.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b37f2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac1f6480 con 0x7f14ac15c990 2023-11-28T13:03:02.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b37f2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac1f6760 con 0x7f14ac161090 2023-11-28T13:03:02.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b158e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac0caeb0 0x7f14ac0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b158e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac0caeb0 0x7f14ac0b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33784/0 (socket says 172.21.15.6:33784) 2023-11-28T13:03:02.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b158e700 1 -- 172.21.15.6:0/3065352363 learned_addr learned my addr 172.21.15.6:0/3065352363 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:02.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b0d8d700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14ac15c990 0x7f14ac15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b1d8f700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14ac161090 0x7f14ac1f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b158e700 1 -- 172.21.15.6:0/3065352363 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14ac161090 msgr2=0x7f14ac1f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b158e700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14ac161090 0x7f14ac1f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b158e700 1 -- 172.21.15.6:0/3065352363 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14ac15c990 msgr2=0x7f14ac15a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b158e700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14ac15c990 0x7f14ac15a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b158e700 1 -- 172.21.15.6:0/3065352363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c009550 con 0x7f14ac0caeb0 2023-11-28T13:03:02.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b0d8d700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14ac15c990 0x7f14ac15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:02.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14b158e700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac0caeb0 0x7f14ac0b3ef0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f14a800b530 tx=0x7f14a800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:02.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.196+0000 7f14a1ffb700 1 -- 172.21.15.6:0/3065352363 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8011830 con 0x7f14ac0caeb0 2023-11-28T13:03:02.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.197+0000 7f14a1ffb700 1 -- 172.21.15.6:0/3065352363 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a80119b0 con 0x7f14ac0caeb0 2023-11-28T13:03:02.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.197+0000 7f14b37f2700 1 -- 172.21.15.6:0/3065352363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ac0cf5b0 con 0x7f14ac0caeb0 2023-11-28T13:03:02.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.197+0000 7f14a1ffb700 1 -- 172.21.15.6:0/3065352363 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8011d60 con 0x7f14ac0caeb0 2023-11-28T13:03:02.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.197+0000 7f14b37f2700 1 -- 172.21.15.6:0/3065352363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac0ce870 con 0x7f14ac0caeb0 2023-11-28T13:03:02.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.198+0000 7f14b37f2700 1 -- 172.21.15.6:0/3065352363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac0cf740 con 0x7f14ac0caeb0 2023-11-28T13:03:02.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.198+0000 7f14a1ffb700 1 -- 172.21.15.6:0/3065352363 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f14a8021070 con 0x7f14ac0caeb0 2023-11-28T13:03:02.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.198+0000 7f14a1ffb700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f14900682f0 0x7f149006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.198+0000 7f14a1ffb700 1 -- 172.21.15.6:0/3065352363 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f14a8085d10 con 0x7f14ac0caeb0 2023-11-28T13:03:02.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.198+0000 7f14b0d8d700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f14900682f0 0x7f149006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.199+0000 7f14b0d8d700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f14900682f0 0x7f149006a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f14ac1614c0 tx=0x7f149c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:02.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.203+0000 7f14a1ffb700 1 -- 172.21.15.6:0/3065352363 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f14a8052430 con 0x7f14ac0caeb0 2023-11-28T13:03:02.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.442+0000 7f14b37f2700 1 -- 172.21.15.6:0/3065352363 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14ac0c6b50 con 0x7f14ac0caeb0 2023-11-28T13:03:02.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.442+0000 7f14a1ffb700 1 -- 172.21.15.6:0/3065352363 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4636 (secure 0 0 0) 0x7f14ac0c6b50 con 0x7f14ac0caeb0 2023-11-28T13:03:02.444 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:02.444 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4004512137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4004512137},{"type":"v1","addr":"172.21.15.61:6833","nonce":4004512137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/1313775424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":1313775424},{"type":"v1","addr":"172.21.15.61:6835","nonce":1313775424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/233781200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":233781200},{"type":"v1","addr":"172.21.15.6:6836","nonce":233781200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:31.562404+0000","modified":"2023-11-28T13:02:34.939442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/451011578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":451011578},{"type":"v1","addr":"172.21.15.6:6837","nonce":451011578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:03:02.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.444+0000 7f14b37f2700 1 -- 172.21.15.6:0/3065352363 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f14900682f0 msgr2=0x7f149006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.444+0000 7f14b37f2700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f14900682f0 0x7f149006a7a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f14ac1614c0 tx=0x7f149c008030 comp rx=0 tx=0).stop 2023-11-28T13:03:02.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.444+0000 7f14b37f2700 1 -- 172.21.15.6:0/3065352363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac0caeb0 msgr2=0x7f14ac0b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.444+0000 7f14b37f2700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac0caeb0 0x7f14ac0b3ef0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f14a800b530 tx=0x7f14a800b8f0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.444+0000 7f14b37f2700 1 -- 172.21.15.6:0/3065352363 shutdown_connections 2023-11-28T13:03:02.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.444+0000 7f14b37f2700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f14900682f0 0x7f149006a7a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.444+0000 7f14b37f2700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f14ac161090 0x7f14ac1f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.444+0000 7f14b37f2700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f14ac0caeb0 0x7f14ac0b3ef0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.445+0000 7f14b37f2700 1 --2- 172.21.15.6:0/3065352363 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14ac15c990 0x7f14ac15a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.445+0000 7f14b37f2700 1 -- 172.21.15.6:0/3065352363 >> 172.21.15.6:0/3065352363 conn(0x7f14ac0b1650 msgr2=0x7f14ac0b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:02.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.445+0000 7f14b37f2700 1 -- 172.21.15.6:0/3065352363 shutdown_connections 2023-11-28T13:03:02.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.445+0000 7f14b37f2700 1 -- 172.21.15.6:0/3065352363 wait complete. 2023-11-28T13:03:02.448 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2023-11-28T13:03:02.459 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:03:02.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.582+0000 7f06e006e700 1 Processor -- start 2023-11-28T13:03:02.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.582+0000 7f06e006e700 1 -- start start 2023-11-28T13:03:02.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.583+0000 7f06e006e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f06d815a5a0 0x7f06d806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.583+0000 7f06e006e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06d80ca280 0x7f06d80cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06e006e700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d80cb570 con 0x7f06d80cbf40 2023-11-28T13:03:02.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06e006e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d80ca6f0 con 0x7f06d815a5a0 2023-11-28T13:03:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06e006e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80ca960 con 0x7f06d80ca280 2023-11-28T13:03:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dde0a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f06d815a5a0 0x7f06d806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dde0a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f06d815a5a0 0x7f06d806b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33792/0 (socket says 172.21.15.6:33792) 2023-11-28T13:03:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06de60b700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f06d80cbf40 0x7f06d80cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49348/0 (socket says 172.21.15.6:49348) 2023-11-28T13:03:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dde0a700 1 -- 172.21.15.6:0/2251855700 learned_addr learned my addr 172.21.15.6:0/2251855700 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dd609700 1 --2- 172.21.15.6:0/2251855700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06d80ca280 0x7f06d80cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dde0a700 1 -- 172.21.15.6:0/2251855700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06d80ca280 msgr2=0x7f06d80cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dde0a700 1 --2- 172.21.15.6:0/2251855700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06d80ca280 0x7f06d80cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dde0a700 1 -- 172.21.15.6:0/2251855700 >> v1:172.21.15.6:6789/0 conn(0x7f06d80cbf40 legacy=0x7f06d80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dce08700 1 -- 172.21.15.6:0/2251855700 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511144503 0 0) 0x7f06d80cb570 con 0x7f06d80cbf40 2023-11-28T13:03:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dde0a700 1 -- 172.21.15.6:0/2251855700 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d80cac40 con 0x7f06d815a5a0 2023-11-28T13:03:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dde0a700 1 --2- 172.21.15.6:0/2251855700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d815a5a0 0x7f06d806b930 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f06c80098f0 tx=0x7f06c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6873f6400889a1e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dce08700 1 -- 172.21.15.6:0/2251855700 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8011830 con 0x7f06d815a5a0 2023-11-28T13:03:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dce08700 1 -- 172.21.15.6:0/2251855700 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c8011990 con 0x7f06d815a5a0 2023-11-28T13:03:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.584+0000 7f06dce08700 1 -- 172.21.15.6:0/2251855700 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8011d20 con 0x7f06d815a5a0 2023-11-28T13:03:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.585+0000 7f06e006e700 1 -- 172.21.15.6:0/2251855700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d815a5a0 msgr2=0x7f06d806b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.585+0000 7f06e006e700 1 --2- 172.21.15.6:0/2251855700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d815a5a0 0x7f06d806b930 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f06c80098f0 tx=0x7f06c8009c00 comp rx=0 tx=0).stop 2023-11-28T13:03:02.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.585+0000 7f06e006e700 1 -- 172.21.15.6:0/2251855700 shutdown_connections 2023-11-28T13:03:02.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.585+0000 7f06e006e700 1 --2- 172.21.15.6:0/2251855700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d815a5a0 0x7f06d806b930 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.585+0000 7f06e006e700 1 --2- 172.21.15.6:0/2251855700 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06d80ca280 0x7f06d80cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.585+0000 7f06e006e700 1 -- 172.21.15.6:0/2251855700 >> 172.21.15.6:0/2251855700 conn(0x7f06d80b1650 msgr2=0x7f06d80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:02.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.585+0000 7f06e006e700 1 -- 172.21.15.6:0/2251855700 shutdown_connections 2023-11-28T13:03:02.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.585+0000 7f06e006e700 1 -- 172.21.15.6:0/2251855700 wait complete. 2023-11-28T13:03:02.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.586+0000 7f06e006e700 1 Processor -- start 2023-11-28T13:03:02.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.586+0000 7f06e006e700 1 -- start start 2023-11-28T13:03:02.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06e006e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d80ca280 0x7f06d80c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06e006e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06d80cbf40 0x7f06d814c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06e006e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06d815a5a0 0x7f06d80c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06e006e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80b2950 con 0x7f06d80ca280 2023-11-28T13:03:02.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06e006e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80b2ba0 con 0x7f06d80cbf40 2023-11-28T13:03:02.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06e006e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80b2e80 con 0x7f06d815a5a0 2023-11-28T13:03:02.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06dde0a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d80ca280 0x7f06d80c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06dde0a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d80ca280 0x7f06d80c5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33798/0 (socket says 172.21.15.6:33798) 2023-11-28T13:03:02.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06dde0a700 1 -- 172.21.15.6:0/3008930075 learned_addr learned my addr 172.21.15.6:0/3008930075 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:02.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06de60b700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06d815a5a0 0x7f06d80c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06dd609700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06d80cbf40 0x7f06d814c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06dde0a700 1 -- 172.21.15.6:0/3008930075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06d815a5a0 msgr2=0x7f06d80c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06dde0a700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06d815a5a0 0x7f06d80c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06dde0a700 1 -- 172.21.15.6:0/3008930075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06d80cbf40 msgr2=0x7f06d814c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06dde0a700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06d80cbf40 0x7f06d814c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.587+0000 7f06dde0a700 1 -- 172.21.15.6:0/3008930075 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c8009550 con 0x7f06d80ca280 2023-11-28T13:03:02.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.588+0000 7f06dde0a700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d80ca280 0x7f06d80c5490 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009c40 tx=0x7f06c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:02.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.588+0000 7f06ce7fc700 1 -- 172.21.15.6:0/3008930075 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8004480 con 0x7f06d80ca280 2023-11-28T13:03:02.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.588+0000 7f06ce7fc700 1 -- 172.21.15.6:0/3008930075 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c801f070 con 0x7f06d80ca280 2023-11-28T13:03:02.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.588+0000 7f06ce7fc700 1 -- 172.21.15.6:0/3008930075 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80038c0 con 0x7f06d80ca280 2023-11-28T13:03:02.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.588+0000 7f06e006e700 1 -- 172.21.15.6:0/3008930075 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d80cb200 con 0x7f06d80ca280 2023-11-28T13:03:02.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.588+0000 7f06e006e700 1 -- 172.21.15.6:0/3008930075 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d80caf20 con 0x7f06d80ca280 2023-11-28T13:03:02.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.589+0000 7f06e006e700 1 -- 172.21.15.6:0/3008930075 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d815a970 con 0x7f06d80ca280 2023-11-28T13:03:02.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.589+0000 7f06ce7fc700 1 -- 172.21.15.6:0/3008930075 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f06c802a020 con 0x7f06d80ca280 2023-11-28T13:03:02.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.590+0000 7f06ce7fc700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06bc0682f0 0x7f06bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.590+0000 7f06ce7fc700 1 -- 172.21.15.6:0/3008930075 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f06c8087410 con 0x7f06d80ca280 2023-11-28T13:03:02.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.590+0000 7f06dd609700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06bc0682f0 0x7f06bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.590+0000 7f06dd609700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06bc0682f0 0x7f06bc06a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f06d8156500 tx=0x7f06d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:02.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.594+0000 7f06ce7fc700 1 -- 172.21.15.6:0/3008930075 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f06c8053b30 con 0x7f06d80ca280 2023-11-28T13:03:02.789 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.788+0000 7f06e006e700 1 -- 172.21.15.6:0/3008930075 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06d80675a0 con 0x7f06d80ca280 2023-11-28T13:03:02.790 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.789+0000 7f06ce7fc700 1 -- 172.21.15.6:0/3008930075 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18026 (secure 0 0 0) 0x7f06c8057150 con 0x7f06d80ca280 2023-11-28T13:03:02.791 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:02.791 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":54,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:32.564104+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:29.490442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:30.534532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:03:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 -- 172.21.15.6:0/3008930075 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06bc0682f0 msgr2=0x7f06bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06bc0682f0 0x7f06bc06a7a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f06d8156500 tx=0x7f06d4008040 comp rx=0 tx=0).stop 2023-11-28T13:03:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 -- 172.21.15.6:0/3008930075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d80ca280 msgr2=0x7f06d80c5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d80ca280 0x7f06d80c5490 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009c40 tx=0x7f06c8004750 comp rx=0 tx=0).stop 2023-11-28T13:03:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 -- 172.21.15.6:0/3008930075 shutdown_connections 2023-11-28T13:03:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06bc0682f0 0x7f06bc06a7a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06d815a5a0 0x7f06d80c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f06d80ca280 0x7f06d80c5490 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 --2- 172.21.15.6:0/3008930075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06d80cbf40 0x7f06d814c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 -- 172.21.15.6:0/3008930075 >> 172.21.15.6:0/3008930075 conn(0x7f06d80b1650 msgr2=0x7f06d80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 -- 172.21.15.6:0/3008930075 shutdown_connections 2023-11-28T13:03:02.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.791+0000 7f06e006e700 1 -- 172.21.15.6:0/3008930075 wait complete. 2023-11-28T13:03:02.805 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:03:02.805 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:02.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.929+0000 7f07bd371700 1 Processor -- start 2023-11-28T13:03:02.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.929+0000 7f07bd371700 1 -- start start 2023-11-28T13:03:02.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07bd371700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f07b80d2820 0x7f07b80b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07bd371700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b80d1470 con 0x7f07b815c640 2023-11-28T13:03:02.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07bd371700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b80cd4e0 con 0x7f07b806bf30 2023-11-28T13:03:02.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07bd371700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b815ca10 con 0x7f07b80d2820 2023-11-28T13:03:02.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b6ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f07b815c640 0x7f07b80cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49358/0 (socket says 172.21.15.6:49358) 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b6ffd700 1 -- 172.21.15.6:0/709174070 learned_addr learned my addr 172.21.15.6:0/709174070 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305922190 0 0) 0x7f07b80d1470 con 0x7f07b815c640 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b67fc700 1 --2- 172.21.15.6:0/709174070 >> v2:172.21.15.61:3300/0 conn(0x7f07b80d2820 0x7f07b80b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07a00035b0 con 0x7f07b815c640 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 230373728 0 0) 0x7f07a00035b0 con 0x7f07b815c640 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 >> v2:172.21.15.61:3300/0 conn(0x7f07b80d2820 msgr2=0x7f07b80b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b5ffb700 1 --2- 172.21.15.6:0/709174070 >> v2:172.21.15.61:3300/0 conn(0x7f07b80d2820 0x7f07b80b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 >> v1:172.21.15.61:6789/0 conn(0x7f07b806bf30 legacy=0x7f07b806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b80cd7c0 con 0x7f07b815c640 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f07a8001670 con 0x7f07b815c640 2023-11-28T13:03:02.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.930+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 >> v1:172.21.15.6:6789/0 conn(0x7f07b815c640 legacy=0x7f07b80cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b5ffb700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07a0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b5ffb700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a0004850 0x7f07a0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b5ffb700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07a0005200 0x7f07a000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b815ca10 con 0x7f07a0003a00 2023-11-28T13:03:02.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a000c060 con 0x7f07a0005200 2023-11-28T13:03:02.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a000c220 con 0x7f07a0004850 2023-11-28T13:03:02.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b6ffd700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07a0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b77fe700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07a0005200 0x7f07a000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b67fc700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a0004850 0x7f07a0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b6ffd700 1 -- 172.21.15.6:0/709174070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a0004850 msgr2=0x7f07a0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:02.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b6ffd700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a0004850 0x7f07a0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b6ffd700 1 -- 172.21.15.6:0/709174070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07a0005200 msgr2=0x7f07a000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:02.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b6ffd700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07a0005200 0x7f07a000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b6ffd700 1 -- 172.21.15.6:0/709174070 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00035b0 con 0x7f07a0003a00 2023-11-28T13:03:02.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b77fe700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07a0005200 0x7f07a000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:02.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b67fc700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07a0004850 0x7f07a0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:02.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b77fe700 1 -- 172.21.15.6:0/709174070 reap_dead start 2023-11-28T13:03:02.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b6ffd700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07a0003e10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f07a800c2a0 tx=0x7f07a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f14b214f31d90c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:02.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8007c10 con 0x7f07a0003a00 2023-11-28T13:03:02.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a800cdc0 con 0x7f07a0003a00 2023-11-28T13:03:02.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.931+0000 7f07b5ffb700 1 -- 172.21.15.6:0/709174070 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a8005b80 con 0x7f07a0003a00 2023-11-28T13:03:02.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.932+0000 7f07bd371700 1 -- 172.21.15.6:0/709174070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 msgr2=0x7f07a0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.932+0000 7f07bd371700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07a0003e10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f07a800c2a0 tx=0x7f07a0005880 comp rx=0 tx=0).stop 2023-11-28T13:03:02.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.932+0000 7f07bd371700 1 -- 172.21.15.6:0/709174070 shutdown_connections 2023-11-28T13:03:02.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.932+0000 7f07bd371700 1 --2- 172.21.15.6:0/709174070 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07a0003e10 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.932+0000 7f07bd371700 1 -- 172.21.15.6:0/709174070 >> 172.21.15.6:0/709174070 conn(0x7f07b80d2df0 msgr2=0x7f07b80d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:02.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.932+0000 7f07bd371700 1 -- 172.21.15.6:0/709174070 shutdown_connections 2023-11-28T13:03:02.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.932+0000 7f07bd371700 1 -- 172.21.15.6:0/709174070 wait complete. 2023-11-28T13:03:02.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.934+0000 7f07bd371700 1 Processor -- start 2023-11-28T13:03:02.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.934+0000 7f07bd371700 1 -- start start 2023-11-28T13:03:02.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07bd371700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07b8147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07bd371700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b80cbca0 0x7f07b81f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07bd371700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07b80b38f0 0x7f07b80b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07bd371700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80ca210 con 0x7f07a0003a00 2023-11-28T13:03:02.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07bd371700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b80b42a0 con 0x7f07b80b38f0 2023-11-28T13:03:02.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07bd371700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b8148200 con 0x7f07b80cbca0 2023-11-28T13:03:02.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07b8147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07b8147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33818/0 (socket says 172.21.15.6:33818) 2023-11-28T13:03:02.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b6ffd700 1 -- 172.21.15.6:0/553631385 learned_addr learned my addr 172.21.15.6:0/553631385 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:02.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b77fe700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07b80b38f0 0x7f07b80b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b67fc700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b80cbca0 0x7f07b81f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b6ffd700 1 -- 172.21.15.6:0/553631385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b80cbca0 msgr2=0x7f07b81f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b6ffd700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b80cbca0 0x7f07b81f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b6ffd700 1 -- 172.21.15.6:0/553631385 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07b80b38f0 msgr2=0x7f07b80b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:02.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b6ffd700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07b80b38f0 0x7f07b80b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:02.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b6ffd700 1 -- 172.21.15.6:0/553631385 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b80cdaa0 con 0x7f07a0003a00 2023-11-28T13:03:02.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07b6ffd700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07b8147cc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c030 tx=0x7f07b80d2cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:02.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f079f7fe700 1 -- 172.21.15.6:0/553631385 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80070d0 con 0x7f07a0003a00 2023-11-28T13:03:02.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f079f7fe700 1 -- 172.21.15.6:0/553631385 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a8007230 con 0x7f07a0003a00 2023-11-28T13:03:02.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07bd371700 1 -- 172.21.15.6:0/553631385 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b80d1710 con 0x7f07a0003a00 2023-11-28T13:03:02.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f079f7fe700 1 -- 172.21.15.6:0/553631385 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80075c0 con 0x7f07a0003a00 2023-11-28T13:03:02.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.935+0000 7f07bd371700 1 -- 172.21.15.6:0/553631385 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b80cdc30 con 0x7f07a0003a00 2023-11-28T13:03:02.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.936+0000 7f07bd371700 1 -- 172.21.15.6:0/553631385 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b8068570 con 0x7f07a0003a00 2023-11-28T13:03:02.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.937+0000 7f079f7fe700 1 -- 172.21.15.6:0/553631385 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f07a8010070 con 0x7f07a0003a00 2023-11-28T13:03:02.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.937+0000 7f079f7fe700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07a40682f0 0x7f07a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:02.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.937+0000 7f079f7fe700 1 -- 172.21.15.6:0/553631385 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f07a80864a0 con 0x7f07a0003a00 2023-11-28T13:03:02.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.937+0000 7f07b67fc700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07a40682f0 0x7f07a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:02.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.938+0000 7f07b67fc700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07a40682f0 0x7f07a406a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f07b80d0c70 tx=0x7f07ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:02.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:02.942+0000 7f079f7fe700 1 -- 172.21.15.6:0/553631385 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f07a8052b40 con 0x7f07a0003a00 2023-11-28T13:03:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.184+0000 7f07bd371700 1 -- 172.21.15.6:0/553631385 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07b815fdb0 con 0x7f07a0003a00 2023-11-28T13:03:03.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.185+0000 7f079f7fe700 1 -- 172.21.15.6:0/553631385 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4636 (secure 0 0 0) 0x7f07a8007d70 con 0x7f07a0003a00 2023-11-28T13:03:03.186 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:03.186 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4004512137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4004512137},{"type":"v1","addr":"172.21.15.61:6833","nonce":4004512137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/1313775424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":1313775424},{"type":"v1","addr":"172.21.15.61:6835","nonce":1313775424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/233781200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":233781200},{"type":"v1","addr":"172.21.15.6:6836","nonce":233781200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:31.562404+0000","modified":"2023-11-28T13:02:34.939442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/451011578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":451011578},{"type":"v1","addr":"172.21.15.6:6837","nonce":451011578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:03:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 -- 172.21.15.6:0/553631385 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07a40682f0 msgr2=0x7f07a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07a40682f0 0x7f07a406a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f07b80d0c70 tx=0x7f07ac008040 comp rx=0 tx=0).stop 2023-11-28T13:03:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 -- 172.21.15.6:0/553631385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 msgr2=0x7f07b8147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07b8147cc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c030 tx=0x7f07b80d2cc0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 -- 172.21.15.6:0/553631385 shutdown_connections 2023-11-28T13:03:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07b80b38f0 0x7f07b80b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07a0003a00 0x7f07b8147cc0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07a40682f0 0x7f07a406a7a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 --2- 172.21.15.6:0/553631385 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b80cbca0 0x7f07b81f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 -- 172.21.15.6:0/553631385 >> 172.21.15.6:0/553631385 conn(0x7f07b80d2df0 msgr2=0x7f07b80b8320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 -- 172.21.15.6:0/553631385 shutdown_connections 2023-11-28T13:03:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.187+0000 7f07bd371700 1 -- 172.21.15.6:0/553631385 wait complete. 2023-11-28T13:03:03.190 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2023-11-28T13:03:03.201 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:03.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.324+0000 7f462e700700 1 Processor -- start 2023-11-28T13:03:03.326 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.324+0000 7f462e700700 1 -- start start 2023-11-28T13:03:03.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.325+0000 7f462e700700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f462815a6c0 0x7f46280cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.325+0000 7f462e700700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46280cfb90 con 0x7f46280b2660 2023-11-28T13:03:03.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.325+0000 7f462e700700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46280cb930 con 0x7f46280d0020 2023-11-28T13:03:03.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.325+0000 7f462e700700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46280cf260 con 0x7f462815a6c0 2023-11-28T13:03:03.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.325+0000 7f462cc9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f46280b2660 0x7f46280b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49382/0 (socket says 172.21.15.6:49382) 2023-11-28T13:03:03.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.325+0000 7f462cc9d700 1 -- 172.21.15.6:0/667150977 learned_addr learned my addr 172.21.15.6:0/667150977 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:03.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.325+0000 7f4627fff700 1 --2- 172.21.15.6:0/667150977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f462815a6c0 0x7f46280cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.328 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.326+0000 7f4626ffd700 1 -- 172.21.15.6:0/667150977 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148623038 0 0) 0x7f46280cfb90 con 0x7f46280b2660 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.326+0000 7f4626ffd700 1 -- 172.21.15.6:0/667150977 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46140035b0 con 0x7f46280b2660 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.326+0000 7f4626ffd700 1 -- 172.21.15.6:0/667150977 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515604520 0 0) 0x7f46280cb930 con 0x7f46280d0020 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.326+0000 7f4627fff700 1 -- 172.21.15.6:0/667150977 >> v1:172.21.15.61:6789/0 conn(0x7f46280d0020 legacy=0x7f46280ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.326+0000 7f4627fff700 1 -- 172.21.15.6:0/667150977 >> v1:172.21.15.6:6789/0 conn(0x7f46280b2660 legacy=0x7f46280b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.326+0000 7f4627fff700 1 -- 172.21.15.6:0/667150977 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46280cf540 con 0x7f462815a6c0 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.326+0000 7f4627fff700 1 --2- 172.21.15.6:0/667150977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f462815a6c0 0x7f46280cb230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4618009870 tx=0x7f4618009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eecc118f740f3c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.326+0000 7f4626ffd700 1 -- 172.21.15.6:0/667150977 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618011830 con 0x7f462815a6c0 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.326+0000 7f4626ffd700 1 -- 172.21.15.6:0/667150977 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4618011990 con 0x7f462815a6c0 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.326+0000 7f4626ffd700 1 -- 172.21.15.6:0/667150977 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618011d20 con 0x7f462815a6c0 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.327+0000 7f462e700700 1 -- 172.21.15.6:0/667150977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f462815a6c0 msgr2=0x7f46280cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.327+0000 7f462e700700 1 --2- 172.21.15.6:0/667150977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f462815a6c0 0x7f46280cb230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4618009870 tx=0x7f4618009b80 comp rx=0 tx=0).stop 2023-11-28T13:03:03.329 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.327+0000 7f462e700700 1 -- 172.21.15.6:0/667150977 shutdown_connections 2023-11-28T13:03:03.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.327+0000 7f462e700700 1 --2- 172.21.15.6:0/667150977 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f462815a6c0 0x7f46280cb230 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.327+0000 7f462e700700 1 -- 172.21.15.6:0/667150977 >> 172.21.15.6:0/667150977 conn(0x7f46280cd660 msgr2=0x7f46280cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:03.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.327+0000 7f462e700700 1 -- 172.21.15.6:0/667150977 shutdown_connections 2023-11-28T13:03:03.330 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.327+0000 7f462e700700 1 -- 172.21.15.6:0/667150977 wait complete. 2023-11-28T13:03:03.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.328+0000 7f462e700700 1 Processor -- start 2023-11-28T13:03:03.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.328+0000 7f462e700700 1 -- start start 2023-11-28T13:03:03.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462e700700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46280b2660 0x7f462814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462e700700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f46280d0020 0x7f46280c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462e700700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462815a6c0 0x7f46281ef400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462e700700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46280caf40 con 0x7f462815a6c0 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462e700700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46280ca0f0 con 0x7f46280d0020 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462e700700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46280ca330 con 0x7f46280b2660 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462cc9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462815a6c0 0x7f46281ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462cc9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462815a6c0 0x7f46281ef400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33836/0 (socket says 172.21.15.6:33836) 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462cc9d700 1 -- 172.21.15.6:0/1624606228 learned_addr learned my addr 172.21.15.6:0/1624606228 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f4627fff700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46280b2660 0x7f462814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f46277fe700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f46280d0020 0x7f46280c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462cc9d700 1 -- 172.21.15.6:0/1624606228 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46280b2660 msgr2=0x7f462814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462cc9d700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46280b2660 0x7f462814bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462cc9d700 1 -- 172.21.15.6:0/1624606228 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f46280d0020 msgr2=0x7f46280c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462cc9d700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f46280d0020 0x7f46280c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.329+0000 7f462cc9d700 1 -- 172.21.15.6:0/1624606228 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180094a0 con 0x7f462815a6c0 2023-11-28T13:03:03.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.330+0000 7f462cc9d700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462815a6c0 0x7f46281ef400 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f462000cb40 tx=0x7f462000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:03.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.330+0000 7f4624ff9700 1 -- 172.21.15.6:0/1624606228 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000aaf0 con 0x7f462815a6c0 2023-11-28T13:03:03.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.330+0000 7f462e700700 1 -- 172.21.15.6:0/1624606228 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46280d0470 con 0x7f462815a6c0 2023-11-28T13:03:03.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.330+0000 7f4624ff9700 1 -- 172.21.15.6:0/1624606228 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462000ac50 con 0x7f462815a6c0 2023-11-28T13:03:03.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.330+0000 7f4624ff9700 1 -- 172.21.15.6:0/1624606228 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620012400 con 0x7f462815a6c0 2023-11-28T13:03:03.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.330+0000 7f462e700700 1 -- 172.21.15.6:0/1624606228 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46280cf820 con 0x7f462815a6c0 2023-11-28T13:03:03.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.331+0000 7f462e700700 1 -- 172.21.15.6:0/1624606228 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46280d0600 con 0x7f462815a6c0 2023-11-28T13:03:03.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.332+0000 7f4624ff9700 1 -- 172.21.15.6:0/1624606228 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4620019030 con 0x7f462815a6c0 2023-11-28T13:03:03.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.332+0000 7f4624ff9700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46080682f0 0x7f460806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.332+0000 7f4624ff9700 1 -- 172.21.15.6:0/1624606228 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4620085ff0 con 0x7f462815a6c0 2023-11-28T13:03:03.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.332+0000 7f4627fff700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46080682f0 0x7f460806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.332+0000 7f4627fff700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46080682f0 0x7f460806a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f46180090b0 tx=0x7f461801c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:03.337 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.336+0000 7f4624ff9700 1 -- 172.21.15.6:0/1624606228 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f46200525e0 con 0x7f462815a6c0 2023-11-28T13:03:03.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.574+0000 7f462e700700 1 -- 172.21.15.6:0/1624606228 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4628149230 con 0x7f462815a6c0 2023-11-28T13:03:03.576 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.575+0000 7f4624ff9700 1 -- 172.21.15.6:0/1624606228 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4636 (secure 0 0 0) 0x7f4620055c00 con 0x7f462815a6c0 2023-11-28T13:03:03.576 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:03.576 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4004512137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4004512137},{"type":"v1","addr":"172.21.15.61:6833","nonce":4004512137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/1313775424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":1313775424},{"type":"v1","addr":"172.21.15.61:6835","nonce":1313775424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/233781200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":233781200},{"type":"v1","addr":"172.21.15.6:6836","nonce":233781200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:02:31.562404+0000","modified":"2023-11-28T13:02:34.939442+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6837/451011578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":451011578},{"type":"v1","addr":"172.21.15.6:6837","nonce":451011578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T13:03:03.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.576+0000 7f462e700700 1 -- 172.21.15.6:0/1624606228 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46080682f0 msgr2=0x7f460806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.576+0000 7f462e700700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46080682f0 0x7f460806a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f46180090b0 tx=0x7f461801c040 comp rx=0 tx=0).stop 2023-11-28T13:03:03.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.577+0000 7f462e700700 1 -- 172.21.15.6:0/1624606228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462815a6c0 msgr2=0x7f46281ef400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.577+0000 7f462e700700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462815a6c0 0x7f46281ef400 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f462000cb40 tx=0x7f462000cf00 comp rx=0 tx=0).stop 2023-11-28T13:03:03.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.577+0000 7f462e700700 1 -- 172.21.15.6:0/1624606228 shutdown_connections 2023-11-28T13:03:03.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.577+0000 7f462e700700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f462815a6c0 0x7f46281ef400 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.577+0000 7f462e700700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f46080682f0 0x7f460806a7a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.577+0000 7f462e700700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f46280b2660 0x7f462814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.577+0000 7f462e700700 1 --2- 172.21.15.6:0/1624606228 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f46280d0020 0x7f46280c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.577+0000 7f462e700700 1 -- 172.21.15.6:0/1624606228 >> 172.21.15.6:0/1624606228 conn(0x7f46280cd660 msgr2=0x7f46280b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:03.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.577+0000 7f462e700700 1 -- 172.21.15.6:0/1624606228 shutdown_connections 2023-11-28T13:03:03.579 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.577+0000 7f462e700700 1 -- 172.21.15.6:0/1624606228 wait complete. 2023-11-28T13:03:03.579 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 24 2023-11-28T13:03:03.591 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:03:03.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.714+0000 7fd6f102d700 1 Processor -- start 2023-11-28T13:03:03.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.714+0000 7fd6f102d700 1 -- start start 2023-11-28T13:03:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6f102d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6ec0cb470 0x7fd6ec0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6f102d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd6ec06bb80 0x7fd6ec14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6f102d700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec06c140 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6f102d700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec0caad0 con 0x7fd6ec0cb470 2023-11-28T13:03:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6f102d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec0c9ab0 con 0x7fd6ec06bb80 2023-11-28T13:03:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6ea59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6ec0cb470 0x7fd6ec0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6ead9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd6ec0cb0a0 0x7fd6ec15a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49392/0 (socket says 172.21.15.6:49392) 2023-11-28T13:03:03.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6ead9d700 1 -- 172.21.15.6:0/2145453028 learned_addr learned my addr 172.21.15.6:0/2145453028 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6ea59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6ec0cb470 0x7fd6ec0ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33842/0 (socket says 172.21.15.6:33842) 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6eb59e700 1 --2- 172.21.15.6:0/2145453028 >> v2:172.21.15.61:3300/0 conn(0x7fd6ec06bb80 0x7fd6ec14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6ea59c700 1 -- 172.21.15.6:0/2145453028 >> v2:172.21.15.61:3300/0 conn(0x7fd6ec06bb80 msgr2=0x7fd6ec14fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6ea59c700 1 --2- 172.21.15.6:0/2145453028 >> v2:172.21.15.61:3300/0 conn(0x7fd6ec06bb80 0x7fd6ec14fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6ea59c700 1 -- 172.21.15.6:0/2145453028 >> v1:172.21.15.6:6789/0 conn(0x7fd6ec0cb0a0 legacy=0x7fd6ec15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6ea59c700 1 -- 172.21.15.6:0/2145453028 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0c9d90 con 0x7fd6ec0cb470 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.715+0000 7fd6e9d9b700 1 -- 172.21.15.6:0/2145453028 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092921977 0 0) 0x7fd6ec06c140 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6ea59c700 1 --2- 172.21.15.6:0/2145453028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb470 0x7fd6ec0ca590 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec070fa0 tx=0x7fd6dc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810b0bfcf455fc65 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6e9d9b700 1 -- 172.21.15.6:0/2145453028 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc011830 con 0x7fd6ec0cb470 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6e9d9b700 1 -- 172.21.15.6:0/2145453028 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc011990 con 0x7fd6ec0cb470 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6e9d9b700 1 -- 172.21.15.6:0/2145453028 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc011d20 con 0x7fd6ec0cb470 2023-11-28T13:03:03.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6f102d700 1 -- 172.21.15.6:0/2145453028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb470 msgr2=0x7fd6ec0ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6f102d700 1 --2- 172.21.15.6:0/2145453028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb470 0x7fd6ec0ca590 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec070fa0 tx=0x7fd6dc009b20 comp rx=0 tx=0).stop 2023-11-28T13:03:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6f102d700 1 -- 172.21.15.6:0/2145453028 shutdown_connections 2023-11-28T13:03:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6f102d700 1 --2- 172.21.15.6:0/2145453028 >> v2:172.21.15.61:3300/0 conn(0x7fd6ec06bb80 0x7fd6ec14fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6f102d700 1 --2- 172.21.15.6:0/2145453028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb470 0x7fd6ec0ca590 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6f102d700 1 -- 172.21.15.6:0/2145453028 >> 172.21.15.6:0/2145453028 conn(0x7fd6ec0b0dd0 msgr2=0x7fd6ec0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6f102d700 1 -- 172.21.15.6:0/2145453028 shutdown_connections 2023-11-28T13:03:03.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.716+0000 7fd6f102d700 1 -- 172.21.15.6:0/2145453028 wait complete. 2023-11-28T13:03:03.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.718+0000 7fd6f102d700 1 Processor -- start 2023-11-28T13:03:03.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.718+0000 7fd6f102d700 1 -- start start 2023-11-28T13:03:03.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.719+0000 7fd6f102d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6ec06bb80 0x7fd6ec156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.719+0000 7fd6f102d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb0a0 0x7fd6ec0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.719+0000 7fd6f102d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6ec0cb470 0x7fd6ec14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.719+0000 7fd6f102d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1565c0 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.719+0000 7fd6f102d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0b07a0 con 0x7fd6ec06bb80 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.719+0000 7fd6f102d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0b09f0 con 0x7fd6ec0cb470 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.719+0000 7fd6ea59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb0a0 0x7fd6ec0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6ea59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb0a0 0x7fd6ec0b0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33848/0 (socket says 172.21.15.6:33848) 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6ea59c700 1 -- 172.21.15.6:0/4123590419 learned_addr learned my addr 172.21.15.6:0/4123590419 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6ead9d700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6ec06bb80 0x7fd6ec156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6eb59e700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6ec0cb470 0x7fd6ec14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6ea59c700 1 -- 172.21.15.6:0/4123590419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6ec0cb470 msgr2=0x7fd6ec14b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6ea59c700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6ec0cb470 0x7fd6ec14b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6ea59c700 1 -- 172.21.15.6:0/4123590419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6ec06bb80 msgr2=0x7fd6ec156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6ea59c700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6ec06bb80 0x7fd6ec156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6ea59c700 1 -- 172.21.15.6:0/4123590419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0094a0 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6ea59c700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb0a0 0x7fd6ec0b0260 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc014040 tx=0x7fd6dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6d37fe700 1 -- 172.21.15.6:0/4123590419 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0038a0 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6d37fe700 1 -- 172.21.15.6:0/4123590419 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc01f070 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6f102d700 1 -- 172.21.15.6:0/4123590419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec1503e0 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6d37fe700 1 -- 172.21.15.6:0/4123590419 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc003c00 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.720+0000 7fd6f102d700 1 -- 172.21.15.6:0/4123590419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec0ca070 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.721+0000 7fd6f102d700 1 -- 172.21.15.6:0/4123590419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec150570 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.722+0000 7fd6d37fe700 1 -- 172.21.15.6:0/4123590419 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd6dc00f050 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.722+0000 7fd6d37fe700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6d80682f0 0x7fd6d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:03.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.722+0000 7fd6d37fe700 1 -- 172.21.15.6:0/4123590419 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd6dc021070 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.722+0000 7fd6ead9d700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6d80682f0 0x7fd6d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:03.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.722+0000 7fd6ead9d700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6d80682f0 0x7fd6d806a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0000f60 tx=0x7fd6e0007be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:03.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.726+0000 7fd6d37fe700 1 -- 172.21.15.6:0/4123590419 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6dc053e10 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.922+0000 7fd6f102d700 1 -- 172.21.15.6:0/4123590419 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6ec0675a0 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.923+0000 7fd6d37fe700 1 -- 172.21.15.6:0/4123590419 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18026 (secure 0 0 0) 0x7fd6dc057430 con 0x7fd6ec0cb0a0 2023-11-28T13:03:03.924 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:03.925 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":54,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:02:32.564104+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:02:29.490442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T13:02:30.534532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 -- 172.21.15.6:0/4123590419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6d80682f0 msgr2=0x7fd6d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6d80682f0 0x7fd6d806a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0000f60 tx=0x7fd6e0007be0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 -- 172.21.15.6:0/4123590419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb0a0 msgr2=0x7fd6ec0b0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb0a0 0x7fd6ec0b0260 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc014040 tx=0x7fd6dc004750 comp rx=0 tx=0).stop 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 -- 172.21.15.6:0/4123590419 shutdown_connections 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6d80682f0 0x7fd6d806a7a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6ec0cb470 0x7fd6ec14b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6ec06bb80 0x7fd6ec156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 --2- 172.21.15.6:0/4123590419 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6ec0cb0a0 0x7fd6ec0b0260 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 -- 172.21.15.6:0/4123590419 >> 172.21.15.6:0/4123590419 conn(0x7fd6ec0b0dd0 msgr2=0x7fd6ec14e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:03.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 -- 172.21.15.6:0/4123590419 shutdown_connections 2023-11-28T13:03:03.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:03.925+0000 7fd6f102d700 1 -- 172.21.15.6:0/4123590419 wait complete. 2023-11-28T13:03:03.939 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:03:04.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.065+0000 7f13ff160700 1 Processor -- start 2023-11-28T13:03:04.066 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.065+0000 7f13ff160700 1 -- start start 2023-11-28T13:03:04.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13ff160700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f13f815c990 0x7f13f80cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13ff160700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f13f80cc310 0x7f13f80cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13ff160700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f80ce340 con 0x7f13f8161090 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13ff160700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f80cebe0 con 0x7f13f80cc310 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13ff160700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f80cccb0 con 0x7f13f815c990 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13fd6fd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f13f80cc310 0x7f13f80cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13fd6fd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f13f80cc310 0x7f13f80cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33858/0 (socket says 172.21.15.6:33858) 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13fd6fd700 1 -- 172.21.15.6:0/1535775550 learned_addr learned my addr 172.21.15.6:0/1535775550 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13fcefc700 1 --2- 172.21.15.6:0/1535775550 >> v2:172.21.15.61:3300/0 conn(0x7f13f815c990 0x7f13f80cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13fd6fd700 1 -- 172.21.15.6:0/1535775550 >> v2:172.21.15.61:3300/0 conn(0x7f13f815c990 msgr2=0x7f13f80cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13fd6fd700 1 --2- 172.21.15.6:0/1535775550 >> v2:172.21.15.61:3300/0 conn(0x7f13f815c990 0x7f13f80cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13fd6fd700 1 -- 172.21.15.6:0/1535775550 >> v1:172.21.15.61:6789/0 conn(0x7f13f8161090 legacy=0x7f13f80ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13fd6fd700 1 -- 172.21.15.6:0/1535775550 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f80cb3f0 con 0x7f13f80cc310 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13fcefc700 1 --2- 172.21.15.6:0/1535775550 >> v2:172.21.15.61:3300/0 conn(0x7f13f815c990 0x7f13f80cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.066+0000 7f13fd6fd700 1 --2- 172.21.15.6:0/1535775550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f80cc310 0x7f13f80cc770 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f13f806ea90 tx=0x7f13f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1015b3adc1652cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13f77fe700 1 -- 172.21.15.6:0/1535775550 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000e070 con 0x7f13f80cc310 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13f77fe700 1 -- 172.21.15.6:0/1535775550 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f000b9c0 con 0x7f13f80cc310 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13f77fe700 1 -- 172.21.15.6:0/1535775550 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000bd70 con 0x7f13f80cc310 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13ff160700 1 -- 172.21.15.6:0/1535775550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f80cc310 msgr2=0x7f13f80cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13ff160700 1 --2- 172.21.15.6:0/1535775550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f80cc310 0x7f13f80cc770 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f13f806ea90 tx=0x7f13f000b330 comp rx=0 tx=0).stop 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13ff160700 1 -- 172.21.15.6:0/1535775550 shutdown_connections 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13ff160700 1 --2- 172.21.15.6:0/1535775550 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f80cc310 0x7f13f80cc770 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13ff160700 1 --2- 172.21.15.6:0/1535775550 >> v2:172.21.15.61:3300/0 conn(0x7f13f815c990 0x7f13f80cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13ff160700 1 -- 172.21.15.6:0/1535775550 >> 172.21.15.6:0/1535775550 conn(0x7f13f80b1650 msgr2=0x7f13f80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13ff160700 1 -- 172.21.15.6:0/1535775550 shutdown_connections 2023-11-28T13:03:04.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.067+0000 7f13ff160700 1 -- 172.21.15.6:0/1535775550 wait complete. 2023-11-28T13:03:04.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.068+0000 7f13ff160700 1 Processor -- start 2023-11-28T13:03:04.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.068+0000 7f13ff160700 1 -- start start 2023-11-28T13:03:04.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.069+0000 7f13ff160700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f13f80cc310 0x7f13f80ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.069+0000 7f13ff160700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f13f815c990 0x7f13f80c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.069+0000 7f13ff160700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f8161090 0x7f13f8159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.069+0000 7f13ff160700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8159ea0 con 0x7f13f8161090 2023-11-28T13:03:04.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.069+0000 7f13ff160700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80c8520 con 0x7f13f80cc310 2023-11-28T13:03:04.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.069+0000 7f13ff160700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f815dc70 con 0x7f13f815c990 2023-11-28T13:03:04.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.069+0000 7f13fd6fd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f8161090 0x7f13f8159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.069+0000 7f13fd6fd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f8161090 0x7f13f8159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33864/0 (socket says 172.21.15.6:33864) 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.069+0000 7f13fd6fd700 1 -- 172.21.15.6:0/3743863796 learned_addr learned my addr 172.21.15.6:0/3743863796 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13fcefc700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f13f80cc310 0x7f13f80ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13f7fff700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f13f815c990 0x7f13f80c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13fd6fd700 1 -- 172.21.15.6:0/3743863796 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f13f815c990 msgr2=0x7f13f80c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13fd6fd700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f13f815c990 0x7f13f80c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13fd6fd700 1 -- 172.21.15.6:0/3743863796 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f13f80cc310 msgr2=0x7f13f80ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13fd6fd700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f13f80cc310 0x7f13f80ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13fd6fd700 1 -- 172.21.15.6:0/3743863796 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0009b30 con 0x7f13f8161090 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13fcefc700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f13f80cc310 0x7f13f80ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13f7fff700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f13f815c990 0x7f13f80c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:04.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13fd6fd700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f8161090 0x7f13f8159960 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f13f000b730 tx=0x7f13f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13f57fa700 1 -- 172.21.15.6:0/3743863796 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00099b0 con 0x7f13f8161090 2023-11-28T13:03:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13f57fa700 1 -- 172.21.15.6:0/3743863796 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f00092e0 con 0x7f13f8161090 2023-11-28T13:03:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13ff160700 1 -- 172.21.15.6:0/3743863796 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f80ce870 con 0x7f13f8161090 2023-11-28T13:03:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13f57fa700 1 -- 172.21.15.6:0/3743863796 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f001b630 con 0x7f13f8161090 2023-11-28T13:03:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.070+0000 7f13ff160700 1 -- 172.21.15.6:0/3743863796 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f80ce590 con 0x7f13f8161090 2023-11-28T13:03:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.071+0000 7f13ff160700 1 -- 172.21.15.6:0/3743863796 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f80cb080 con 0x7f13f8161090 2023-11-28T13:03:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.071+0000 7f13f57fa700 1 -- 172.21.15.6:0/3743863796 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f13f002b070 con 0x7f13f8161090 2023-11-28T13:03:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.072+0000 7f13f57fa700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f13dc0682f0 0x7f13dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.072+0000 7f13fcefc700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f13dc0682f0 0x7f13dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:04.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.072+0000 7f13f57fa700 1 -- 172.21.15.6:0/3743863796 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f13f00875e0 con 0x7f13f8161090 2023-11-28T13:03:04.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.072+0000 7f13fcefc700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f13dc0682f0 0x7f13dc06a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0099f0 tx=0x7f13ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:04.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.076+0000 7f13f57fa700 1 -- 172.21.15.6:0/3743863796 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f13f0053d00 con 0x7f13f8161090 2023-11-28T13:03:04.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:04.315+0000 7f13ff160700 1 -- 172.21.15.6:0/3743863796 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f13f80675a0 con 0x7f13f8161090 2023-11-28T13:03:05.053 INFO:tasks.ceph.mon.a.smithi006.stderr:2023-11-28T13:03:05.052+0000 7fa6275a1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:03:05.060 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.059+0000 7f13f57fa700 1 -- 172.21.15.6:0/3743863796 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f13f0057320 con 0x7f13f8161090 2023-11-28T13:03:05.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 -- 172.21.15.6:0/3743863796 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f13dc0682f0 msgr2=0x7f13dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f13dc0682f0 0x7f13dc06a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0099f0 tx=0x7f13ec008040 comp rx=0 tx=0).stop 2023-11-28T13:03:05.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 -- 172.21.15.6:0/3743863796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f8161090 msgr2=0x7f13f8159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f8161090 0x7f13f8159960 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f13f000b730 tx=0x7f13f0004ad0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 -- 172.21.15.6:0/3743863796 shutdown_connections 2023-11-28T13:03:05.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f13f8161090 0x7f13f8159960 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f13f80cc310 0x7f13f80ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f13dc0682f0 0x7f13dc06a7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 --2- 172.21.15.6:0/3743863796 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f13f815c990 0x7f13f80c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 -- 172.21.15.6:0/3743863796 >> 172.21.15.6:0/3743863796 conn(0x7f13f80b1650 msgr2=0x7f13f80af060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:05.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 -- 172.21.15.6:0/3743863796 shutdown_connections 2023-11-28T13:03:05.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.062+0000 7f13ff160700 1 -- 172.21.15.6:0/3743863796 wait complete. 2023-11-28T13:03:05.065 INFO:tasks.ceph.mds.a.smithi006.stderr: -3890> 2023-11-28T13:02:37.129+0000 7fae1385f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:03:05.065 INFO:teuthology.orchestra.run.smithi006.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:03:05.066 INFO:tasks.ceph.mds.a.smithi006.stderr: -2380> 2023-11-28T13:02:41.130+0000 7fae10058700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6180=0-4194304@1} caps={6180=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6180 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x557f306a5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T13:03:05.079 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:03:05.096 INFO:tasks.ceph.mds.a.smithi006.stdout:starting mds.a at 2023-11-28T13:03:05.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.214+0000 7f4ba201f700 1 Processor -- start 2023-11-28T13:03:05.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.214+0000 7f4ba201f700 1 -- start start 2023-11-28T13:03:05.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4ba201f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4b9c0cf5f0 0x7f4b9c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4ba201f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 0x7f4b9c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4ba201f700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9c0ce550 con 0x7f4b9c0b1630 2023-11-28T13:03:05.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4ba201f700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b9c0ce830 con 0x7f4b9c0cf5f0 2023-11-28T13:03:05.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4ba201f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c0ceb10 con 0x7f4b9c0cb3e0 2023-11-28T13:03:05.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4b9affd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4b9c0cf5f0 0x7f4b9c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4b9b7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f4b9c0b1630 0x7f4b9c0b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49432/0 (socket says 172.21.15.6:49432) 2023-11-28T13:03:05.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4b9b7fe700 1 -- 172.21.15.6:0/2939485289 learned_addr learned my addr 172.21.15.6:0/2939485289 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:05.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4b9affd700 1 -- 172.21.15.6:0/2939485289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 msgr2=0x7f4b9c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:05.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4b9bfff700 1 --2- 172.21.15.6:0/2939485289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 0x7f4b9c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4b9affd700 1 --2- 172.21.15.6:0/2939485289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 0x7f4b9c06afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4b9affd700 1 -- 172.21.15.6:0/2939485289 >> v1:172.21.15.6:6789/0 conn(0x7f4b9c0b1630 legacy=0x7f4b9c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4b9affd700 1 -- 172.21.15.6:0/2939485289 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9c0cedf0 con 0x7f4b9c0cf5f0 2023-11-28T13:03:05.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.215+0000 7f4b9a7fc700 1 -- 172.21.15.6:0/2939485289 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150728641 0 0) 0x7f4b9c0ce550 con 0x7f4b9c0b1630 2023-11-28T13:03:05.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4b9affd700 1 --2- 172.21.15.6:0/2939485289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0cf5f0 0x7f4b9c0caea0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009870 tx=0x7f4b84009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a213e4010792824 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4b9a7fc700 1 -- 172.21.15.6:0/2939485289 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b84011830 con 0x7f4b9c0cf5f0 2023-11-28T13:03:05.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4b9a7fc700 1 -- 172.21.15.6:0/2939485289 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b84011990 con 0x7f4b9c0cf5f0 2023-11-28T13:03:05.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4b9a7fc700 1 -- 172.21.15.6:0/2939485289 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b84011d20 con 0x7f4b9c0cf5f0 2023-11-28T13:03:05.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4ba201f700 1 -- 172.21.15.6:0/2939485289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0cf5f0 msgr2=0x7f4b9c0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4ba201f700 1 --2- 172.21.15.6:0/2939485289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0cf5f0 0x7f4b9c0caea0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009870 tx=0x7f4b84009b80 comp rx=0 tx=0).stop 2023-11-28T13:03:05.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4ba201f700 1 -- 172.21.15.6:0/2939485289 shutdown_connections 2023-11-28T13:03:05.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4ba201f700 1 --2- 172.21.15.6:0/2939485289 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 0x7f4b9c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4ba201f700 1 --2- 172.21.15.6:0/2939485289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0cf5f0 0x7f4b9c0caea0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4ba201f700 1 -- 172.21.15.6:0/2939485289 >> 172.21.15.6:0/2939485289 conn(0x7f4b9c0cc630 msgr2=0x7f4b9c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:05.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4ba201f700 1 -- 172.21.15.6:0/2939485289 shutdown_connections 2023-11-28T13:03:05.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.216+0000 7f4ba201f700 1 -- 172.21.15.6:0/2939485289 wait complete. 2023-11-28T13:03:05.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.218+0000 7f4ba201f700 1 Processor -- start 2023-11-28T13:03:05.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.218+0000 7f4ba201f700 1 -- start start 2023-11-28T13:03:05.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.218+0000 7f4ba201f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0b1630 0x7f4b9c15a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.218+0000 7f4ba201f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 0x7f4b9c0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.218+0000 7f4ba201f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b9c0cf5f0 0x7f4b9c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.218+0000 7f4ba201f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c15b460 con 0x7f4b9c0b1630 2023-11-28T13:03:05.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.218+0000 7f4ba201f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c15a850 con 0x7f4b9c0cf5f0 2023-11-28T13:03:05.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.218+0000 7f4ba201f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c15ab30 con 0x7f4b9c0cb3e0 2023-11-28T13:03:05.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0b1630 0x7f4b9c15a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9b7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0b1630 0x7f4b9c15a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33886/0 (socket says 172.21.15.6:33886) 2023-11-28T13:03:05.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9b7fe700 1 -- 172.21.15.6:0/3325003757 learned_addr learned my addr 172.21.15.6:0/3325003757 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:05.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9affd700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 0x7f4b9c0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9bfff700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b9c0cf5f0 0x7f4b9c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9b7fe700 1 -- 172.21.15.6:0/3325003757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 msgr2=0x7f4b9c0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9b7fe700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 0x7f4b9c0b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9b7fe700 1 -- 172.21.15.6:0/3325003757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b9c0cf5f0 msgr2=0x7f4b9c0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:05.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9b7fe700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b9c0cf5f0 0x7f4b9c0b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:05.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9b7fe700 1 -- 172.21.15.6:0/3325003757 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840094a0 con 0x7f4b9c0b1630 2023-11-28T13:03:05.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9affd700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 0x7f4b9c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:05.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b9b7fe700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0b1630 0x7f4b9c15a000 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0011e0 tx=0x7f4b8c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b93fff700 1 -- 172.21.15.6:0/3325003757 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c004c80 con 0x7f4b9c0b1630 2023-11-28T13:03:05.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b93fff700 1 -- 172.21.15.6:0/3325003757 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8c004e00 con 0x7f4b9c0b1630 2023-11-28T13:03:05.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4ba201f700 1 -- 172.21.15.6:0/3325003757 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9c0ce140 con 0x7f4b9c0b1630 2023-11-28T13:03:05.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4b93fff700 1 -- 172.21.15.6:0/3325003757 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c007310 con 0x7f4b9c0b1630 2023-11-28T13:03:05.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.219+0000 7f4ba201f700 1 -- 172.21.15.6:0/3325003757 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c0cf0d0 con 0x7f4b9c0b1630 2023-11-28T13:03:05.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.220+0000 7f4ba201f700 1 -- 172.21.15.6:0/3325003757 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9c0ce2d0 con 0x7f4b9c0b1630 2023-11-28T13:03:05.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.221+0000 7f4b93fff700 1 -- 172.21.15.6:0/3325003757 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4b8c01a020 con 0x7f4b9c0b1630 2023-11-28T13:03:05.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.221+0000 7f4b93fff700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4b7c0682f0 0x7f4b7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:05.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.221+0000 7f4b93fff700 1 -- 172.21.15.6:0/3325003757 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f4b8c085a80 con 0x7f4b9c0b1630 2023-11-28T13:03:05.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.221+0000 7f4b9affd700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4b7c0682f0 0x7f4b7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:05.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.222+0000 7f4b9affd700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4b7c0682f0 0x7f4b7c06a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000c00 tx=0x7f4b84008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:05.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.225+0000 7f4b93fff700 1 -- 172.21.15.6:0/3325003757 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4b8c052140 con 0x7f4b9c0b1630 2023-11-28T13:03:05.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:05.466+0000 7f4ba201f700 1 -- 172.21.15.6:0/3325003757 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4b9c0739f0 con 0x7f4b9c0b1630 2023-11-28T13:03:06.061 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.060+0000 7f4b93fff700 1 -- 172.21.15.6:0/3325003757 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4b8c055760 con 0x7f4b9c0b1630 2023-11-28T13:03:06.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 -- 172.21.15.6:0/3325003757 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4b7c0682f0 msgr2=0x7f4b7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:06.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4b7c0682f0 0x7f4b7c06a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000c00 tx=0x7f4b84008030 comp rx=0 tx=0).stop 2023-11-28T13:03:06.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 -- 172.21.15.6:0/3325003757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0b1630 msgr2=0x7f4b9c15a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:06.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0b1630 0x7f4b9c15a000 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0011e0 tx=0x7f4b8c00cae0 comp rx=0 tx=0).stop 2023-11-28T13:03:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 -- 172.21.15.6:0/3325003757 shutdown_connections 2023-11-28T13:03:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b9c0cf5f0 0x7f4b9c0b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4b9c0b1630 0x7f4b9c15a000 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4b7c0682f0 0x7f4b7c06a7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 --2- 172.21.15.6:0/3325003757 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4b9c0cb3e0 0x7f4b9c0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 -- 172.21.15.6:0/3325003757 >> 172.21.15.6:0/3325003757 conn(0x7f4b9c0cc630 msgr2=0x7f4b9c0b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 -- 172.21.15.6:0/3325003757 shutdown_connections 2023-11-28T13:03:06.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.063+0000 7f4ba201f700 1 -- 172.21.15.6:0/3325003757 wait complete. 2023-11-28T13:03:06.080 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:03:06.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.256+0000 7f44719e0700 1 Processor -- start 2023-11-28T13:03:06.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.256+0000 7f44719e0700 1 -- start start 2023-11-28T13:03:06.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f44719e0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f446c06c640 0x7f446c160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:06.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f44719e0700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f446c15d950 0x7f446c1607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f44719e0700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446c160cf0 con 0x7f446c0cd9f0 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f44719e0700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446c153bc0 con 0x7f446c15d950 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f44719e0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446c15cc80 con 0x7f446c06c640 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f446affd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f446c0cd9f0 0x7f446c06bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49448/0 (socket says 172.21.15.6:49448) 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f446affd700 1 -- 172.21.15.6:0/1453411227 learned_addr learned my addr 172.21.15.6:0/1453411227 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f446b7fe700 1 --2- 172.21.15.6:0/1453411227 >> v2:172.21.15.6:3300/0 conn(0x7f446c15d950 0x7f446c1607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f446a7fc700 1 --2- 172.21.15.6:0/1453411227 >> v2:172.21.15.61:3300/0 conn(0x7f446c06c640 0x7f446c160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f446b7fe700 1 -- 172.21.15.6:0/1453411227 >> v2:172.21.15.61:3300/0 conn(0x7f446c06c640 msgr2=0x7f446c160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f446b7fe700 1 --2- 172.21.15.6:0/1453411227 >> v2:172.21.15.61:3300/0 conn(0x7f446c06c640 0x7f446c160270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f446b7fe700 1 -- 172.21.15.6:0/1453411227 >> v1:172.21.15.6:6789/0 conn(0x7f446c0cd9f0 legacy=0x7f446c06bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f4469ffb700 1 -- 172.21.15.6:0/1453411227 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864284076 0 0) 0x7f445c001110 con 0x7f446c0cd9f0 2023-11-28T13:03:06.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f446b7fe700 1 -- 172.21.15.6:0/1453411227 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0cddc0 con 0x7f446c15d950 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f446b7fe700 1 --2- 172.21.15.6:0/1453411227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c15d950 0x7f446c1607b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f446c06ca10 tx=0x7f446400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcfc7beb1494a1a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.257+0000 7f4469ffb700 1 -- 172.21.15.6:0/1453411227 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446400e070 con 0x7f446c15d950 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.258+0000 7f4469ffb700 1 -- 172.21.15.6:0/1453411227 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446400bb20 con 0x7f446c15d950 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.258+0000 7f4469ffb700 1 -- 172.21.15.6:0/1453411227 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464012430 con 0x7f446c15d950 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.258+0000 7f44719e0700 1 -- 172.21.15.6:0/1453411227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c15d950 msgr2=0x7f446c1607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.258+0000 7f44719e0700 1 --2- 172.21.15.6:0/1453411227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c15d950 0x7f446c1607b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f446c06ca10 tx=0x7f446400b330 comp rx=0 tx=0).stop 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.258+0000 7f44719e0700 1 -- 172.21.15.6:0/1453411227 shutdown_connections 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.258+0000 7f44719e0700 1 --2- 172.21.15.6:0/1453411227 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c15d950 0x7f446c1607b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.258+0000 7f44719e0700 1 --2- 172.21.15.6:0/1453411227 >> v2:172.21.15.61:3300/0 conn(0x7f446c06c640 0x7f446c160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.258+0000 7f44719e0700 1 -- 172.21.15.6:0/1453411227 >> 172.21.15.6:0/1453411227 conn(0x7f446c0d2e70 msgr2=0x7f446c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.258+0000 7f44719e0700 1 -- 172.21.15.6:0/1453411227 shutdown_connections 2023-11-28T13:03:06.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.258+0000 7f44719e0700 1 -- 172.21.15.6:0/1453411227 wait complete. 2023-11-28T13:03:06.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.259+0000 7f44719e0700 1 Processor -- start 2023-11-28T13:03:06.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.260+0000 7f44719e0700 1 -- start start 2023-11-28T13:03:06.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.260+0000 7f44719e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c06c640 0x7f446c0c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:06.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.260+0000 7f44719e0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f446c0cd9f0 0x7f446c0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:06.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.260+0000 7f44719e0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f446c15d950 0x7f446c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:06.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.260+0000 7f44719e0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0c71a0 con 0x7f446c06c640 2023-11-28T13:03:06.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.260+0000 7f44719e0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0cb100 con 0x7f446c0cd9f0 2023-11-28T13:03:06.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.260+0000 7f44719e0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446c0cb3e0 con 0x7f446c15d950 2023-11-28T13:03:06.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.260+0000 7f446affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c06c640 0x7f446c0c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.260+0000 7f446affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c06c640 0x7f446c0c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33902/0 (socket says 172.21.15.6:33902) 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.260+0000 7f446affd700 1 -- 172.21.15.6:0/1987671444 learned_addr learned my addr 172.21.15.6:0/1987671444 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f446a7fc700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f446c0cd9f0 0x7f446c0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f446b7fe700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f446c15d950 0x7f446c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f446affd700 1 -- 172.21.15.6:0/1987671444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f446c15d950 msgr2=0x7f446c0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f446affd700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f446c15d950 0x7f446c0cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f446affd700 1 -- 172.21.15.6:0/1987671444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f446c0cd9f0 msgr2=0x7f446c0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f446affd700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f446c0cd9f0 0x7f446c0ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f446affd700 1 -- 172.21.15.6:0/1987671444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464009a70 con 0x7f446c06c640 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f446a7fc700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f446c0cd9f0 0x7f446c0ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f446affd700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c06c640 0x7f446c0c6970 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c730 tx=0x7f445c00caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f4453fff700 1 -- 172.21.15.6:0/1987671444 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c004c80 con 0x7f446c06c640 2023-11-28T13:03:06.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f4453fff700 1 -- 172.21.15.6:0/1987671444 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c004e00 con 0x7f446c06c640 2023-11-28T13:03:06.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f44719e0700 1 -- 172.21.15.6:0/1987671444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446c15ce70 con 0x7f446c06c640 2023-11-28T13:03:06.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f4453fff700 1 -- 172.21.15.6:0/1987671444 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c007310 con 0x7f446c06c640 2023-11-28T13:03:06.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.261+0000 7f44719e0700 1 -- 172.21.15.6:0/1987671444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446c160f30 con 0x7f446c06c640 2023-11-28T13:03:06.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.262+0000 7f44719e0700 1 -- 172.21.15.6:0/1987671444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446c15d150 con 0x7f446c06c640 2023-11-28T13:03:06.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.263+0000 7f4453fff700 1 -- 172.21.15.6:0/1987671444 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f445c01a020 con 0x7f446c06c640 2023-11-28T13:03:06.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.263+0000 7f4453fff700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44580682f0 0x7f445806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:06.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.263+0000 7f4453fff700 1 -- 172.21.15.6:0/1987671444 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f445c085930 con 0x7f446c06c640 2023-11-28T13:03:06.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.263+0000 7f446a7fc700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44580682f0 0x7f445806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:06.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.263+0000 7f446a7fc700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44580682f0 0x7f445806a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f446c1f55f0 tx=0x7f4460008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:06.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.267+0000 7f4453fff700 1 -- 172.21.15.6:0/1987671444 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f445c051ff0 con 0x7f446c06c640 2023-11-28T13:03:06.460 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:06.459+0000 7f44719e0700 1 -- 172.21.15.6:0/1987671444 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f446c160cf0 con 0x7f446c06c640 2023-11-28T13:03:07.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.070+0000 7f4453fff700 1 -- 172.21.15.6:0/1987671444 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7f446c160cf0 con 0x7f446c06c640 2023-11-28T13:03:07.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.073+0000 7f44719e0700 1 -- 172.21.15.6:0/1987671444 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44580682f0 msgr2=0x7f445806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.073+0000 7f44719e0700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44580682f0 0x7f445806a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f446c1f55f0 tx=0x7f4460008040 comp rx=0 tx=0).stop 2023-11-28T13:03:07.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.073+0000 7f44719e0700 1 -- 172.21.15.6:0/1987671444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c06c640 msgr2=0x7f446c0c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.073+0000 7f44719e0700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c06c640 0x7f446c0c6970 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f445c00c730 tx=0x7f445c00caf0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.074+0000 7f44719e0700 1 -- 172.21.15.6:0/1987671444 shutdown_connections 2023-11-28T13:03:07.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.074+0000 7f44719e0700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44580682f0 0x7f445806a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.074+0000 7f44719e0700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f446c15d950 0x7f446c0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.074+0000 7f44719e0700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f446c06c640 0x7f446c0c6970 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.074+0000 7f44719e0700 1 --2- 172.21.15.6:0/1987671444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f446c0cd9f0 0x7f446c0ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.074+0000 7f44719e0700 1 -- 172.21.15.6:0/1987671444 >> 172.21.15.6:0/1987671444 conn(0x7f446c0d2e70 msgr2=0x7f446c0d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.074+0000 7f44719e0700 1 -- 172.21.15.6:0/1987671444 shutdown_connections 2023-11-28T13:03:07.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.074+0000 7f44719e0700 1 -- 172.21.15.6:0/1987671444 wait complete. 2023-11-28T13:03:07.076 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:03:07.091 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:03:07.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.225+0000 7f00e91dc700 1 Processor -- start 2023-11-28T13:03:07.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.225+0000 7f00e91dc700 1 -- start start 2023-11-28T13:03:07.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e91dc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f00e4161090 0x7f00e40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e91dc700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e40ce340 con 0x7f00e415c990 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e91dc700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e40cebe0 con 0x7f00e40caeb0 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e91dc700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e40cf780 con 0x7f00e4161090 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e2d9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f00e415c990 0x7f00e40cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49462/0 (socket says 172.21.15.6:49462) 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e2d9d700 1 -- 172.21.15.6:0/4051453701 learned_addr learned my addr 172.21.15.6:0/4051453701 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e259c700 1 --2- 172.21.15.6:0/4051453701 >> v2:172.21.15.61:3300/0 conn(0x7f00e4161090 0x7f00e40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16761608 0 0) 0x7f00e40ce340 con 0x7f00e415c990 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00cc0035b0 con 0x7f00e415c990 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3016929147 0 0) 0x7f00cc0035b0 con 0x7f00e415c990 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.226+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 >> v2:172.21.15.61:3300/0 conn(0x7f00e4161090 msgr2=0x7f00e40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 --2- 172.21.15.6:0/4051453701 >> v2:172.21.15.61:3300/0 conn(0x7f00e4161090 0x7f00e40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 >> v1:172.21.15.61:6789/0 conn(0x7f00e40caeb0 legacy=0x7f00e40cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e40cb3f0 con 0x7f00e415c990 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f00d8001670 con 0x7f00e415c990 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e259c700 1 --2- 172.21.15.6:0/4051453701 >> v2:172.21.15.61:3300/0 conn(0x7f00e4161090 0x7f00e40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 >> v1:172.21.15.6:6789/0 conn(0x7f00e415c990 legacy=0x7f00e40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00cc003a00 0x7f00cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00cc004850 0x7f00cc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f00cc005320 0x7f00cc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e40cf780 con 0x7f00cc003a00 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc00c250 con 0x7f00cc005320 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc00c410 con 0x7f00cc004850 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e2d9d700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00cc003a00 0x7f00cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e259c700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00cc004850 0x7f00cc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e359e700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f00cc005320 0x7f00cc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e2d9d700 1 -- 172.21.15.6:0/4051453701 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00cc004850 msgr2=0x7f00cc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e2d9d700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00cc004850 0x7f00cc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e2d9d700 1 -- 172.21.15.6:0/4051453701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f00cc005320 msgr2=0x7f00cc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e2d9d700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f00cc005320 0x7f00cc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e2d9d700 1 -- 172.21.15.6:0/4051453701 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc0035b0 con 0x7f00cc003a00 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e359e700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f00cc005320 0x7f00cc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e359e700 1 -- 172.21.15.6:0/4051453701 reap_dead start 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e2d9d700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00cc003a00 0x7f00cc003e10 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f00d800c240 tx=0x7f00cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1969a907ea0a5008 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8007c10 con 0x7f00cc003a00 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d800ce00 con 0x7f00cc003a00 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.227+0000 7f00e1d9b700 1 -- 172.21.15.6:0/4051453701 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8005ba0 con 0x7f00cc003a00 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.228+0000 7f00e91dc700 1 -- 172.21.15.6:0/4051453701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00cc003a00 msgr2=0x7f00cc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.228+0000 7f00e91dc700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00cc003a00 0x7f00cc003e10 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f00d800c240 tx=0x7f00cc005880 comp rx=0 tx=0).stop 2023-11-28T13:03:07.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.228+0000 7f00e91dc700 1 -- 172.21.15.6:0/4051453701 shutdown_connections 2023-11-28T13:03:07.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.228+0000 7f00e91dc700 1 --2- 172.21.15.6:0/4051453701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00cc003a00 0x7f00cc003e10 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.228+0000 7f00e91dc700 1 -- 172.21.15.6:0/4051453701 >> 172.21.15.6:0/4051453701 conn(0x7f00e40b1650 msgr2=0x7f00e40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:07.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.228+0000 7f00e91dc700 1 -- 172.21.15.6:0/4051453701 shutdown_connections 2023-11-28T13:03:07.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.228+0000 7f00e91dc700 1 -- 172.21.15.6:0/4051453701 wait complete. 2023-11-28T13:03:07.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.230+0000 7f00e91dc700 1 Processor -- start 2023-11-28T13:03:07.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.230+0000 7f00e91dc700 1 -- start start 2023-11-28T13:03:07.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e91dc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00cc003a00 0x7f00e415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e91dc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00e40c4400 0x7f00e40c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e91dc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f00e406b8c0 0x7f00e406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e91dc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e40c4db0 con 0x7f00e40c4400 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e91dc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e406c250 con 0x7f00e406b8c0 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e91dc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e40c63b0 con 0x7f00cc003a00 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00e40c4400 0x7f00e40c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00e40c4400 0x7f00e40c4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33924/0 (socket says 172.21.15.6:33924) 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e259c700 1 -- 172.21.15.6:0/2319161063 learned_addr learned my addr 172.21.15.6:0/2319161063 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e2d9d700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00cc003a00 0x7f00e415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e359e700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f00e406b8c0 0x7f00e406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e259c700 1 -- 172.21.15.6:0/2319161063 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00cc003a00 msgr2=0x7f00e415b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e259c700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00cc003a00 0x7f00e415b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e259c700 1 -- 172.21.15.6:0/2319161063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f00e406b8c0 msgr2=0x7f00e406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e259c700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f00e406b8c0 0x7f00e406bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e259c700 1 -- 172.21.15.6:0/2319161063 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e40ce870 con 0x7f00e40c4400 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.231+0000 7f00e259c700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00e40c4400 0x7f00e40c4870 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f00d4011930 tx=0x7f00d4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.232+0000 7f00cb7fe700 1 -- 172.21.15.6:0/2319161063 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d400f870 con 0x7f00e40c4400 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.232+0000 7f00cb7fe700 1 -- 172.21.15.6:0/2319161063 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d400f9d0 con 0x7f00e40c4400 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.232+0000 7f00e91dc700 1 -- 172.21.15.6:0/2319161063 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e40ce590 con 0x7f00e40c4400 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.232+0000 7f00cb7fe700 1 -- 172.21.15.6:0/2319161063 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d400fd40 con 0x7f00e40c4400 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.232+0000 7f00e91dc700 1 -- 172.21.15.6:0/2319161063 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e40ca970 con 0x7f00e40c4400 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.233+0000 7f00e91dc700 1 -- 172.21.15.6:0/2319161063 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e40b4520 con 0x7f00e40c4400 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.233+0000 7f00cb7fe700 1 -- 172.21.15.6:0/2319161063 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f00d4012040 con 0x7f00e40c4400 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.233+0000 7f00cb7fe700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00d00682f0 0x7f00d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:07.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.234+0000 7f00cb7fe700 1 -- 172.21.15.6:0/2319161063 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6030+0+0 (secure 0 0 0) 0x7f00d4085e00 con 0x7f00e40c4400 2023-11-28T13:03:07.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.234+0000 7f00e2d9d700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00d00682f0 0x7f00d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:07.238 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.234+0000 7f00e2d9d700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00d00682f0 0x7f00d006a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f00d8000c00 tx=0x7f00d8007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:07.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.238+0000 7f00cb7fe700 1 -- 172.21.15.6:0/2319161063 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f00d4052680 con 0x7f00e40c4400 2023-11-28T13:03:07.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:07.433+0000 7f00e91dc700 1 -- 172.21.15.6:0/2319161063 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f00e4066700 con 0x7f00e40c4400 2023-11-28T13:03:08.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.075+0000 7f00cb7fe700 1 -- 172.21.15.6:0/2319161063 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7f00d40220d0 con 0x7f00e40c4400 2023-11-28T13:03:08.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 -- 172.21.15.6:0/2319161063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00d00682f0 msgr2=0x7f00d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00d00682f0 0x7f00d006a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f00d8000c00 tx=0x7f00d8007000 comp rx=0 tx=0).stop 2023-11-28T13:03:08.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 -- 172.21.15.6:0/2319161063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00e40c4400 msgr2=0x7f00e40c4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00e40c4400 0x7f00e40c4870 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f00d4011930 tx=0x7f00d4011c40 comp rx=0 tx=0).stop 2023-11-28T13:03:08.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 -- 172.21.15.6:0/2319161063 shutdown_connections 2023-11-28T13:03:08.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f00e406b8c0 0x7f00e406bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f00d00682f0 0x7f00d006a7a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f00cc003a00 0x7f00e415b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 --2- 172.21.15.6:0/2319161063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f00e40c4400 0x7f00e40c4870 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 -- 172.21.15.6:0/2319161063 >> 172.21.15.6:0/2319161063 conn(0x7f00e40b1650 msgr2=0x7f00e406b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.080+0000 7f00e91dc700 1 -- 172.21.15.6:0/2319161063 shutdown_connections 2023-11-28T13:03:08.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.081+0000 7f00e91dc700 1 -- 172.21.15.6:0/2319161063 wait complete. 2023-11-28T13:03:08.084 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' removed 2023-11-28T13:03:08.100 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T13:03:08.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.233+0000 7fc8850f1700 1 Processor -- start 2023-11-28T13:03:08.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.233+0000 7fc8850f1700 1 -- start start 2023-11-28T13:03:08.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc8850f1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fc8800b1630 0x7fc8800cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc8850f1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88006afc0 0x7fc88006b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc8850f1700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8800ce550 con 0x7fc8800b1a00 2023-11-28T13:03:08.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc8850f1700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8800ce830 con 0x7fc8800b1630 2023-11-28T13:03:08.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc8850f1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800ceb10 con 0x7fc88006afc0 2023-11-28T13:03:08.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87e59c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fc8800b1a00 0x7fc8800caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49482/0 (socket says 172.21.15.6:49482) 2023-11-28T13:03:08.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87e59c700 1 -- 172.21.15.6:0/2736184908 learned_addr learned my addr 172.21.15.6:0/2736184908 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:08.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87ed9d700 1 --2- 172.21.15.6:0/2736184908 >> v2:172.21.15.61:3300/0 conn(0x7fc8800b1630 0x7fc8800cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87f59e700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88006afc0 0x7fc88006b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708267554 0 0) 0x7fc8800ce550 con 0x7fc8800b1a00 2023-11-28T13:03:08.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8680035b0 con 0x7fc8800b1a00 2023-11-28T13:03:08.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 875977689 0 0) 0x7fc8680035b0 con 0x7fc8800b1a00 2023-11-28T13:03:08.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88006afc0 msgr2=0x7fc88006b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:08.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88006afc0 0x7fc88006b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 >> v2:172.21.15.61:3300/0 conn(0x7fc8800b1630 msgr2=0x7fc8800cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:08.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 --2- 172.21.15.6:0/2736184908 >> v2:172.21.15.61:3300/0 conn(0x7fc8800b1630 0x7fc8800cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8800cedf0 con 0x7fc8800b1a00 2023-11-28T13:03:08.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fc874001670 con 0x7fc8800b1a00 2023-11-28T13:03:08.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 >> v1:172.21.15.6:6789/0 conn(0x7fc8800b1a00 legacy=0x7fc8800caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc868003a00 0x7fc868003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc868004430 0x7fc8680048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.234+0000 7fc87dd9b700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88006afc0 0x7fc868004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800ce830 con 0x7fc868003a00 2023-11-28T13:03:08.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800ceb10 con 0x7fc868004430 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86800ba10 con 0x7fc88006afc0 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87ed9d700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc868003a00 0x7fc868003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87e59c700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc868004430 0x7fc8680048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87f59e700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88006afc0 0x7fc868004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87ed9d700 1 -- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88006afc0 msgr2=0x7fc868004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87ed9d700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc88006afc0 0x7fc868004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87ed9d700 1 -- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc868004430 msgr2=0x7fc8680048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87ed9d700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc868004430 0x7fc8680048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87ed9d700 1 -- 172.21.15.6:0/2736184908 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680035b0 con 0x7fc868003a00 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87ed9d700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc868003a00 0x7fc868003e10 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc870009b60 tx=0x7fc868005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e513b5cfcdbf7997 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000e6c0 con 0x7fc868003a00 2023-11-28T13:03:08.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000e820 con 0x7fc868003a00 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.235+0000 7fc87dd9b700 1 -- 172.21.15.6:0/2736184908 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000eb90 con 0x7fc868003a00 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.236+0000 7fc8850f1700 1 -- 172.21.15.6:0/2736184908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc868003a00 msgr2=0x7fc868003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.236+0000 7fc8850f1700 1 --2- 172.21.15.6:0/2736184908 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc868003a00 0x7fc868003e10 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc870009b60 tx=0x7fc868005880 comp rx=0 tx=0).stop 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.236+0000 7fc87f59e700 1 -- 172.21.15.6:0/2736184908 reap_dead start 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.236+0000 7fc8850f1700 1 -- 172.21.15.6:0/2736184908 shutdown_connections 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.236+0000 7fc8850f1700 1 -- 172.21.15.6:0/2736184908 >> 172.21.15.6:0/2736184908 conn(0x7fc8800cc630 msgr2=0x7fc8800cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.236+0000 7fc8850f1700 1 -- 172.21.15.6:0/2736184908 shutdown_connections 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.236+0000 7fc8850f1700 1 -- 172.21.15.6:0/2736184908 wait complete. 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.237+0000 7fc8850f1700 1 Processor -- start 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.237+0000 7fc8850f1700 1 -- start start 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc8850f1700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc868003a00 0x7fc88015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc8850f1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc88015a5c0 0x7fc8800b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc8850f1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8800c8ea0 0x7fc8800c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc8850f1700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800c4730 con 0x7fc88015a5c0 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc8850f1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800b4360 con 0x7fc8800c8ea0 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc8850f1700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800b4640 con 0x7fc868003a00 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc87e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc88015a5c0 0x7fc8800b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc87e59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc88015a5c0 0x7fc8800b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33944/0 (socket says 172.21.15.6:33944) 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc87e59c700 1 -- 172.21.15.6:0/989765374 learned_addr learned my addr 172.21.15.6:0/989765374 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc87ed9d700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc868003a00 0x7fc88015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc87f59e700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8800c8ea0 0x7fc8800c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.238+0000 7fc87e59c700 1 -- 172.21.15.6:0/989765374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc868003a00 msgr2=0x7fc88015a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.239+0000 7fc87e59c700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc868003a00 0x7fc88015a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.239+0000 7fc87e59c700 1 -- 172.21.15.6:0/989765374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8800c8ea0 msgr2=0x7fc8800c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.239+0000 7fc87e59c700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8800c8ea0 0x7fc8800c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.239+0000 7fc87e59c700 1 -- 172.21.15.6:0/989765374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8800b1ea0 con 0x7fc88015a5c0 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.239+0000 7fc87e59c700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc88015a5c0 0x7fc8800b5130 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc874007290 tx=0x7fc8740075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:08.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.239+0000 7fc8677fe700 1 -- 172.21.15.6:0/989765374 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874011580 con 0x7fc88015a5c0 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.239+0000 7fc8677fe700 1 -- 172.21.15.6:0/989765374 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8740116e0 con 0x7fc88015a5c0 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.239+0000 7fc8850f1700 1 -- 172.21.15.6:0/989765374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8800cf0d0 con 0x7fc88015a5c0 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.239+0000 7fc8677fe700 1 -- 172.21.15.6:0/989765374 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874011a90 con 0x7fc88015a5c0 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.239+0000 7fc8850f1700 1 -- 172.21.15.6:0/989765374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8800cb5b0 con 0x7fc88015a5c0 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.240+0000 7fc8850f1700 1 -- 172.21.15.6:0/989765374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8800739f0 con 0x7fc88015a5c0 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.240+0000 7fc8677fe700 1 -- 172.21.15.6:0/989765374 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc874011bf0 con 0x7fc88015a5c0 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.241+0000 7fc8677fe700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 0x7fc86c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.241+0000 7fc87ed9d700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 0x7fc86c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.241+0000 7fc8677fe700 1 -- 172.21.15.6:0/989765374 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fc874085c80 con 0x7fc88015a5c0 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.241+0000 7fc87ed9d700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 0x7fc86c06a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc8700090b0 tx=0x7fc870008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:08.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.245+0000 7fc8677fe700 1 -- 172.21.15.6:0/989765374 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc8740526a0 con 0x7fc88015a5c0 2023-11-28T13:03:08.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.446+0000 7fc8850f1700 1 -- 172.21.15.6:0/989765374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc8800cac80 con 0x7fc88015a5c0 2023-11-28T13:03:08.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.454+0000 7fc8677fe700 1 -- 172.21.15.6:0/989765374 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc874055cc0 con 0x7fc88015a5c0 2023-11-28T13:03:08.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 -- 172.21.15.6:0/989765374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 msgr2=0x7fc86c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.461 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 0x7fc86c06a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc8700090b0 tx=0x7fc870008030 comp rx=0 tx=0).stop 2023-11-28T13:03:08.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 -- 172.21.15.6:0/989765374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc88015a5c0 msgr2=0x7fc8800b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc88015a5c0 0x7fc8800b5130 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc874007290 tx=0x7fc8740075a0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 -- 172.21.15.6:0/989765374 shutdown_connections 2023-11-28T13:03:08.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8800c8ea0 0x7fc8800c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc86c0682f0 0x7fc86c06a7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc868003a00 0x7fc88015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 --2- 172.21.15.6:0/989765374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc88015a5c0 0x7fc8800b5130 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 -- 172.21.15.6:0/989765374 >> 172.21.15.6:0/989765374 conn(0x7fc8800cc630 msgr2=0x7fc8800cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:08.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 -- 172.21.15.6:0/989765374 shutdown_connections 2023-11-28T13:03:08.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.457+0000 7fc8850f1700 1 -- 172.21.15.6:0/989765374 wait complete. 2023-11-28T13:03:08.471 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T13:03:08.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.595+0000 7f9662d43700 1 Processor -- start 2023-11-28T13:03:08.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.595+0000 7f9662d43700 1 -- start start 2023-11-28T13:03:08.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9662d43700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f965c15c990 0x7f965c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9662d43700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f965c161090 0x7f965c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9662d43700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c0ce340 con 0x7f965c0caeb0 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9662d43700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965c0cebe0 con 0x7f965c15c990 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9662d43700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0cb3f0 con 0x7f965c161090 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9660adf700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f965c15c990 0x7f965c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f96612e0700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f965c0caeb0 0x7f965c0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49504/0 (socket says 172.21.15.6:49504) 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f96612e0700 1 -- 172.21.15.6:0/2732413169 learned_addr learned my addr 172.21.15.6:0/2732413169 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f965bfff700 1 --2- 172.21.15.6:0/2732413169 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f965c161090 0x7f965c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9660adf700 1 -- 172.21.15.6:0/2732413169 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f965c161090 msgr2=0x7f965c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9660adf700 1 --2- 172.21.15.6:0/2732413169 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f965c161090 0x7f965c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9660adf700 1 -- 172.21.15.6:0/2732413169 >> v1:172.21.15.6:6789/0 conn(0x7f965c0caeb0 legacy=0x7f965c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9660adf700 1 -- 172.21.15.6:0/2732413169 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c0ce590 con 0x7f965c15c990 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f965b7fe700 1 -- 172.21.15.6:0/2732413169 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083127574 0 0) 0x7f965c0ce340 con 0x7f965c0caeb0 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.596+0000 7f9660adf700 1 --2- 172.21.15.6:0/2732413169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c15c990 0x7f965c0cf070 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9650009840 tx=0x7f9650009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0e8f6c439c78f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f965b7fe700 1 -- 172.21.15.6:0/2732413169 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650011830 con 0x7f965c15c990 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f965b7fe700 1 -- 172.21.15.6:0/2732413169 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9650011990 con 0x7f965c15c990 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f965b7fe700 1 -- 172.21.15.6:0/2732413169 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650011d20 con 0x7f965c15c990 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f9662d43700 1 -- 172.21.15.6:0/2732413169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c15c990 msgr2=0x7f965c0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f9662d43700 1 --2- 172.21.15.6:0/2732413169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c15c990 0x7f965c0cf070 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f9650009840 tx=0x7f9650009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f9662d43700 1 -- 172.21.15.6:0/2732413169 shutdown_connections 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f9662d43700 1 --2- 172.21.15.6:0/2732413169 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c15c990 0x7f965c0cf070 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f9662d43700 1 --2- 172.21.15.6:0/2732413169 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f965c161090 0x7f965c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f9662d43700 1 -- 172.21.15.6:0/2732413169 >> 172.21.15.6:0/2732413169 conn(0x7f965c0b1650 msgr2=0x7f965c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f9662d43700 1 -- 172.21.15.6:0/2732413169 shutdown_connections 2023-11-28T13:03:08.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.597+0000 7f9662d43700 1 -- 172.21.15.6:0/2732413169 wait complete. 2023-11-28T13:03:08.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.599+0000 7f9662d43700 1 Processor -- start 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.599+0000 7f9662d43700 1 -- start start 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9662d43700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c0caeb0 0x7f965c0b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9662d43700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f965c15c990 0x7f965c1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9662d43700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f965c161090 0x7f965c0c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9662d43700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c0b5960 con 0x7f965c0caeb0 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9662d43700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c15f810 con 0x7f965c161090 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9662d43700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c15faf0 con 0x7f965c15c990 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9660adf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c0caeb0 0x7f965c0b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9660adf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c0caeb0 0x7f965c0b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33954/0 (socket says 172.21.15.6:33954) 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9660adf700 1 -- 172.21.15.6:0/1380745067 learned_addr learned my addr 172.21.15.6:0/1380745067 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f965bfff700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f965c15c990 0x7f965c1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f96612e0700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f965c161090 0x7f965c0c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9660adf700 1 -- 172.21.15.6:0/1380745067 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f965c15c990 msgr2=0x7f965c1510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9660adf700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f965c15c990 0x7f965c1510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9660adf700 1 -- 172.21.15.6:0/1380745067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f965c161090 msgr2=0x7f965c0c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9660adf700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f965c161090 0x7f965c0c35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f9660adf700 1 -- 172.21.15.6:0/1380745067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96500094a0 con 0x7f965c0caeb0 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.600+0000 7f96612e0700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f965c161090 0x7f965c0c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.601+0000 7f9660adf700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c0caeb0 0x7f965c0b5420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9650000c00 tx=0x7f9650004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.601+0000 7f96597fa700 1 -- 172.21.15.6:0/1380745067 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96500036a0 con 0x7f965c0caeb0 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.601+0000 7f96597fa700 1 -- 172.21.15.6:0/1380745067 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965001f070 con 0x7f965c0caeb0 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.601+0000 7f96597fa700 1 -- 172.21.15.6:0/1380745067 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650003a20 con 0x7f965c0caeb0 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.601+0000 7f9662d43700 1 -- 172.21.15.6:0/1380745067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965c0cf5b0 con 0x7f965c0caeb0 2023-11-28T13:03:08.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.601+0000 7f9662d43700 1 -- 172.21.15.6:0/1380745067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965c0ce870 con 0x7f965c0caeb0 2023-11-28T13:03:08.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.602+0000 7f9662d43700 1 -- 172.21.15.6:0/1380745067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c0cf740 con 0x7f965c0caeb0 2023-11-28T13:03:08.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.602+0000 7f96597fa700 1 -- 172.21.15.6:0/1380745067 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f965000f050 con 0x7f965c0caeb0 2023-11-28T13:03:08.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.603+0000 7f96597fa700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96400682f0 0x7f964006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:08.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.603+0000 7f96597fa700 1 -- 172.21.15.6:0/1380745067 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9650021070 con 0x7f965c0caeb0 2023-11-28T13:03:08.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.603+0000 7f965bfff700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96400682f0 0x7f964006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:08.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.603+0000 7f965bfff700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96400682f0 0x7f964006a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f965c0b5060 tx=0x7f9648008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:08.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.607+0000 7f96597fa700 1 -- 172.21.15.6:0/1380745067 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9650053ed0 con 0x7f965c0caeb0 2023-11-28T13:03:08.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:08.881+0000 7f9662d43700 1 -- 172.21.15.6:0/1380745067 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f965c15c540 con 0x7f965c0caeb0 2023-11-28T13:03:09.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.084+0000 7f96597fa700 1 -- 172.21.15.6:0/1380745067 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v165) v1 ==== 149+0+0 (secure 0 0 0) 0x7f9650085e60 con 0x7f965c0caeb0 2023-11-28T13:03:09.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 -- 172.21.15.6:0/1380745067 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96400682f0 msgr2=0x7f964006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:09.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96400682f0 0x7f964006a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f965c0b5060 tx=0x7f9648008040 comp rx=0 tx=0).stop 2023-11-28T13:03:09.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 -- 172.21.15.6:0/1380745067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c0caeb0 msgr2=0x7f965c0b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:09.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c0caeb0 0x7f965c0b5420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9650000c00 tx=0x7f9650004750 comp rx=0 tx=0).stop 2023-11-28T13:03:09.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 -- 172.21.15.6:0/1380745067 shutdown_connections 2023-11-28T13:03:09.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f965c161090 0x7f965c0c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:09.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f965c0caeb0 0x7f965c0b5420 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:09.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96400682f0 0x7f964006a7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:09.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 --2- 172.21.15.6:0/1380745067 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f965c15c990 0x7f965c1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:09.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 -- 172.21.15.6:0/1380745067 >> 172.21.15.6:0/1380745067 conn(0x7f965c0b1650 msgr2=0x7f965c0c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:09.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 -- 172.21.15.6:0/1380745067 shutdown_connections 2023-11-28T13:03:09.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.087+0000 7f9662d43700 1 -- 172.21.15.6:0/1380745067 wait complete. 2023-11-28T13:03:09.101 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-28T13:03:09.101 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-28T13:03:09.101 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:03:09.101 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T13:03:09.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.226+0000 7f6306867700 1 Processor -- start 2023-11-28T13:03:09.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.227+0000 7f6306867700 1 -- start start 2023-11-28T13:03:09.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.227+0000 7f6306867700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f630015c990 0x7f63000cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:09.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.227+0000 7f6306867700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f63000cc310 0x7f63000cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:09.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.227+0000 7f6306867700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63000ce340 con 0x7f6300161090 2023-11-28T13:03:09.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.227+0000 7f6306867700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63000cebe0 con 0x7f63000cc310 2023-11-28T13:03:09.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.227+0000 7f6306867700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63000cccf0 con 0x7f630015c990 2023-11-28T13:03:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f62fffff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f630015c990 0x7f63000cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f6304e04700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f63000cc310 0x7f63000cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f62ff7fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f6300161090 0x7f63000ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:39544/0 (socket says 172.21.15.6:39544) 2023-11-28T13:03:09.232 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f62ff7fe700 1 -- 172.21.15.6:0/1826420094 learned_addr learned my addr 172.21.15.6:0/1826420094 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:09.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f6304e04700 1 -- 172.21.15.6:0/1826420094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f630015c990 msgr2=0x7f63000cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:09.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f6304e04700 1 --2- 172.21.15.6:0/1826420094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f630015c990 0x7f63000cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:09.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f6304e04700 1 -- 172.21.15.6:0/1826420094 >> v1:172.21.15.61:6789/0 conn(0x7f6300161090 legacy=0x7f63000ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:09.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f6304e04700 1 -- 172.21.15.6:0/1826420094 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63000cb3f0 con 0x7f63000cc310 2023-11-28T13:03:09.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f62feffd700 1 -- 172.21.15.6:0/1826420094 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532659253 0 0) 0x7f63000ce340 con 0x7f6300161090 2023-11-28T13:03:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f6304e04700 1 --2- 172.21.15.6:0/1826420094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63000cc310 0x7f63000cc7b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009d90 tx=0x7f62f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24c701ea77bcefd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f62feffd700 1 -- 172.21.15.6:0/1826420094 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f800e070 con 0x7f63000cc310 2023-11-28T13:03:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f62feffd700 1 -- 172.21.15.6:0/1826420094 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f800bb20 con 0x7f63000cc310 2023-11-28T13:03:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.228+0000 7f62feffd700 1 -- 172.21.15.6:0/1826420094 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8012430 con 0x7f63000cc310 2023-11-28T13:03:09.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.229+0000 7f6306867700 1 -- 172.21.15.6:0/1826420094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63000cc310 msgr2=0x7f63000cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:09.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.229+0000 7f6306867700 1 --2- 172.21.15.6:0/1826420094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63000cc310 0x7f63000cc7b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009d90 tx=0x7f62f800b330 comp rx=0 tx=0).stop 2023-11-28T13:03:09.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.229+0000 7f6306867700 1 -- 172.21.15.6:0/1826420094 shutdown_connections 2023-11-28T13:03:09.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.229+0000 7f6306867700 1 --2- 172.21.15.6:0/1826420094 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63000cc310 0x7f63000cc7b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:09.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.229+0000 7f6306867700 1 --2- 172.21.15.6:0/1826420094 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f630015c990 0x7f63000cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:09.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.229+0000 7f6306867700 1 -- 172.21.15.6:0/1826420094 >> 172.21.15.6:0/1826420094 conn(0x7f63000b1650 msgr2=0x7f63000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:09.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.229+0000 7f6306867700 1 -- 172.21.15.6:0/1826420094 shutdown_connections 2023-11-28T13:03:09.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.229+0000 7f6306867700 1 -- 172.21.15.6:0/1826420094 wait complete. 2023-11-28T13:03:09.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.230+0000 7f6306867700 1 Processor -- start 2023-11-28T13:03:09.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.230+0000 7f6306867700 1 -- start start 2023-11-28T13:03:09.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f6306867700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63000cc310 0x7f6300159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:09.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f6306867700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f630015c990 0x7f63000b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:09.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f6306867700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6300161090 0x7f63000cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:09.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f6306867700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630015d500 con 0x7f630015c990 2023-11-28T13:03:09.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f6306867700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630015a500 con 0x7f6300161090 2023-11-28T13:03:09.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f6306867700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63001f5560 con 0x7f63000cc310 2023-11-28T13:03:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f62ff7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f630015c990 0x7f63000b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f62ff7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f630015c990 0x7f63000b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33968/0 (socket says 172.21.15.6:33968) 2023-11-28T13:03:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f62ff7fe700 1 -- 172.21.15.6:0/2317440292 learned_addr learned my addr 172.21.15.6:0/2317440292 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f62fffff700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63000cc310 0x7f6300159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.231+0000 7f6304e04700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6300161090 0x7f63000cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f62ff7fe700 1 -- 172.21.15.6:0/2317440292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63000cc310 msgr2=0x7f6300159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f62ff7fe700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63000cc310 0x7f6300159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f62ff7fe700 1 -- 172.21.15.6:0/2317440292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6300161090 msgr2=0x7f63000cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f62ff7fe700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6300161090 0x7f63000cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:09.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f62ff7fe700 1 -- 172.21.15.6:0/2317440292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f8009a70 con 0x7f630015c990 2023-11-28T13:03:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f62fffff700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63000cc310 0x7f6300159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f62ff7fe700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f630015c990 0x7f63000b45b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f62f400c540 tx=0x7f62f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f62fcff9700 1 -- 172.21.15.6:0/2317440292 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4004c80 con 0x7f630015c990 2023-11-28T13:03:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f62fcff9700 1 -- 172.21.15.6:0/2317440292 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f4004de0 con 0x7f630015c990 2023-11-28T13:03:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f6306867700 1 -- 172.21.15.6:0/2317440292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63000ce870 con 0x7f630015c990 2023-11-28T13:03:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f62fcff9700 1 -- 172.21.15.6:0/2317440292 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4007310 con 0x7f630015c990 2023-11-28T13:03:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.232+0000 7f6306867700 1 -- 172.21.15.6:0/2317440292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63000ce590 con 0x7f630015c990 2023-11-28T13:03:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.233+0000 7f6306867700 1 -- 172.21.15.6:0/2317440292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63000cb080 con 0x7f630015c990 2023-11-28T13:03:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.233+0000 7f62fcff9700 1 -- 172.21.15.6:0/2317440292 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f62f401a020 con 0x7f630015c990 2023-11-28T13:03:09.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.234+0000 7f62fcff9700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f62ec0682f0 0x7f62ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.234+0000 7f62fcff9700 1 -- 172.21.15.6:0/2317440292 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f62f4085770 con 0x7f630015c990 2023-11-28T13:03:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.234+0000 7f62fffff700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f62ec0682f0 0x7f62ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.234+0000 7f62fffff700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f62ec0682f0 0x7f62ec06a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009c30 tx=0x7f62f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:09.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.238+0000 7f62fcff9700 1 -- 172.21.15.6:0/2317440292 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f62f40521c0 con 0x7f630015c990 2023-11-28T13:03:09.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:09.512+0000 7f6306867700 1 -- 172.21.15.6:0/2317440292 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f63000675a0 con 0x7f630015c990 2023-11-28T13:03:10.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.087+0000 7f62fcff9700 1 -- 172.21.15.6:0/2317440292 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v166) v1 ==== 172+0+0 (secure 0 0 0) 0x7f62f40557e0 con 0x7f630015c990 2023-11-28T13:03:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.089+0000 7f6306867700 1 -- 172.21.15.6:0/2317440292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f62ec0682f0 msgr2=0x7f62ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.089+0000 7f6306867700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f62ec0682f0 0x7f62ec06a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f62f0009c30 tx=0x7f62f0008040 comp rx=0 tx=0).stop 2023-11-28T13:03:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.089+0000 7f6306867700 1 -- 172.21.15.6:0/2317440292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f630015c990 msgr2=0x7f63000b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.089+0000 7f6306867700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f630015c990 0x7f63000b45b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f62f400c540 tx=0x7f62f400c850 comp rx=0 tx=0).stop 2023-11-28T13:03:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.089+0000 7f6306867700 1 -- 172.21.15.6:0/2317440292 shutdown_connections 2023-11-28T13:03:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.089+0000 7f6306867700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6300161090 0x7f63000cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.089+0000 7f6306867700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f62ec0682f0 0x7f62ec06a7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.089+0000 7f6306867700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f63000cc310 0x7f6300159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.089+0000 7f6306867700 1 --2- 172.21.15.6:0/2317440292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f630015c990 0x7f63000b45b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.089+0000 7f6306867700 1 -- 172.21.15.6:0/2317440292 >> 172.21.15.6:0/2317440292 conn(0x7f63000b1650 msgr2=0x7f630006b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:10.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.090+0000 7f6306867700 1 -- 172.21.15.6:0/2317440292 shutdown_connections 2023-11-28T13:03:10.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.090+0000 7f6306867700 1 -- 172.21.15.6:0/2317440292 wait complete. 2023-11-28T13:03:10.107 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T13:03:10.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.243+0000 7ff85df64700 1 Processor -- start 2023-11-28T13:03:10.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.244+0000 7ff85df64700 1 -- start start 2023-11-28T13:03:10.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.244+0000 7ff85df64700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff8580b1630 0x7ff8580cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.244+0000 7ff85df64700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff85806afc0 0x7ff85806b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.244+0000 7ff85df64700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8580ce550 con 0x7ff8580b1a00 2023-11-28T13:03:10.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.244+0000 7ff85df64700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8580ce830 con 0x7ff8580b1630 2023-11-28T13:03:10.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff85df64700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8580ceb10 con 0x7ff85806afc0 2023-11-28T13:03:10.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8577fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff8580b1630 0x7ff8580cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8577fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff8580b1630 0x7ff8580cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33976/0 (socket says 172.21.15.6:33976) 2023-11-28T13:03:10.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8577fe700 1 -- 172.21.15.6:0/1450608905 learned_addr learned my addr 172.21.15.6:0/1450608905 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:10.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff857fff700 1 --2- 172.21.15.6:0/1450608905 >> v2:172.21.15.61:3300/0 conn(0x7ff85806afc0 0x7ff85806b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8577fe700 1 -- 172.21.15.6:0/1450608905 >> v2:172.21.15.61:3300/0 conn(0x7ff85806afc0 msgr2=0x7ff85806b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:10.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8577fe700 1 --2- 172.21.15.6:0/1450608905 >> v2:172.21.15.61:3300/0 conn(0x7ff85806afc0 0x7ff85806b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8577fe700 1 -- 172.21.15.6:0/1450608905 >> v1:172.21.15.61:6789/0 conn(0x7ff8580b1a00 legacy=0x7ff8580caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8577fe700 1 -- 172.21.15.6:0/1450608905 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8580cedf0 con 0x7ff8580b1630 2023-11-28T13:03:10.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8577fe700 1 --2- 172.21.15.6:0/1450608905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1630 0x7ff8580cf5f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff8480098f0 tx=0x7ff848009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e6f26e5ad1fa0d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:10.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8567fc700 1 -- 172.21.15.6:0/1450608905 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848011830 con 0x7ff8580b1630 2023-11-28T13:03:10.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8567fc700 1 -- 172.21.15.6:0/1450608905 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff848011990 con 0x7ff8580b1630 2023-11-28T13:03:10.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.245+0000 7ff8567fc700 1 -- 172.21.15.6:0/1450608905 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848011d20 con 0x7ff8580b1630 2023-11-28T13:03:10.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.246+0000 7ff85df64700 1 -- 172.21.15.6:0/1450608905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1630 msgr2=0x7ff8580cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.246+0000 7ff85df64700 1 --2- 172.21.15.6:0/1450608905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1630 0x7ff8580cf5f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff8480098f0 tx=0x7ff848009c00 comp rx=0 tx=0).stop 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.246+0000 7ff85df64700 1 -- 172.21.15.6:0/1450608905 shutdown_connections 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.246+0000 7ff85df64700 1 --2- 172.21.15.6:0/1450608905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1630 0x7ff8580cf5f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.246+0000 7ff85df64700 1 --2- 172.21.15.6:0/1450608905 >> v2:172.21.15.61:3300/0 conn(0x7ff85806afc0 0x7ff85806b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.246+0000 7ff85df64700 1 -- 172.21.15.6:0/1450608905 >> 172.21.15.6:0/1450608905 conn(0x7ff8580cc630 msgr2=0x7ff8580cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.246+0000 7ff85df64700 1 -- 172.21.15.6:0/1450608905 shutdown_connections 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.246+0000 7ff85df64700 1 -- 172.21.15.6:0/1450608905 wait complete. 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.247+0000 7ff85df64700 1 Processor -- start 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.248+0000 7ff85df64700 1 -- start start 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff85df64700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff85806afc0 0x7ff85815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff85df64700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8580b1630 0x7ff8581e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff85df64700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1a00 0x7ff8581e3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff85df64700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85815a9c0 con 0x7ff8580b1a00 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff85df64700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580af670 con 0x7ff85806afc0 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff85df64700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580af950 con 0x7ff8580b1630 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff857fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1a00 0x7ff8581e3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff857fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1a00 0x7ff8581e3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33986/0 (socket says 172.21.15.6:33986) 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff857fff700 1 -- 172.21.15.6:0/517675520 learned_addr learned my addr 172.21.15.6:0/517675520 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff8577fe700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff85806afc0 0x7ff85815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff856ffd700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8580b1630 0x7ff8581e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff857fff700 1 -- 172.21.15.6:0/517675520 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8580b1630 msgr2=0x7ff8581e1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff857fff700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8580b1630 0x7ff8581e1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff857fff700 1 -- 172.21.15.6:0/517675520 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff85806afc0 msgr2=0x7ff85815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff857fff700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff85806afc0 0x7ff85815a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff857fff700 1 -- 172.21.15.6:0/517675520 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff848009550 con 0x7ff8580b1a00 2023-11-28T13:03:10.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff857fff700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1a00 0x7ff8581e3c70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff85000efc0 tx=0x7ff85000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:10.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.249+0000 7ff83ffff700 1 -- 172.21.15.6:0/517675520 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850009ea0 con 0x7ff8580b1a00 2023-11-28T13:03:10.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.250+0000 7ff83ffff700 1 -- 172.21.15.6:0/517675520 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85000f040 con 0x7ff8580b1a00 2023-11-28T13:03:10.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.250+0000 7ff85df64700 1 -- 172.21.15.6:0/517675520 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8580b1ea0 con 0x7ff8580b1a00 2023-11-28T13:03:10.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.250+0000 7ff83ffff700 1 -- 172.21.15.6:0/517675520 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85000ced0 con 0x7ff8580b1a00 2023-11-28T13:03:10.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.250+0000 7ff85df64700 1 -- 172.21.15.6:0/517675520 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8580cf0d0 con 0x7ff8580b1a00 2023-11-28T13:03:10.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.251+0000 7ff85df64700 1 -- 172.21.15.6:0/517675520 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8580cb5b0 con 0x7ff8580b1a00 2023-11-28T13:03:10.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.251+0000 7ff83ffff700 1 -- 172.21.15.6:0/517675520 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff850014480 con 0x7ff8580b1a00 2023-11-28T13:03:10.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.251+0000 7ff83ffff700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8440682f0 0x7ff84406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.252+0000 7ff83ffff700 1 -- 172.21.15.6:0/517675520 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7ff8500863f0 con 0x7ff8580b1a00 2023-11-28T13:03:10.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.252+0000 7ff8577fe700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8440682f0 0x7ff84406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.252+0000 7ff8577fe700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8440682f0 0x7ff84406a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff848009c40 tx=0x7ff848008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:10.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.256+0000 7ff83ffff700 1 -- 172.21.15.6:0/517675520 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff850052db0 con 0x7ff8580b1a00 2023-11-28T13:03:10.449 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.448+0000 7ff85df64700 1 -- 172.21.15.6:0/517675520 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff8580ce140 con 0x7ff8580b1a00 2023-11-28T13:03:10.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.452+0000 7ff83ffff700 1 -- 172.21.15.6:0/517675520 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff8500563d0 con 0x7ff8580b1a00 2023-11-28T13:03:10.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.453+0000 7ff85df64700 1 -- 172.21.15.6:0/517675520 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8440682f0 msgr2=0x7ff84406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.453+0000 7ff85df64700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8440682f0 0x7ff84406a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff848009c40 tx=0x7ff848008030 comp rx=0 tx=0).stop 2023-11-28T13:03:10.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.453+0000 7ff85df64700 1 -- 172.21.15.6:0/517675520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1a00 msgr2=0x7ff8581e3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.453+0000 7ff85df64700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1a00 0x7ff8581e3c70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff85000efc0 tx=0x7ff85000c3a0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.454+0000 7ff85df64700 1 -- 172.21.15.6:0/517675520 shutdown_connections 2023-11-28T13:03:10.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.454+0000 7ff85df64700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8580b1a00 0x7ff8581e3c70 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.454+0000 7ff85df64700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff85806afc0 0x7ff85815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.454+0000 7ff85df64700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8440682f0 0x7ff84406a7a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.454+0000 7ff85df64700 1 --2- 172.21.15.6:0/517675520 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8580b1630 0x7ff8581e1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.454+0000 7ff85df64700 1 -- 172.21.15.6:0/517675520 >> 172.21.15.6:0/517675520 conn(0x7ff8580cc630 msgr2=0x7ff85806b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:10.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.454+0000 7ff85df64700 1 -- 172.21.15.6:0/517675520 shutdown_connections 2023-11-28T13:03:10.456 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.454+0000 7ff85df64700 1 -- 172.21.15.6:0/517675520 wait complete. 2023-11-28T13:03:10.467 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:03:10.467 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:03:10.512 DEBUG:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:03:10.556 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:03:10.556 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:03:10.556 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:03:10.574 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3446 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:03:10.574 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:03:10.575 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:03:10.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:03:10.595 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:03:10.595 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:03:10.632 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3446 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:03:10.632 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:03:10.632 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:03:10.655 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:03:10.655 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T13:03:10.655 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:10.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.795+0000 7f955ab20700 1 Processor -- start 2023-11-28T13:03:10.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.795+0000 7f955ab20700 1 -- start start 2023-11-28T13:03:10.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f955ab20700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f95540b2660 0x7f95540b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f955ab20700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95540cfb90 con 0x7f95540cb230 2023-11-28T13:03:10.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f955ab20700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95540cb680 con 0x7f955415a6c0 2023-11-28T13:03:10.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f955ab20700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95540b3020 con 0x7f95540b2660 2023-11-28T13:03:10.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f95590bd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f95540b2660 0x7f95540b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f9553fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f95540cb230 0x7f95540ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49542/0 (socket says 172.21.15.6:49542) 2023-11-28T13:03:10.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f9553fff700 1 -- 172.21.15.6:0/3693911274 learned_addr learned my addr 172.21.15.6:0/3693911274 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:10.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f95590bd700 1 -- 172.21.15.6:0/3693911274 >> v1:172.21.15.61:6789/0 conn(0x7f955415a6c0 legacy=0x7f95540d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:10.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f95590bd700 1 -- 172.21.15.6:0/3693911274 >> v1:172.21.15.6:6789/0 conn(0x7f95540cb230 legacy=0x7f95540ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f95537fe700 1 -- 172.21.15.6:0/3693911274 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968393975 0 0) 0x7f95540cfb90 con 0x7f95540cb230 2023-11-28T13:03:10.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.796+0000 7f95590bd700 1 -- 172.21.15.6:0/3693911274 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95540cb930 con 0x7f95540b2660 2023-11-28T13:03:10.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f95590bd700 1 --2- 172.21.15.6:0/3693911274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 0x7f95540b2ae0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f954c009d90 tx=0x7f954c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8145bbc40d40248 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:10.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f95537fe700 1 -- 172.21.15.6:0/3693911274 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c00e070 con 0x7f95540b2660 2023-11-28T13:03:10.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f95537fe700 1 -- 172.21.15.6:0/3693911274 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c00bb90 con 0x7f95540b2660 2023-11-28T13:03:10.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f95537fe700 1 -- 172.21.15.6:0/3693911274 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c003a20 con 0x7f95540b2660 2023-11-28T13:03:10.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f955ab20700 1 -- 172.21.15.6:0/3693911274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 msgr2=0x7f95540b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f955ab20700 1 --2- 172.21.15.6:0/3693911274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 0x7f95540b2ae0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f954c009d90 tx=0x7f954c00b330 comp rx=0 tx=0).stop 2023-11-28T13:03:10.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f955ab20700 1 -- 172.21.15.6:0/3693911274 shutdown_connections 2023-11-28T13:03:10.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f955ab20700 1 --2- 172.21.15.6:0/3693911274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 0x7f95540b2ae0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.802 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f955ab20700 1 -- 172.21.15.6:0/3693911274 >> 172.21.15.6:0/3693911274 conn(0x7f95540cd660 msgr2=0x7f95540cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f955ab20700 1 -- 172.21.15.6:0/3693911274 shutdown_connections 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.797+0000 7f955ab20700 1 -- 172.21.15.6:0/3693911274 wait complete. 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.798+0000 7f955ab20700 1 Processor -- start 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.798+0000 7f955ab20700 1 -- start start 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.799+0000 7f955ab20700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 0x7f95541f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.799+0000 7f955ab20700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95540cb230 0x7f95540ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.799+0000 7f955ab20700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f955415a6c0 0x7f95540b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.799+0000 7f955ab20700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540c8220 con 0x7f95540b2660 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.799+0000 7f955ab20700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95541f1c40 con 0x7f955415a6c0 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.799+0000 7f955ab20700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95540ace10 con 0x7f95540cb230 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.799+0000 7f95588bc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 0x7f95541f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.799+0000 7f95588bc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 0x7f95541f1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:33996/0 (socket says 172.21.15.6:33996) 2023-11-28T13:03:10.803 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.799+0000 7f95588bc700 1 -- 172.21.15.6:0/608536314 learned_addr learned my addr 172.21.15.6:0/608536314 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95590bd700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f955415a6c0 0x7f95540b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f9553fff700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95540cb230 0x7f95540ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95588bc700 1 -- 172.21.15.6:0/608536314 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95540cb230 msgr2=0x7f95540ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95588bc700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95540cb230 0x7f95540ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95588bc700 1 -- 172.21.15.6:0/608536314 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f955415a6c0 msgr2=0x7f95540b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95588bc700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f955415a6c0 0x7f95540b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95588bc700 1 -- 172.21.15.6:0/608536314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c009a70 con 0x7f95540b2660 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95590bd700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f955415a6c0 0x7f95540b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95588bc700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 0x7f95541f1700 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f954800c4d0 tx=0x7f954800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95517fa700 1 -- 172.21.15.6:0/608536314 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548011790 con 0x7f95540b2660 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95517fa700 1 -- 172.21.15.6:0/608536314 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95480118f0 con 0x7f95540b2660 2023-11-28T13:03:10.804 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f955ab20700 1 -- 172.21.15.6:0/608536314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95540cf540 con 0x7f95540b2660 2023-11-28T13:03:10.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f95517fa700 1 -- 172.21.15.6:0/608536314 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548011ca0 con 0x7f95540b2660 2023-11-28T13:03:10.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.800+0000 7f955ab20700 1 -- 172.21.15.6:0/608536314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95540cf260 con 0x7f95540b2660 2023-11-28T13:03:10.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.801+0000 7f955ab20700 1 -- 172.21.15.6:0/608536314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95540cf820 con 0x7f95540b2660 2023-11-28T13:03:10.805 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.802+0000 7f95517fa700 1 -- 172.21.15.6:0/608536314 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9548010050 con 0x7f95540b2660 2023-11-28T13:03:10.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.802+0000 7f95517fa700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95380682f0 0x7f953806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:10.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.802+0000 7f95517fa700 1 -- 172.21.15.6:0/608536314 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9548085850 con 0x7f95540b2660 2023-11-28T13:03:10.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.802+0000 7f9553fff700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95380682f0 0x7f953806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:10.806 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.803+0000 7f9553fff700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95380682f0 0x7f953806a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9554159270 tx=0x7f9540007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:10.807 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:10.806+0000 7f95517fa700 1 -- 172.21.15.6:0/608536314 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9548052270 con 0x7f95540b2660 2023-11-28T13:03:11.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.042+0000 7f955ab20700 1 -- 172.21.15.6:0/608536314 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95540675a0 con 0x7f95540b2660 2023-11-28T13:03:11.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.043+0000 7f95517fa700 1 -- 172.21.15.6:0/608536314 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3520 (secure 0 0 0) 0x7f9548055890 con 0x7f95540b2660 2023-11-28T13:03:11.044 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:11.045 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5258,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/4004512137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":4004512137},{"type":"v1","addr":"172.21.15.61:6833","nonce":4004512137}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5261,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/1313775424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":1313775424},{"type":"v1","addr":"172.21.15.61:6835","nonce":1313775424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/233781200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":233781200},{"type":"v1","addr":"172.21.15.6:6836","nonce":233781200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6393,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/3496314163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":3496314163},{"type":"v1","addr":"172.21.15.6:6837","nonce":3496314163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-11-28T13:03:11.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 -- 172.21.15.6:0/608536314 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95380682f0 msgr2=0x7f953806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:11.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95380682f0 0x7f953806a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9554159270 tx=0x7f9540007c90 comp rx=0 tx=0).stop 2023-11-28T13:03:11.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 -- 172.21.15.6:0/608536314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 msgr2=0x7f95541f1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:11.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 0x7f95541f1700 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f954800c4d0 tx=0x7f954800c890 comp rx=0 tx=0).stop 2023-11-28T13:03:11.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 -- 172.21.15.6:0/608536314 shutdown_connections 2023-11-28T13:03:11.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f955415a6c0 0x7f95540b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:11.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f95540b2660 0x7f95541f1700 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:11.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95380682f0 0x7f953806a7a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:11.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 --2- 172.21.15.6:0/608536314 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f95540cb230 0x7f95540ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:11.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 -- 172.21.15.6:0/608536314 >> 172.21.15.6:0/608536314 conn(0x7f95540cd660 msgr2=0x7f95540b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:11.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 -- 172.21.15.6:0/608536314 shutdown_connections 2023-11-28T13:03:11.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.045+0000 7f955ab20700 1 -- 172.21.15.6:0/608536314 wait complete. 2023-11-28T13:03:11.048 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 26 2023-11-28T13:03:11.059 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T13:03:11.059 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T13:03:11.060 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:03:11.060 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:03:11.060 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T13:03:11.060 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T13:03:11.061 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:03:11.061 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:03:11.061 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T13:03:11.061 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T13:03:11.061 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:03:11.061 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:03:11.061 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T13:03:11.062 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T13:03:11.062 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:03:11.062 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:03:11.069 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:03:11.069 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T13:03:11.070 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:03:11.070 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T13:03:11.073 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:03:11.073 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T13:03:11.073 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:03:11.074 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T13:03:11.111 INFO:tasks.ceph.mds.a:Started 2023-11-28T13:03:11.112 DEBUG:teuthology.parallel:result is None 2023-11-28T13:03:11.113 INFO:tasks.ceph.mds.c:Started 2023-11-28T13:03:11.113 DEBUG:teuthology.parallel:result is None 2023-11-28T13:03:11.116 INFO:tasks.ceph.mds.b:Started 2023-11-28T13:03:11.116 DEBUG:teuthology.parallel:result is None 2023-11-28T13:03:11.116 INFO:tasks.ceph.mds.d:Started 2023-11-28T13:03:11.116 DEBUG:teuthology.parallel:result is None 2023-11-28T13:03:11.117 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T13:03:11.193 INFO:tasks.ceph.mds.a.smithi006.stdout:starting mds.a at 2023-11-28T13:03:11.194 INFO:tasks.ceph.mds.c.smithi006.stdout:starting mds.c at 2023-11-28T13:03:11.194 INFO:tasks.ceph.mds.a.smithi006.stderr:2023-11-28T13:03:11.193+0000 7f0d9b8b0700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-28T13:03:11.209 INFO:tasks.ceph.mds.d.smithi061.stdout:starting mds.d at 2023-11-28T13:03:11.211 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2023-11-28T13:03:11.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.253+0000 7f0688edb700 1 Processor -- start 2023-11-28T13:03:11.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.253+0000 7f0688edb700 1 -- start start 2023-11-28T13:03:11.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.254+0000 7f0688edb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f068415a6c0 0x7f06840cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:11.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.254+0000 7f0688edb700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06840cfb90 con 0x7f06840d0020 2023-11-28T13:03:11.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.254+0000 7f0688edb700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06840cb930 con 0x7f06840b2660 2023-11-28T13:03:11.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.254+0000 7f0688edb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06840cf260 con 0x7f068415a6c0 2023-11-28T13:03:11.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f0681d9b700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f06840d0020 0x7f06840ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49582/0 (socket says 172.21.15.6:49582) 2023-11-28T13:03:11.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f0681d9b700 1 -- 172.21.15.6:0/1878046381 learned_addr learned my addr 172.21.15.6:0/1878046381 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:11.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f068259c700 1 --2- 172.21.15.6:0/1878046381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f068415a6c0 0x7f06840cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:11.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f068159a700 1 -- 172.21.15.6:0/1878046381 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861752069 0 0) 0x7f06840cfb90 con 0x7f06840d0020 2023-11-28T13:03:11.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f068159a700 1 -- 172.21.15.6:0/1878046381 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f066c0035b0 con 0x7f06840d0020 2023-11-28T13:03:11.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f068159a700 1 -- 172.21.15.6:0/1878046381 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566640845 0 0) 0x7f06840cb930 con 0x7f06840b2660 2023-11-28T13:03:11.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f068159a700 1 -- 172.21.15.6:0/1878046381 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06840cfb90 con 0x7f06840b2660 2023-11-28T13:03:11.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f068259c700 1 -- 172.21.15.6:0/1878046381 >> v1:172.21.15.61:6789/0 conn(0x7f06840b2660 legacy=0x7f06840b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:11.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f068259c700 1 -- 172.21.15.6:0/1878046381 >> v1:172.21.15.6:6789/0 conn(0x7f06840d0020 legacy=0x7f06840ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:11.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f068159a700 1 -- 172.21.15.6:0/1878046381 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1008692754 0 0) 0x7f066c0035b0 con 0x7f06840d0020 2023-11-28T13:03:11.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f068259c700 1 -- 172.21.15.6:0/1878046381 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06840cf540 con 0x7f068415a6c0 2023-11-28T13:03:11.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.255+0000 7f068259c700 1 --2- 172.21.15.6:0/1878046381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f068415a6c0 0x7f06840cb230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0674009870 tx=0x7f0674009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe62e191a206c147 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:11.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.256+0000 7f068159a700 1 -- 172.21.15.6:0/1878046381 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678001670 con 0x7f068415a6c0 2023-11-28T13:03:11.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.256+0000 7f068159a700 1 -- 172.21.15.6:0/1878046381 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0674004030 con 0x7f068415a6c0 2023-11-28T13:03:11.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.256+0000 7f068159a700 1 -- 172.21.15.6:0/1878046381 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06740043c0 con 0x7f068415a6c0 2023-11-28T13:03:11.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.256+0000 7f0688edb700 1 -- 172.21.15.6:0/1878046381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f068415a6c0 msgr2=0x7f06840cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:11.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.256+0000 7f0688edb700 1 --2- 172.21.15.6:0/1878046381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f068415a6c0 0x7f06840cb230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0674009870 tx=0x7f0674009b80 comp rx=0 tx=0).stop 2023-11-28T13:03:11.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.256+0000 7f0688edb700 1 -- 172.21.15.6:0/1878046381 shutdown_connections 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.256+0000 7f0688edb700 1 --2- 172.21.15.6:0/1878046381 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f068415a6c0 0x7f06840cb230 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.256+0000 7f0688edb700 1 -- 172.21.15.6:0/1878046381 >> 172.21.15.6:0/1878046381 conn(0x7f06840cd660 msgr2=0x7f06840cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.256+0000 7f0688edb700 1 -- 172.21.15.6:0/1878046381 shutdown_connections 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.256+0000 7f0688edb700 1 -- 172.21.15.6:0/1878046381 wait complete. 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.258+0000 7f0688edb700 1 Processor -- start 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.258+0000 7f0688edb700 1 -- start start 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.258+0000 7f0688edb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06840b2660 0x7f06840ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.258+0000 7f0688edb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06840d0020 0x7f0684158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.258+0000 7f0688edb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f068415a6c0 0x7f06840adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.258+0000 7f0688edb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06840af050 con 0x7f068415a6c0 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.258+0000 7f0688edb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06840ab9c0 con 0x7f06840b2660 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.258+0000 7f0688edb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06840ac910 con 0x7f06840d0020 2023-11-28T13:03:11.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0682d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f068415a6c0 0x7f06840adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0682d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f068415a6c0 0x7f06840adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34038/0 (socket says 172.21.15.6:34038) 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0682d9d700 1 -- 172.21.15.6:0/1738645364 learned_addr learned my addr 172.21.15.6:0/1738645364 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f068259c700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06840b2660 0x7f06840ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0682d9d700 1 -- 172.21.15.6:0/1738645364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06840d0020 msgr2=0x7f0684158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0681d9b700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06840d0020 0x7f0684158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0682d9d700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06840d0020 0x7f0684158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0682d9d700 1 -- 172.21.15.6:0/1738645364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06840b2660 msgr2=0x7f06840ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0682d9d700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06840b2660 0x7f06840ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0682d9d700 1 -- 172.21.15.6:0/1738645364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06740094a0 con 0x7f068415a6c0 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f068259c700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06840b2660 0x7f06840ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0681d9b700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06840d0020 0x7f0684158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0682d9d700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f068415a6c0 0x7f06840adda0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f067c00cc20 tx=0x7f067c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0672ffd700 1 -- 172.21.15.6:0/1738645364 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c00ac20 con 0x7f068415a6c0 2023-11-28T13:03:11.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0672ffd700 1 -- 172.21.15.6:0/1738645364 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c00ad80 con 0x7f068415a6c0 2023-11-28T13:03:11.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0688edb700 1 -- 172.21.15.6:0/1738645364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06840d0470 con 0x7f068415a6c0 2023-11-28T13:03:11.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0672ffd700 1 -- 172.21.15.6:0/1738645364 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c003c20 con 0x7f068415a6c0 2023-11-28T13:03:11.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.259+0000 7f0688edb700 1 -- 172.21.15.6:0/1738645364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06840cf820 con 0x7f068415a6c0 2023-11-28T13:03:11.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.260+0000 7f0688edb700 1 -- 172.21.15.6:0/1738645364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06840d0600 con 0x7f068415a6c0 2023-11-28T13:03:11.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.261+0000 7f0672ffd700 1 -- 172.21.15.6:0/1738645364 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f067c010050 con 0x7f068415a6c0 2023-11-28T13:03:11.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.261+0000 7f0672ffd700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06640682f0 0x7f066406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:11.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.261+0000 7f0672ffd700 1 -- 172.21.15.6:0/1738645364 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f067c085bb0 con 0x7f068415a6c0 2023-11-28T13:03:11.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.261+0000 7f068259c700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06640682f0 0x7f066406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:11.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.262+0000 7f068259c700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06640682f0 0x7f066406a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06840b6800 tx=0x7f0674013040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:11.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.265+0000 7f0672ffd700 1 -- 172.21.15.6:0/1738645364 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f067c052550 con 0x7f068415a6c0 2023-11-28T13:03:11.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:11.474+0000 7f0688edb700 1 -- 172.21.15.6:0/1738645364 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f06840af190 con 0x7f068415a6c0 2023-11-28T13:03:12.459 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.457+0000 7f0672ffd700 1 -- 172.21.15.6:0/1738645364 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7f06840af190 con 0x7f068415a6c0 2023-11-28T13:03:12.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.460+0000 7f0688edb700 1 -- 172.21.15.6:0/1738645364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06640682f0 msgr2=0x7f066406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:12.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.460+0000 7f0688edb700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06640682f0 0x7f066406a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f06840b6800 tx=0x7f0674013040 comp rx=0 tx=0).stop 2023-11-28T13:03:12.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.460+0000 7f0688edb700 1 -- 172.21.15.6:0/1738645364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f068415a6c0 msgr2=0x7f06840adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:12.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.460+0000 7f0688edb700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f068415a6c0 0x7f06840adda0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f067c00cc20 tx=0x7f067c00a060 comp rx=0 tx=0).stop 2023-11-28T13:03:12.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.460+0000 7f0688edb700 1 -- 172.21.15.6:0/1738645364 shutdown_connections 2023-11-28T13:03:12.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.460+0000 7f0688edb700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f068415a6c0 0x7f06840adda0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.462 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.460+0000 7f0688edb700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f06840b2660 0x7f06840ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.460+0000 7f0688edb700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f06640682f0 0x7f066406a7a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.460+0000 7f0688edb700 1 --2- 172.21.15.6:0/1738645364 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f06840d0020 0x7f0684158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.460+0000 7f0688edb700 1 -- 172.21.15.6:0/1738645364 >> 172.21.15.6:0/1738645364 conn(0x7f06840cd660 msgr2=0x7f06840b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:12.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.461+0000 7f0688edb700 1 -- 172.21.15.6:0/1738645364 shutdown_connections 2023-11-28T13:03:12.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.461+0000 7f0688edb700 1 -- 172.21.15.6:0/1738645364 wait complete. 2023-11-28T13:03:12.463 INFO:teuthology.orchestra.run.smithi006.stderr: removed all blocklist entries 2023-11-28T13:03:12.477 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T13:03:12.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.607+0000 7fad4c446700 1 Processor -- start 2023-11-28T13:03:12.609 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.607+0000 7fad4c446700 1 -- start start 2023-11-28T13:03:12.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.608+0000 7fad4c446700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415c560 0x7fad4406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:12.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.608+0000 7fad4c446700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad440b30b0 con 0x7fad440ceec0 2023-11-28T13:03:12.612 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.608+0000 7fad4c446700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4406c020 con 0x7fad440b2660 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.608+0000 7fad4c446700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad440d09c0 con 0x7fad4415c560 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.608+0000 7fad4a9e3700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fad440ceec0 0x7fad4406c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49602/0 (socket says 172.21.15.6:49602) 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.608+0000 7fad4a9e3700 1 -- 172.21.15.6:0/3833444578 learned_addr learned my addr 172.21.15.6:0/3833444578 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad4a1e2700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415c560 0x7fad4406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264908556 0 0) 0x7fad440b30b0 con 0x7fad440ceec0 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad300035b0 con 0x7fad440ceec0 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1065047146 0 0) 0x7fad300035b0 con 0x7fad440ceec0 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415c560 msgr2=0x7fad4406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415c560 0x7fad4406f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 >> v1:172.21.15.61:6789/0 conn(0x7fad440b2660 legacy=0x7fad440ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:12.613 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4406fdf0 con 0x7fad440ceec0 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fad3c0019f0 con 0x7fad440ceec0 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 >> v1:172.21.15.6:6789/0 conn(0x7fad440ceec0 legacy=0x7fad4406c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad30003a30 0x7fad30003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad30004420 0x7fad300048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415c560 0x7fad300051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad440d09c0 con 0x7fad30003a30 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3000baf0 con 0x7fad30004420 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3000bcb0 con 0x7fad4415c560 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad4a1e2700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad30003a30 0x7fad30003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad499e1700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad30004420 0x7fad300048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:12.614 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad4a9e3700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415c560 0x7fad300051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad4a1e2700 1 -- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415c560 msgr2=0x7fad300051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad4a1e2700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415c560 0x7fad300051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad4a1e2700 1 -- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad30004420 msgr2=0x7fad300048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.609+0000 7fad4a1e2700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad30004420 0x7fad300048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad4a1e2700 1 -- 172.21.15.6:0/3833444578 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad300035b0 con 0x7fad30003a30 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad4a9e3700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415c560 0x7fad300051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad4a1e2700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad30003a30 0x7fad30003e00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fad34009b30 tx=0x7fad30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d164d1ad9e73cbed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3400e6c0 con 0x7fad30003a30 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3400e820 con 0x7fad30003a30 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad491e0700 1 -- 172.21.15.6:0/3833444578 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3400eb90 con 0x7fad30003a30 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad4c446700 1 -- 172.21.15.6:0/3833444578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad30003a30 msgr2=0x7fad30003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:12.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad4c446700 1 --2- 172.21.15.6:0/3833444578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad30003a30 0x7fad30003e00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fad34009b30 tx=0x7fad30005880 comp rx=0 tx=0).stop 2023-11-28T13:03:12.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad4a9e3700 1 -- 172.21.15.6:0/3833444578 reap_dead start 2023-11-28T13:03:12.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad4c446700 1 -- 172.21.15.6:0/3833444578 shutdown_connections 2023-11-28T13:03:12.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.610+0000 7fad4c446700 1 -- 172.21.15.6:0/3833444578 >> 172.21.15.6:0/3833444578 conn(0x7fad440cd660 msgr2=0x7fad440cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:12.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.611+0000 7fad4c446700 1 -- 172.21.15.6:0/3833444578 shutdown_connections 2023-11-28T13:03:12.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.611+0000 7fad4c446700 1 -- 172.21.15.6:0/3833444578 wait complete. 2023-11-28T13:03:12.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.612+0000 7fad4c446700 1 Processor -- start 2023-11-28T13:03:12.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.612+0000 7fad4c446700 1 -- start start 2023-11-28T13:03:12.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4c446700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad30003a30 0x7fad441f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:12.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4c446700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415df80 0x7fad440af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4c446700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad4415e3f0 0x7fad440b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4c446700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad441f1f20 con 0x7fad4415e3f0 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4c446700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad440ac550 con 0x7fad30003a30 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4c446700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad440ac830 con 0x7fad4415df80 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a9e3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad4415e3f0 0x7fad440b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a9e3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad4415e3f0 0x7fad440b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34062/0 (socket says 172.21.15.6:34062) 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a9e3700 1 -- 172.21.15.6:0/2978147569 learned_addr learned my addr 172.21.15.6:0/2978147569 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a1e2700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad30003a30 0x7fad441f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad499e1700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415df80 0x7fad440af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a9e3700 1 -- 172.21.15.6:0/2978147569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415df80 msgr2=0x7fad440af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:12.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a9e3700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415df80 0x7fad440af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a9e3700 1 -- 172.21.15.6:0/2978147569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad30003a30 msgr2=0x7fad441f19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:12.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a9e3700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad30003a30 0x7fad441f19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a9e3700 1 -- 172.21.15.6:0/2978147569 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad440b2dd0 con 0x7fad4415e3f0 2023-11-28T13:03:12.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a1e2700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad30003a30 0x7fad441f19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:12.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4a9e3700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad4415e3f0 0x7fad440b0640 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fad3c001330 tx=0x7fad3c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:12.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad3affd700 1 -- 172.21.15.6:0/2978147569 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00edd0 con 0x7fad4415e3f0 2023-11-28T13:03:12.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad3affd700 1 -- 172.21.15.6:0/2978147569 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c013420 con 0x7fad4415e3f0 2023-11-28T13:03:12.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4c446700 1 -- 172.21.15.6:0/2978147569 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad440b2af0 con 0x7fad4415e3f0 2023-11-28T13:03:12.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad3affd700 1 -- 172.21.15.6:0/2978147569 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c013810 con 0x7fad4415e3f0 2023-11-28T13:03:12.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.613+0000 7fad4c446700 1 -- 172.21.15.6:0/2978147569 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4406bcb0 con 0x7fad4415e3f0 2023-11-28T13:03:12.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.614+0000 7fad4c446700 1 -- 172.21.15.6:0/2978147569 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad440675a0 con 0x7fad4415e3f0 2023-11-28T13:03:12.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.615+0000 7fad3affd700 1 -- 172.21.15.6:0/2978147569 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad3c013970 con 0x7fad4415e3f0 2023-11-28T13:03:12.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.615+0000 7fad3affd700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fad280682f0 0x7fad2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:12.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.615+0000 7fad3affd700 1 -- 172.21.15.6:0/2978147569 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fad3c0861e0 con 0x7fad4415e3f0 2023-11-28T13:03:12.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.615+0000 7fad4a1e2700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fad280682f0 0x7fad2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:12.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.616+0000 7fad4a1e2700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fad280682f0 0x7fad2806a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fad340090b0 tx=0x7fad34008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:12.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.620+0000 7fad3affd700 1 -- 172.21.15.6:0/2978147569 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad3c052c50 con 0x7fad4415e3f0 2023-11-28T13:03:12.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.892+0000 7fad4c446700 1 -- 172.21.15.6:0/2978147569 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fad440b6b60 con 0x7fad4415e3f0 2023-11-28T13:03:12.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.893+0000 7fad3affd700 1 -- 172.21.15.6:0/2978147569 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fad3c056270 con 0x7fad4415e3f0 2023-11-28T13:03:12.894 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:12.894 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "auth_dump": [ 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.a", 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVl8nvUCxAA8XF3IdK1G4diiM0tQ5npVA==", 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.895 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.b", 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlN9+EGBAA8WROC+gNBWJbQmf9ch69iA==", 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.c", 2023-11-28T13:03:12.896 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVl/bQ5ExAAerGXuTwAj5uC6cL7ScyCZA==", 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mds.d", 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlo58FIBAAMZ4QjC/4ODXdO7T8j7SEYA==", 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.897 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.0", 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA15GVlma1kARAA50lI2mIZlVoGT0ks8qcvzw==", 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:03:12.898 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.1", 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA45GVl4YRiARAAZ7Qktx0lxuvP0H4ceAeVig==", 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.899 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.2", 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA75GVlqfAkARAAFyiIXHxyw6s3bPBB2+KINA==", 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.3", 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQA+5GVlkjURARAANkhMOAGKdlDfEF9L29hxgA==", 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:03:12.900 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.4", 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBE5GVlNMQ9BxAA/gFfGDxTZ9TNAMQcCV2k9w==", 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.901 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.5", 2023-11-28T13:03:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBH5GVl91Q8BxAAlVlRbZqylgmq9LkQp/O6jA==", 2023-11-28T13:03:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:03:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:03:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.902 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.6", 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBK5GVlDoTjBhAAOXrgM8Q5MfN3I3aMZgjVvw==", 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "osd.7", 2023-11-28T13:03:12.903 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQBN5GVlFzINBxAAN6E1A938lHTLYA1GamwdwQ==", 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow profile osd", 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile osd", 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.0", 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlRYR3JBAADaTghk6Ix3IQVes6gTE56A==", 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:03:12.904 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow r", 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.1", 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVloippKxAAsZK5iYerb15VlfiZ4svbdg==", 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow r", 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow rw", 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rwx" 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.905 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.2", 2023-11-28T13:03:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlDlnhJxAAQG5yCtKJUjh6KGJ3Ennxjg==", 2023-11-28T13:03:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:03:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow r", 2023-11-28T13:03:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow rw", 2023-11-28T13:03:12.906 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rwx" 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.3", 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlbJU6MRAAd4WTJLkmaaL+ATXGlIH6ig==", 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow", 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow r", 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow rw", 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow rwx" 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.907 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "client.admin", 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAw5GVlXWKtKRAAbAVVrygFCntIL9ljdMCXbg==", 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow *", 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "mgr": "allow *", 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow *", 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mgr.x", 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVlIRViCBAADpzuU8y4HYepMLT1WkF08g==", 2023-11-28T13:03:12.908 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow *", 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile mgr", 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "entity": "mgr.y", 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "key": "AQAx5GVl4uVeBBAACyO9UoUP+RyJxFxE8rsFoQ==", 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "caps": { 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "mds": "allow *", 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "mon": "allow profile mgr", 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: "osd": "allow *" 2023-11-28T13:03:12.909 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.910 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:12.910 INFO:teuthology.orchestra.run.smithi006.stdout: ] 2023-11-28T13:03:12.910 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:03:12.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 -- 172.21.15.6:0/2978147569 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fad280682f0 msgr2=0x7fad2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:12.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fad280682f0 0x7fad2806a7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fad340090b0 tx=0x7fad34008030 comp rx=0 tx=0).stop 2023-11-28T13:03:12.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 -- 172.21.15.6:0/2978147569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad4415e3f0 msgr2=0x7fad440b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:12.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad4415e3f0 0x7fad440b0640 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fad3c001330 tx=0x7fad3c001ae0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 -- 172.21.15.6:0/2978147569 shutdown_connections 2023-11-28T13:03:12.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fad4415e3f0 0x7fad440b0640 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fad30003a30 0x7fad441f19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fad280682f0 0x7fad2806a7a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 --2- 172.21.15.6:0/2978147569 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fad4415df80 0x7fad440af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:12.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 -- 172.21.15.6:0/2978147569 >> 172.21.15.6:0/2978147569 conn(0x7fad440cd660 msgr2=0x7fad440ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:12.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 -- 172.21.15.6:0/2978147569 shutdown_connections 2023-11-28T13:03:12.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:12.895+0000 7fad4c446700 1 -- 172.21.15.6:0/2978147569 wait complete. 2023-11-28T13:03:12.913 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:03:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.036+0000 7ff1a6f33700 1 Processor -- start 2023-11-28T13:03:13.037 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.036+0000 7ff1a6f33700 1 -- start start 2023-11-28T13:03:13.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff1a6f33700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff1a006d9d0 0x7ff1a015b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff1a6f33700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff1a006bc30 0x7ff1a015e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff1a6f33700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006dda0 0x7ff1a0150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff1a6f33700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a015ee90 con 0x7ff1a006bc30 2023-11-28T13:03:13.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff1a6f33700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a006c3a0 con 0x7ff1a006d9d0 2023-11-28T13:03:13.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff1a6f33700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a006c0c0 con 0x7ff1a006dda0 2023-11-28T13:03:13.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff19ffff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff1a006bc30 0x7ff1a015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff19ffff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff1a006bc30 0x7ff1a015e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34068/0 (socket says 172.21.15.6:34068) 2023-11-28T13:03:13.040 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff19ffff700 1 -- 172.21.15.6:0/1815571969 learned_addr learned my addr 172.21.15.6:0/1815571969 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff1a54d0700 1 --2- 172.21.15.6:0/1815571969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006dda0 0x7ff1a0150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff1a4ccf700 1 --2- 172.21.15.6:0/1815571969 >> v2:172.21.15.61:3300/0 conn(0x7ff1a006d9d0 0x7ff1a015b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff19ffff700 1 -- 172.21.15.6:0/1815571969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006dda0 msgr2=0x7ff1a0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff19ffff700 1 --2- 172.21.15.6:0/1815571969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006dda0 0x7ff1a0150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff19ffff700 1 -- 172.21.15.6:0/1815571969 >> v2:172.21.15.61:3300/0 conn(0x7ff1a006d9d0 msgr2=0x7ff1a015b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff19ffff700 1 --2- 172.21.15.6:0/1815571969 >> v2:172.21.15.61:3300/0 conn(0x7ff1a006d9d0 0x7ff1a015b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.037+0000 7ff19ffff700 1 -- 172.21.15.6:0/1815571969 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a015ab60 con 0x7ff1a006bc30 2023-11-28T13:03:13.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff19ffff700 1 --2- 172.21.15.6:0/1815571969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006bc30 0x7ff1a015e950 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00cf1a0 tx=0x7ff18c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d28dcb8d6c2d80a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:13.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff19f7fe700 1 -- 172.21.15.6:0/1815571969 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c011830 con 0x7ff1a006bc30 2023-11-28T13:03:13.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff19f7fe700 1 -- 172.21.15.6:0/1815571969 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c011990 con 0x7ff1a006bc30 2023-11-28T13:03:13.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff19f7fe700 1 -- 172.21.15.6:0/1815571969 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c011d20 con 0x7ff1a006bc30 2023-11-28T13:03:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff1a6f33700 1 -- 172.21.15.6:0/1815571969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006bc30 msgr2=0x7ff1a015e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff1a6f33700 1 --2- 172.21.15.6:0/1815571969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006bc30 0x7ff1a015e950 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00cf1a0 tx=0x7ff18c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff1a6f33700 1 -- 172.21.15.6:0/1815571969 shutdown_connections 2023-11-28T13:03:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff1a6f33700 1 --2- 172.21.15.6:0/1815571969 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006dda0 0x7ff1a0150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff1a6f33700 1 --2- 172.21.15.6:0/1815571969 >> v2:172.21.15.61:3300/0 conn(0x7ff1a006d9d0 0x7ff1a015b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff1a6f33700 1 --2- 172.21.15.6:0/1815571969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006bc30 0x7ff1a015e950 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff1a6f33700 1 -- 172.21.15.6:0/1815571969 >> 172.21.15.6:0/1815571969 conn(0x7ff1a00b0dd0 msgr2=0x7ff1a00b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff1a6f33700 1 -- 172.21.15.6:0/1815571969 shutdown_connections 2023-11-28T13:03:13.044 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.038+0000 7ff1a6f33700 1 -- 172.21.15.6:0/1815571969 wait complete. 2023-11-28T13:03:13.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.039+0000 7ff1a6f33700 1 Processor -- start 2023-11-28T13:03:13.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.039+0000 7ff1a6f33700 1 -- start start 2023-11-28T13:03:13.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.040+0000 7ff1a6f33700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff1a006bc30 0x7ff1a0151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.040+0000 7ff1a6f33700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006d9d0 0x7ff1a00b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.040+0000 7ff1a6f33700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006dda0 0x7ff1a01562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.040+0000 7ff1a6f33700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a0155210 con 0x7ff1a006dda0 2023-11-28T13:03:13.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.040+0000 7ff1a6f33700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01554f0 con 0x7ff1a006bc30 2023-11-28T13:03:13.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.040+0000 7ff1a6f33700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01557d0 con 0x7ff1a006d9d0 2023-11-28T13:03:13.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.040+0000 7ff1a54d0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006dda0 0x7ff1a01562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.040+0000 7ff1a54d0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006dda0 0x7ff1a01562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34076/0 (socket says 172.21.15.6:34076) 2023-11-28T13:03:13.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.040+0000 7ff1a54d0700 1 -- 172.21.15.6:0/104072764 learned_addr learned my addr 172.21.15.6:0/104072764 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:13.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff19ffff700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006d9d0 0x7ff1a00b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff1a4ccf700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff1a006bc30 0x7ff1a0151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff1a54d0700 1 -- 172.21.15.6:0/104072764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006d9d0 msgr2=0x7ff1a00b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff1a54d0700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006d9d0 0x7ff1a00b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff1a54d0700 1 -- 172.21.15.6:0/104072764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff1a006bc30 msgr2=0x7ff1a0151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff1a54d0700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff1a006bc30 0x7ff1a0151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff1a54d0700 1 -- 172.21.15.6:0/104072764 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c009550 con 0x7ff1a006dda0 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff19ffff700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006d9d0 0x7ff1a00b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff1a54d0700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006dda0 0x7ff1a01562b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff19800efc0 tx=0x7ff19800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff19d7fa700 1 -- 172.21.15.6:0/104072764 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198009ba0 con 0x7ff1a006dda0 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff19d7fa700 1 -- 172.21.15.6:0/104072764 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19800f040 con 0x7ff1a006dda0 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff1a6f33700 1 -- 172.21.15.6:0/104072764 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a015b120 con 0x7ff1a006dda0 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff19d7fa700 1 -- 172.21.15.6:0/104072764 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198014480 con 0x7ff1a006dda0 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.041+0000 7ff1a6f33700 1 -- 172.21.15.6:0/104072764 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a015ae40 con 0x7ff1a006dda0 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.042+0000 7ff1a6f33700 1 -- 172.21.15.6:0/104072764 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a006fac0 con 0x7ff1a006dda0 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.042+0000 7ff19d7fa700 1 -- 172.21.15.6:0/104072764 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff1980145e0 con 0x7ff1a006dda0 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.043+0000 7ff19d7fa700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1840682f0 0x7ff18406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.043+0000 7ff19d7fa700 1 -- 172.21.15.6:0/104072764 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ff198086410 con 0x7ff1a006dda0 2023-11-28T13:03:13.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.043+0000 7ff1a4ccf700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1840682f0 0x7ff18406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.043+0000 7ff1a4ccf700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1840682f0 0x7ff18406a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff194009a30 tx=0x7ff194008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:13.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.047+0000 7ff19d7fa700 1 -- 172.21.15.6:0/104072764 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff198052e80 con 0x7ff1a006dda0 2023-11-28T13:03:13.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.242+0000 7ff1a6f33700 1 -- 172.21.15.6:0/104072764 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1a00675a0 con 0x7ff1a006dda0 2023-11-28T13:03:13.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.243+0000 7ff19d7fa700 1 -- 172.21.15.6:0/104072764 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+14318 (secure 0 0 0) 0x7ff1980564a0 con 0x7ff1a006dda0 2023-11-28T13:03:13.244 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:13.245 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":62,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:03:12.452162+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 -- 172.21.15.6:0/104072764 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1840682f0 msgr2=0x7ff18406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1840682f0 0x7ff18406a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff194009a30 tx=0x7ff194008040 comp rx=0 tx=0).stop 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 -- 172.21.15.6:0/104072764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006dda0 msgr2=0x7ff1a01562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006dda0 0x7ff1a01562b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff19800efc0 tx=0x7ff19800c3a0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 -- 172.21.15.6:0/104072764 shutdown_connections 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff1a006dda0 0x7ff1a01562b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff1a006bc30 0x7ff1a0151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff1840682f0 0x7ff18406a7a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 --2- 172.21.15.6:0/104072764 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff1a006d9d0 0x7ff1a00b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 -- 172.21.15.6:0/104072764 >> 172.21.15.6:0/104072764 conn(0x7ff1a00b0dd0 msgr2=0x7ff1a00ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:13.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 -- 172.21.15.6:0/104072764 shutdown_connections 2023-11-28T13:03:13.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.245+0000 7ff1a6f33700 1 -- 172.21.15.6:0/104072764 wait complete. 2023-11-28T13:03:13.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:03:13.259 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:03:13.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.386+0000 7f5229c10700 1 Processor -- start 2023-11-28T13:03:13.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.386+0000 7f5229c10700 1 -- start start 2023-11-28T13:03:13.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f5229c10700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5224165670 0x7f5224167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f5229c10700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52240ce550 con 0x7f52240cf5f0 2023-11-28T13:03:13.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f5229c10700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52240ce830 con 0x7f52240b1630 2023-11-28T13:03:13.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f5229c10700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52240ceb10 con 0x7f5224165670 2023-11-28T13:03:13.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f5222ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f52240cf5f0 0x7f52240caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49634/0 (socket says 172.21.15.6:49634) 2023-11-28T13:03:13.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f5222ffd700 1 -- 172.21.15.6:0/1109271260 learned_addr learned my addr 172.21.15.6:0/1109271260 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:13.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f5223fff700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5224165670 0x7f5224167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567890278 0 0) 0x7f52240ce550 con 0x7f52240cf5f0 2023-11-28T13:03:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52100035b0 con 0x7f52240cf5f0 2023-11-28T13:03:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640962704 0 0) 0x7f52240ce830 con 0x7f52240b1630 2023-11-28T13:03:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52240ce550 con 0x7f52240b1630 2023-11-28T13:03:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1345707720 0 0) 0x7f52100035b0 con 0x7f52240cf5f0 2023-11-28T13:03:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5224165670 msgr2=0x7f5224167a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:13.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f52227fc700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5224165670 0x7f5224167a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.387+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 >> v1:172.21.15.61:6789/0 conn(0x7f52240b1630 legacy=0x7f52240b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240cedf0 con 0x7f52240cf5f0 2023-11-28T13:03:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f5214001670 con 0x7f52240cf5f0 2023-11-28T13:03:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 >> v1:172.21.15.6:6789/0 conn(0x7f52240cf5f0 legacy=0x7f52240caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5210003bc0 0x7f5210003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5224165670 0x7f5210004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5210004d90 0x7f5210005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240ceb10 con 0x7f5210004d90 2023-11-28T13:03:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521000baf0 con 0x7f5210003bc0 2023-11-28T13:03:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521000bcb0 con 0x7f5224165670 2023-11-28T13:03:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f5223fff700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5210004d90 0x7f5210005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f5222ffd700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5224165670 0x7f5210004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52237fe700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5210003bc0 0x7f5210003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f5223fff700 1 -- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5224165670 msgr2=0x7f5210004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f5223fff700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5224165670 0x7f5210004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f5223fff700 1 -- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5210003bc0 msgr2=0x7f5210003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f5223fff700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5210003bc0 0x7f5210003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f5223fff700 1 -- 172.21.15.6:0/1109271260 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52100035b0 con 0x7f5210004d90 2023-11-28T13:03:13.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52237fe700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5210003bc0 0x7f5210003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f5223fff700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5210004d90 0x7f5210005200 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f522406eb30 tx=0x7f5210005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8158da975b8cc4de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c00eba0 con 0x7f5210004d90 2023-11-28T13:03:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c00f040 con 0x7f5210004d90 2023-11-28T13:03:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.388+0000 7f52227fc700 1 -- 172.21.15.6:0/1109271260 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c014480 con 0x7f5210004d90 2023-11-28T13:03:13.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.389+0000 7f5229c10700 1 -- 172.21.15.6:0/1109271260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5210004d90 msgr2=0x7f5210005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.389+0000 7f5229c10700 1 --2- 172.21.15.6:0/1109271260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5210004d90 0x7f5210005200 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f522406eb30 tx=0x7f5210005880 comp rx=0 tx=0).stop 2023-11-28T13:03:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.389+0000 7f5223fff700 1 -- 172.21.15.6:0/1109271260 reap_dead start 2023-11-28T13:03:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.389+0000 7f5229c10700 1 -- 172.21.15.6:0/1109271260 shutdown_connections 2023-11-28T13:03:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.389+0000 7f5229c10700 1 -- 172.21.15.6:0/1109271260 >> 172.21.15.6:0/1109271260 conn(0x7f52240cc630 msgr2=0x7f52240cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.389+0000 7f5229c10700 1 -- 172.21.15.6:0/1109271260 shutdown_connections 2023-11-28T13:03:13.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.389+0000 7f5229c10700 1 -- 172.21.15.6:0/1109271260 wait complete. 2023-11-28T13:03:13.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.390+0000 7f5229c10700 1 Processor -- start 2023-11-28T13:03:13.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.390+0000 7f5229c10700 1 -- start start 2023-11-28T13:03:13.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5229c10700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5210004d90 0x7f52240b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5229c10700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52240acd70 0x7f52240ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5229c10700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f522415afd0 0x7f5224157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5229c10700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240adef0 con 0x7f52240acd70 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5229c10700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240ad2e0 con 0x7f5210004d90 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5229c10700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52240ad5c0 con 0x7f522415afd0 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5222ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52240acd70 0x7f52240ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5222ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52240acd70 0x7f52240ae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34094/0 (socket says 172.21.15.6:34094) 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5222ffd700 1 -- 172.21.15.6:0/2749549035 learned_addr learned my addr 172.21.15.6:0/2749549035 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f52237fe700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5210004d90 0x7f52240b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5223fff700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f522415afd0 0x7f5224157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5222ffd700 1 -- 172.21.15.6:0/2749549035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f522415afd0 msgr2=0x7f5224157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5222ffd700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f522415afd0 0x7f5224157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5222ffd700 1 -- 172.21.15.6:0/2749549035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5210004d90 msgr2=0x7f52240b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5222ffd700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5210004d90 0x7f52240b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.391+0000 7f5222ffd700 1 -- 172.21.15.6:0/2749549035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240cb5b0 con 0x7f52240acd70 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.392+0000 7f5223fff700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f522415afd0 0x7f5224157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.392+0000 7f5222ffd700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52240acd70 0x7f52240ae380 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5214007290 tx=0x7f52140075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.392+0000 7f521bfff700 1 -- 172.21.15.6:0/2749549035 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214011580 con 0x7f52240acd70 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.392+0000 7f521bfff700 1 -- 172.21.15.6:0/2749549035 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5214011700 con 0x7f52240acd70 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.392+0000 7f5229c10700 1 -- 172.21.15.6:0/2749549035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52240cf0d0 con 0x7f52240acd70 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.392+0000 7f521bfff700 1 -- 172.21.15.6:0/2749549035 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214011ab0 con 0x7f52240acd70 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.392+0000 7f5229c10700 1 -- 172.21.15.6:0/2749549035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52240ce140 con 0x7f52240acd70 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.393+0000 7f5229c10700 1 -- 172.21.15.6:0/2749549035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52240739f0 con 0x7f52240acd70 2023-11-28T13:03:13.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.393+0000 7f521bfff700 1 -- 172.21.15.6:0/2749549035 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5214011c10 con 0x7f52240acd70 2023-11-28T13:03:13.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.394+0000 7f521bfff700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52040682f0 0x7f520406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.394+0000 7f52237fe700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52040682f0 0x7f520406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.394+0000 7f521bfff700 1 -- 172.21.15.6:0/2749549035 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5214085b80 con 0x7f52240acd70 2023-11-28T13:03:13.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.394+0000 7f52237fe700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52040682f0 0x7f520406a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f520c007fd0 tx=0x7f520c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:13.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.398+0000 7f521bfff700 1 -- 172.21.15.6:0/2749549035 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f52140525f0 con 0x7f52240acd70 2023-11-28T13:03:13.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.592+0000 7f5229c10700 1 -- 172.21.15.6:0/2749549035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f52240ae030 con 0x7f52240acd70 2023-11-28T13:03:13.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.593+0000 7f521bfff700 1 -- 172.21.15.6:0/2749549035 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5214055c10 con 0x7f52240acd70 2023-11-28T13:03:13.594 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:03:13.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 -- 172.21.15.6:0/2749549035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52040682f0 msgr2=0x7f520406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52040682f0 0x7f520406a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f520c007fd0 tx=0x7f520c009040 comp rx=0 tx=0).stop 2023-11-28T13:03:13.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 -- 172.21.15.6:0/2749549035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52240acd70 msgr2=0x7f52240ae380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52240acd70 0x7f52240ae380 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5214007290 tx=0x7f52140075a0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 -- 172.21.15.6:0/2749549035 shutdown_connections 2023-11-28T13:03:13.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f52040682f0 0x7f520406a7a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f522415afd0 0x7f5224157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5210004d90 0x7f52240b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 --2- 172.21.15.6:0/2749549035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f52240acd70 0x7f52240ae380 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 -- 172.21.15.6:0/2749549035 >> 172.21.15.6:0/2749549035 conn(0x7f52240cc630 msgr2=0x7f522406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:13.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 -- 172.21.15.6:0/2749549035 shutdown_connections 2023-11-28T13:03:13.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.595+0000 7f5229c10700 1 -- 172.21.15.6:0/2749549035 wait complete. 2023-11-28T13:03:13.608 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:03:13.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.735+0000 7f9f7a889700 1 Processor -- start 2023-11-28T13:03:13.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.735+0000 7f9f7a889700 1 -- start start 2023-11-28T13:03:13.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f7a889700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9f740b1630 0x7f9f740cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f7a889700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9f740b1a00 0x7f9f740caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f7a889700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f740ce550 con 0x7f9f740cb3e0 2023-11-28T13:03:13.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f7a889700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f740ce830 con 0x7f9f740b1a00 2023-11-28T13:03:13.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f7a889700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f740ceb10 con 0x7f9f740b1630 2023-11-28T13:03:13.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f737fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9f740b1a00 0x7f9f740caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f78e26700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f9f740cb3e0 0x7f9f74160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49656/0 (socket says 172.21.15.6:49656) 2023-11-28T13:03:13.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f78e26700 1 -- 172.21.15.6:0/3341212259 learned_addr learned my addr 172.21.15.6:0/3341212259 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f73fff700 1 --2- 172.21.15.6:0/3341212259 >> v2:172.21.15.61:3300/0 conn(0x7f9f740b1630 0x7f9f740cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f72ffd700 1 -- 172.21.15.6:0/3341212259 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469701043 0 0) 0x7f9f740ce550 con 0x7f9f740cb3e0 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f72ffd700 1 -- 172.21.15.6:0/3341212259 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f600035b0 con 0x7f9f740cb3e0 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f737fe700 1 -- 172.21.15.6:0/3341212259 >> v2:172.21.15.61:3300/0 conn(0x7f9f740b1630 msgr2=0x7f9f740cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f737fe700 1 --2- 172.21.15.6:0/3341212259 >> v2:172.21.15.61:3300/0 conn(0x7f9f740b1630 0x7f9f740cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f737fe700 1 -- 172.21.15.6:0/3341212259 >> v1:172.21.15.6:6789/0 conn(0x7f9f740cb3e0 legacy=0x7f9f74160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.736+0000 7f9f737fe700 1 -- 172.21.15.6:0/3341212259 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f740cedf0 con 0x7f9f740b1a00 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f737fe700 1 --2- 172.21.15.6:0/3341212259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1a00 0x7f9f740caea0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009840 tx=0x7f9f5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111dc82bb00a7248 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f72ffd700 1 -- 172.21.15.6:0/3341212259 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c011830 con 0x7f9f740b1a00 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f72ffd700 1 -- 172.21.15.6:0/3341212259 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c011990 con 0x7f9f740b1a00 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f72ffd700 1 -- 172.21.15.6:0/3341212259 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c011d20 con 0x7f9f740b1a00 2023-11-28T13:03:13.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f7a889700 1 -- 172.21.15.6:0/3341212259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1a00 msgr2=0x7f9f740caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f7a889700 1 --2- 172.21.15.6:0/3341212259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1a00 0x7f9f740caea0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009840 tx=0x7f9f5c009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:13.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f7a889700 1 -- 172.21.15.6:0/3341212259 shutdown_connections 2023-11-28T13:03:13.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f7a889700 1 --2- 172.21.15.6:0/3341212259 >> v2:172.21.15.61:3300/0 conn(0x7f9f740b1630 0x7f9f740cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f7a889700 1 --2- 172.21.15.6:0/3341212259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1a00 0x7f9f740caea0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f7a889700 1 -- 172.21.15.6:0/3341212259 >> 172.21.15.6:0/3341212259 conn(0x7f9f740cc630 msgr2=0x7f9f740cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:13.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f7a889700 1 -- 172.21.15.6:0/3341212259 shutdown_connections 2023-11-28T13:03:13.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.737+0000 7f9f7a889700 1 -- 172.21.15.6:0/3341212259 wait complete. 2023-11-28T13:03:13.743 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.739+0000 7f9f7a889700 1 Processor -- start 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.739+0000 7f9f7a889700 1 -- start start 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.739+0000 7f9f7a889700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1630 0x7f9f741f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.739+0000 7f9f7a889700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f740b1a00 0x7f9f741e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.739+0000 7f9f7a889700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9f740cb3e0 0x7f9f741e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.739+0000 7f9f7a889700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740b4710 con 0x7f9f740b1630 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.739+0000 7f9f7a889700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740b3a90 con 0x7f9f740b1a00 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.739+0000 7f9f7a889700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740b3d70 con 0x7f9f740cb3e0 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f73fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1630 0x7f9f741f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f73fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1630 0x7f9f741f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34106/0 (socket says 172.21.15.6:34106) 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f73fff700 1 -- 172.21.15.6:0/894267909 learned_addr learned my addr 172.21.15.6:0/894267909 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f737fe700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f740b1a00 0x7f9f741e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f78e26700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9f740cb3e0 0x7f9f741e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f73fff700 1 -- 172.21.15.6:0/894267909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9f740cb3e0 msgr2=0x7f9f741e19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f73fff700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9f740cb3e0 0x7f9f741e19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f73fff700 1 -- 172.21.15.6:0/894267909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f740b1a00 msgr2=0x7f9f741e70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f73fff700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f740b1a00 0x7f9f741e70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f73fff700 1 -- 172.21.15.6:0/894267909 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0094a0 con 0x7f9f740b1630 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f737fe700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f740b1a00 0x7f9f741e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f73fff700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1630 0x7f9f741f2660 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400e870 tx=0x7f9f6400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f70ff9700 1 -- 172.21.15.6:0/894267909 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f64009590 con 0x7f9f740b1630 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f70ff9700 1 -- 172.21.15.6:0/894267909 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6400ee70 con 0x7f9f740b1630 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f7a889700 1 -- 172.21.15.6:0/894267909 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f740b1ea0 con 0x7f9f740b1630 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f70ff9700 1 -- 172.21.15.6:0/894267909 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6400ca50 con 0x7f9f740b1630 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.740+0000 7f9f7a889700 1 -- 172.21.15.6:0/894267909 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f740cf0d0 con 0x7f9f740b1630 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.741+0000 7f9f7a889700 1 -- 172.21.15.6:0/894267909 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f740ce140 con 0x7f9f740b1630 2023-11-28T13:03:13.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.742+0000 7f9f70ff9700 1 -- 172.21.15.6:0/894267909 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9f64019030 con 0x7f9f740b1630 2023-11-28T13:03:13.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.742+0000 7f9f70ff9700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9f540682f0 0x7f9f5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:13.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.742+0000 7f9f70ff9700 1 -- 172.21.15.6:0/894267909 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9f64085910 con 0x7f9f740b1630 2023-11-28T13:03:13.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.742+0000 7f9f737fe700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9f540682f0 0x7f9f5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:13.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.743+0000 7f9f737fe700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9f540682f0 0x7f9f5406a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9f741e2650 tx=0x7f9f5c010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:13.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.746+0000 7f9f70ff9700 1 -- 172.21.15.6:0/894267909 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9f64052380 con 0x7f9f740b1630 2023-11-28T13:03:13.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.938+0000 7f9f7a889700 1 -- 172.21.15.6:0/894267909 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f740b4980 con 0x7f9f740b1630 2023-11-28T13:03:13.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.938+0000 7f9f70ff9700 1 -- 172.21.15.6:0/894267909 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f640559a0 con 0x7f9f740b1630 2023-11-28T13:03:13.939 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:03:13.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 -- 172.21.15.6:0/894267909 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9f540682f0 msgr2=0x7f9f5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9f540682f0 0x7f9f5406a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9f741e2650 tx=0x7f9f5c010000 comp rx=0 tx=0).stop 2023-11-28T13:03:13.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 -- 172.21.15.6:0/894267909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1630 msgr2=0x7f9f741f2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:13.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1630 0x7f9f741f2660 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400e870 tx=0x7f9f6400ec30 comp rx=0 tx=0).stop 2023-11-28T13:03:13.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 -- 172.21.15.6:0/894267909 shutdown_connections 2023-11-28T13:03:13.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9f540682f0 0x7f9f5406a7a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9f740cb3e0 0x7f9f741e19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f740b1630 0x7f9f741f2660 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 --2- 172.21.15.6:0/894267909 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9f740b1a00 0x7f9f741e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:13.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 -- 172.21.15.6:0/894267909 >> 172.21.15.6:0/894267909 conn(0x7f9f740cc630 msgr2=0x7f9f740b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:13.942 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 -- 172.21.15.6:0/894267909 shutdown_connections 2023-11-28T13:03:13.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:13.940+0000 7f9f7a889700 1 -- 172.21.15.6:0/894267909 wait complete. 2023-11-28T13:03:13.954 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T13:03:14.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.078+0000 7f97e5718700 1 Processor -- start 2023-11-28T13:03:14.079 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.078+0000 7f97e5718700 1 -- start start 2023-11-28T13:03:14.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.078+0000 7f97e5718700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f97e015c990 0x7f97e00cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.078+0000 7f97e5718700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e00ce340 con 0x7f97e00cc310 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.078+0000 7f97e5718700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e00cebe0 con 0x7f97e0161090 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.078+0000 7f97e5718700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e00cb3f0 con 0x7f97e015c990 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97deffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f97e015c990 0x7f97e00cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97df7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f97e00cc310 0x7f97e00cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49668/0 (socket says 172.21.15.6:49668) 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97deffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f97e015c990 0x7f97e00cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34114/0 (socket says 172.21.15.6:34114) 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97df7fe700 1 -- 172.21.15.6:0/3492384302 learned_addr learned my addr 172.21.15.6:0/3492384302 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97deffd700 1 -- 172.21.15.6:0/3492384302 >> v1:172.21.15.61:6789/0 conn(0x7f97e0161090 legacy=0x7f97e00ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97deffd700 1 -- 172.21.15.6:0/3492384302 >> v1:172.21.15.6:6789/0 conn(0x7f97e00cc310 legacy=0x7f97e00cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97ddffb700 1 -- 172.21.15.6:0/3492384302 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746477679 0 0) 0x7f97e00ce340 con 0x7f97e00cc310 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97deffd700 1 -- 172.21.15.6:0/3492384302 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e00ce590 con 0x7f97e015c990 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97deffd700 1 --2- 172.21.15.6:0/3492384302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e015c990 0x7f97e00cf070 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009840 tx=0x7f97d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9589287be95f8cc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97ddffb700 1 -- 172.21.15.6:0/3492384302 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0011830 con 0x7f97e015c990 2023-11-28T13:03:14.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97ddffb700 1 -- 172.21.15.6:0/3492384302 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d0011990 con 0x7f97e015c990 2023-11-28T13:03:14.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.079+0000 7f97ddffb700 1 -- 172.21.15.6:0/3492384302 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0011d20 con 0x7f97e015c990 2023-11-28T13:03:14.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.080+0000 7f97e5718700 1 -- 172.21.15.6:0/3492384302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e015c990 msgr2=0x7f97e00cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.080+0000 7f97e5718700 1 --2- 172.21.15.6:0/3492384302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e015c990 0x7f97e00cf070 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009840 tx=0x7f97d0009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:14.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.080+0000 7f97e5718700 1 -- 172.21.15.6:0/3492384302 shutdown_connections 2023-11-28T13:03:14.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.080+0000 7f97e5718700 1 --2- 172.21.15.6:0/3492384302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e015c990 0x7f97e00cf070 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.080+0000 7f97e5718700 1 -- 172.21.15.6:0/3492384302 >> 172.21.15.6:0/3492384302 conn(0x7f97e00b1650 msgr2=0x7f97e00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:14.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.080+0000 7f97e5718700 1 -- 172.21.15.6:0/3492384302 shutdown_connections 2023-11-28T13:03:14.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.080+0000 7f97e5718700 1 -- 172.21.15.6:0/3492384302 wait complete. 2023-11-28T13:03:14.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.082+0000 7f97e5718700 1 Processor -- start 2023-11-28T13:03:14.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.082+0000 7f97e5718700 1 -- start start 2023-11-28T13:03:14.085 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97e5718700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cc310 0x7f97e015b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97e5718700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e015c990 0x7f97e0160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97e5718700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e0161090 0x7f97e015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97e5718700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e015b730 con 0x7f97e0161090 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97e5718700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e01f3080 con 0x7f97e015c990 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97e5718700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e01f3360 con 0x7f97e00cc310 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97df7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e0161090 0x7f97e015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97df7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e0161090 0x7f97e015b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34122/0 (socket says 172.21.15.6:34122) 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97df7fe700 1 -- 172.21.15.6:0/3643729252 learned_addr learned my addr 172.21.15.6:0/3643729252 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97deffd700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cc310 0x7f97e015b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97de7fc700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e015c990 0x7f97e0160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.086 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97df7fe700 1 -- 172.21.15.6:0/3643729252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cc310 msgr2=0x7f97e015b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97df7fe700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cc310 0x7f97e015b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97df7fe700 1 -- 172.21.15.6:0/3643729252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e015c990 msgr2=0x7f97e0160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97df7fe700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e015c990 0x7f97e0160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97df7fe700 1 -- 172.21.15.6:0/3643729252 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d00094a0 con 0x7f97e0161090 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.083+0000 7f97df7fe700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e0161090 0x7f97e015b1f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f97d800cb60 tx=0x7f97d800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.084+0000 7f97c77fe700 1 -- 172.21.15.6:0/3643729252 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d800a9e0 con 0x7f97e0161090 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.084+0000 7f97c77fe700 1 -- 172.21.15.6:0/3643729252 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d800ab40 con 0x7f97e0161090 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.084+0000 7f97e5718700 1 -- 172.21.15.6:0/3643729252 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e00cb080 con 0x7f97e0161090 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.084+0000 7f97c77fe700 1 -- 172.21.15.6:0/3643729252 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d80070d0 con 0x7f97e0161090 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.084+0000 7f97e5718700 1 -- 172.21.15.6:0/3643729252 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e00ce870 con 0x7f97e0161090 2023-11-28T13:03:14.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.084+0000 7f97e5718700 1 -- 172.21.15.6:0/3643729252 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e00cf5b0 con 0x7f97e0161090 2023-11-28T13:03:14.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.085+0000 7f97c77fe700 1 -- 172.21.15.6:0/3643729252 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97d8010070 con 0x7f97e0161090 2023-11-28T13:03:14.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.085+0000 7f97c77fe700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 0x7f97cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.085+0000 7f97c77fe700 1 -- 172.21.15.6:0/3643729252 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f97d8085260 con 0x7f97e0161090 2023-11-28T13:03:14.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.086+0000 7f97deffd700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 0x7f97cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.086+0000 7f97deffd700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 0x7f97cc06a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f97d00090b0 tx=0x7f97d001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:14.090 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.090+0000 7f97c77fe700 1 -- 172.21.15.6:0/3643729252 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97d8051c70 con 0x7f97e0161090 2023-11-28T13:03:14.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.286+0000 7f97e5718700 1 -- 172.21.15.6:0/3643729252 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f97e00ccee0 con 0x7f97e0161090 2023-11-28T13:03:14.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.286+0000 7f97c77fe700 1 -- 172.21.15.6:0/3643729252 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+126 (secure 0 0 0) 0x7f97d8018090 con 0x7f97e0161090 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout:[ 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 1, 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": "rbd" 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "poolnum": 2, 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: "poolname": ".mgr" 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:14.288 INFO:teuthology.orchestra.run.smithi006.stdout:] 2023-11-28T13:03:14.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 -- 172.21.15.6:0/3643729252 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 msgr2=0x7f97cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 0x7f97cc06a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f97d00090b0 tx=0x7f97d001c040 comp rx=0 tx=0).stop 2023-11-28T13:03:14.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 -- 172.21.15.6:0/3643729252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e0161090 msgr2=0x7f97e015b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e0161090 0x7f97e015b1f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f97d800cb60 tx=0x7f97d800ce70 comp rx=0 tx=0).stop 2023-11-28T13:03:14.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 -- 172.21.15.6:0/3643729252 shutdown_connections 2023-11-28T13:03:14.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97e0161090 0x7f97e015b1f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97cc0682f0 0x7f97cc06a7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97e00cc310 0x7f97e015b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 --2- 172.21.15.6:0/3643729252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e015c990 0x7f97e0160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 -- 172.21.15.6:0/3643729252 >> 172.21.15.6:0/3643729252 conn(0x7f97e00b1650 msgr2=0x7f97e006b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:14.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 -- 172.21.15.6:0/3643729252 shutdown_connections 2023-11-28T13:03:14.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.288+0000 7f97e5718700 1 -- 172.21.15.6:0/3643729252 wait complete. 2023-11-28T13:03:14.302 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T13:03:14.302 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T13:03:14.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.429+0000 7f28706ec700 1 Processor -- start 2023-11-28T13:03:14.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.429+0000 7f28706ec700 1 -- start start 2023-11-28T13:03:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.430+0000 7f28706ec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806d9d0 0x7f286815b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.430+0000 7f28706ec700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f286806bc30 0x7f286815e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.430+0000 7f28706ec700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286806c3a0 con 0x7f286806dda0 2023-11-28T13:03:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.430+0000 7f28706ec700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286815ee90 con 0x7f286806bc30 2023-11-28T13:03:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.430+0000 7f28706ec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286806c0c0 con 0x7f286806d9d0 2023-11-28T13:03:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.430+0000 7f286dc87700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f286806bc30 0x7f286815e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.430+0000 7f286e488700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806d9d0 0x7f286815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286ec89700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f286806dda0 0x7f2868150eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:39708/0 (socket says 172.21.15.6:39708) 2023-11-28T13:03:14.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286ec89700 1 -- 172.21.15.6:0/1531036561 learned_addr learned my addr 172.21.15.6:0/1531036561 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:14.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286e488700 1 -- 172.21.15.6:0/1531036561 >> v2:172.21.15.61:3300/0 conn(0x7f286806bc30 msgr2=0x7f286815e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:14.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286e488700 1 --2- 172.21.15.6:0/1531036561 >> v2:172.21.15.61:3300/0 conn(0x7f286806bc30 0x7f286815e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286e488700 1 -- 172.21.15.6:0/1531036561 >> v1:172.21.15.61:6789/0 conn(0x7f286806dda0 legacy=0x7f2868150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286e488700 1 -- 172.21.15.6:0/1531036561 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286815ab60 con 0x7f286806d9d0 2023-11-28T13:03:14.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286dc87700 1 --2- 172.21.15.6:0/1531036561 >> v2:172.21.15.61:3300/0 conn(0x7f286806bc30 0x7f286815e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:03:14.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286e488700 1 --2- 172.21.15.6:0/1531036561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806d9d0 0x7f286815b640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2858009840 tx=0x7f2858009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50d19bbe9f81fc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286d486700 1 -- 172.21.15.6:0/1531036561 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858011830 con 0x7f286806d9d0 2023-11-28T13:03:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286d486700 1 -- 172.21.15.6:0/1531036561 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2858011990 con 0x7f286806d9d0 2023-11-28T13:03:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.431+0000 7f286d486700 1 -- 172.21.15.6:0/1531036561 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858011d20 con 0x7f286806d9d0 2023-11-28T13:03:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.432+0000 7f28706ec700 1 -- 172.21.15.6:0/1531036561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806d9d0 msgr2=0x7f286815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.432+0000 7f28706ec700 1 --2- 172.21.15.6:0/1531036561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806d9d0 0x7f286815b640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2858009840 tx=0x7f2858009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.432+0000 7f28706ec700 1 -- 172.21.15.6:0/1531036561 shutdown_connections 2023-11-28T13:03:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.432+0000 7f28706ec700 1 --2- 172.21.15.6:0/1531036561 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806d9d0 0x7f286815b640 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.432+0000 7f28706ec700 1 --2- 172.21.15.6:0/1531036561 >> v2:172.21.15.61:3300/0 conn(0x7f286806bc30 0x7f286815e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.432+0000 7f28706ec700 1 -- 172.21.15.6:0/1531036561 >> 172.21.15.6:0/1531036561 conn(0x7f28680b0dd0 msgr2=0x7f28680b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.432+0000 7f28706ec700 1 -- 172.21.15.6:0/1531036561 shutdown_connections 2023-11-28T13:03:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.432+0000 7f28706ec700 1 -- 172.21.15.6:0/1531036561 wait complete. 2023-11-28T13:03:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.433+0000 7f28706ec700 1 Processor -- start 2023-11-28T13:03:14.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.433+0000 7f28706ec700 1 -- start start 2023-11-28T13:03:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f28706ec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806bc30 0x7f28681e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f28706ec700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f286806d9d0 0x7f28681e4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f28706ec700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f286806dda0 0x7f28681e5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f28706ec700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28680b41d0 con 0x7f286806dda0 2023-11-28T13:03:14.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f28706ec700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28680c3510 con 0x7f286806d9d0 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f28706ec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28680c37f0 con 0x7f286806bc30 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f286ec89700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f286806dda0 0x7f28681e5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f286ec89700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f286806dda0 0x7f28681e5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34134/0 (socket says 172.21.15.6:34134) 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f286ec89700 1 -- 172.21.15.6:0/962568463 learned_addr learned my addr 172.21.15.6:0/962568463 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f286e488700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806bc30 0x7f28681e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f286dc87700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f286806d9d0 0x7f28681e4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.434+0000 7f286ec89700 1 -- 172.21.15.6:0/962568463 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806bc30 msgr2=0x7f28681e3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.435+0000 7f286ec89700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806bc30 0x7f28681e3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.435+0000 7f286ec89700 1 -- 172.21.15.6:0/962568463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f286806d9d0 msgr2=0x7f28681e4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.435+0000 7f286ec89700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f286806d9d0 0x7f28681e4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.435+0000 7f286ec89700 1 -- 172.21.15.6:0/962568463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28640094a0 con 0x7f286806dda0 2023-11-28T13:03:14.441 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.435+0000 7f286ec89700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f286806dda0 0x7f28681e5bb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f286000ca80 tx=0x7f286000cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.435+0000 7f285effd700 1 -- 172.21.15.6:0/962568463 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286000a9e0 con 0x7f286806dda0 2023-11-28T13:03:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.435+0000 7f285effd700 1 -- 172.21.15.6:0/962568463 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286000ab40 con 0x7f286806dda0 2023-11-28T13:03:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.435+0000 7f28706ec700 1 -- 172.21.15.6:0/962568463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28580094a0 con 0x7f286806dda0 2023-11-28T13:03:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.435+0000 7f285effd700 1 -- 172.21.15.6:0/962568463 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860012400 con 0x7f286806dda0 2023-11-28T13:03:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.435+0000 7f28706ec700 1 -- 172.21.15.6:0/962568463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286815b120 con 0x7f286806dda0 2023-11-28T13:03:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.436+0000 7f28706ec700 1 -- 172.21.15.6:0/962568463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286815ae40 con 0x7f286806dda0 2023-11-28T13:03:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.436+0000 7f285effd700 1 -- 172.21.15.6:0/962568463 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2860019030 con 0x7f286806dda0 2023-11-28T13:03:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.437+0000 7f285effd700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f284c0682f0 0x7f284c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.437+0000 7f285effd700 1 -- 172.21.15.6:0/962568463 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f2860085320 con 0x7f286806dda0 2023-11-28T13:03:14.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.437+0000 7f286e488700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f284c0682f0 0x7f284c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.437+0000 7f286e488700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f284c0682f0 0x7f284c06a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2858000c00 tx=0x7f2858008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:14.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.441+0000 7f285effd700 1 -- 172.21.15.6:0/962568463 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f286004b5e0 con 0x7f286806dda0 2023-11-28T13:03:14.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:14.633+0000 7f28706ec700 1 -- 172.21.15.6:0/962568463 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f286806fac0 con 0x7f286806dda0 2023-11-28T13:03:15.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.105+0000 7f285effd700 1 -- 172.21.15.6:0/962568463 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2860051e40 con 0x7f286806dda0 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 -- 172.21.15.6:0/962568463 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f284c0682f0 msgr2=0x7f284c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f284c0682f0 0x7f284c06a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2858000c00 tx=0x7f2858008030 comp rx=0 tx=0).stop 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 -- 172.21.15.6:0/962568463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f286806dda0 msgr2=0x7f28681e5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f286806dda0 0x7f28681e5bb0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f286000ca80 tx=0x7f286000cd90 comp rx=0 tx=0).stop 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 -- 172.21.15.6:0/962568463 shutdown_connections 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f286806dda0 0x7f28681e5bb0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f284c0682f0 0x7f284c06a7a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f286806bc30 0x7f28681e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 --2- 172.21.15.6:0/962568463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f286806d9d0 0x7f28681e4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 -- 172.21.15.6:0/962568463 >> 172.21.15.6:0/962568463 conn(0x7f28680b0dd0 msgr2=0x7f286815ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:15.110 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 -- 172.21.15.6:0/962568463 shutdown_connections 2023-11-28T13:03:15.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.108+0000 7f28706ec700 1 -- 172.21.15.6:0/962568463 wait complete. 2023-11-28T13:03:15.112 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' created 2023-11-28T13:03:15.126 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T13:03:15.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.260+0000 7fd7999ae700 1 Processor -- start 2023-11-28T13:03:15.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.260+0000 7fd7999ae700 1 -- start start 2023-11-28T13:03:15.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7999ae700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd7940d2820 0x7fd7940b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7999ae700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd79406bf30 0x7fd79406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7999ae700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79406c870 con 0x7fd79415c640 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7999ae700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79415ca10 con 0x7fd7940d2820 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7999ae700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940d1470 con 0x7fd79406bf30 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7927fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd7940d2820 0x7fd7940b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7927fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd7940d2820 0x7fd7940b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34140/0 (socket says 172.21.15.6:34140) 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7927fc700 1 -- 172.21.15.6:0/3983125920 learned_addr learned my addr 172.21.15.6:0/3983125920 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7937fe700 1 --2- 172.21.15.6:0/3983125920 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd79406bf30 0x7fd79406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7927fc700 1 -- 172.21.15.6:0/3983125920 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd79406bf30 msgr2=0x7fd79406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7927fc700 1 --2- 172.21.15.6:0/3983125920 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd79406bf30 0x7fd79406c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7927fc700 1 -- 172.21.15.6:0/3983125920 >> v1:172.21.15.61:6789/0 conn(0x7fd79415c640 legacy=0x7fd7940cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:15.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7927fc700 1 -- 172.21.15.6:0/3983125920 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940cd4e0 con 0x7fd7940d2820 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd7927fc700 1 --2- 172.21.15.6:0/3983125920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7940d2820 0x7fd7940b5c40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd7880098f0 tx=0x7fd788009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e1ab4be5e52507 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.261+0000 7fd791ffb700 1 -- 172.21.15.6:0/3983125920 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788004230 con 0x7fd7940d2820 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.262+0000 7fd791ffb700 1 -- 172.21.15.6:0/3983125920 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd788004390 con 0x7fd7940d2820 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.262+0000 7fd791ffb700 1 -- 172.21.15.6:0/3983125920 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880036a0 con 0x7fd7940d2820 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.262+0000 7fd7999ae700 1 -- 172.21.15.6:0/3983125920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7940d2820 msgr2=0x7fd7940b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.262+0000 7fd7999ae700 1 --2- 172.21.15.6:0/3983125920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7940d2820 0x7fd7940b5c40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd7880098f0 tx=0x7fd788009c00 comp rx=0 tx=0).stop 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.262+0000 7fd7999ae700 1 -- 172.21.15.6:0/3983125920 shutdown_connections 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.262+0000 7fd7999ae700 1 --2- 172.21.15.6:0/3983125920 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd79406bf30 0x7fd79406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.262+0000 7fd7999ae700 1 --2- 172.21.15.6:0/3983125920 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7940d2820 0x7fd7940b5c40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.262+0000 7fd7999ae700 1 -- 172.21.15.6:0/3983125920 >> 172.21.15.6:0/3983125920 conn(0x7fd7940d2df0 msgr2=0x7fd7940d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.262+0000 7fd7999ae700 1 -- 172.21.15.6:0/3983125920 shutdown_connections 2023-11-28T13:03:15.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.262+0000 7fd7999ae700 1 -- 172.21.15.6:0/3983125920 wait complete. 2023-11-28T13:03:15.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.263+0000 7fd7999ae700 1 Processor -- start 2023-11-28T13:03:15.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.264+0000 7fd7999ae700 1 -- start start 2023-11-28T13:03:15.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.264+0000 7fd7999ae700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd79406bf30 0x7fd7940b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:15.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.264+0000 7fd7999ae700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7940d2820 0x7fd7940c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:15.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.264+0000 7fd7999ae700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd79415c640 0x7fd794144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:15.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.264+0000 7fd7999ae700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940cbcc0 con 0x7fd79415c640 2023-11-28T13:03:15.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.264+0000 7fd7999ae700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7941f1fb0 con 0x7fd79406bf30 2023-11-28T13:03:15.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.264+0000 7fd7999ae700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940cb0b0 con 0x7fd7940d2820 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.264+0000 7fd7937fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd79415c640 0x7fd794144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.264+0000 7fd7937fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd79415c640 0x7fd794144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34148/0 (socket says 172.21.15.6:34148) 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.264+0000 7fd7937fe700 1 -- 172.21.15.6:0/1442457005 learned_addr learned my addr 172.21.15.6:0/1442457005 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd7927fc700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7940d2820 0x7fd7940c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd792ffd700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd79406bf30 0x7fd7940b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd7937fe700 1 -- 172.21.15.6:0/1442457005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7940d2820 msgr2=0x7fd7940c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd7937fe700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7940d2820 0x7fd7940c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd7937fe700 1 -- 172.21.15.6:0/1442457005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd79406bf30 msgr2=0x7fd7940b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd7937fe700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd79406bf30 0x7fd7940b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd7937fe700 1 -- 172.21.15.6:0/1442457005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd788009550 con 0x7fd79415c640 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd792ffd700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd79406bf30 0x7fd7940b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:15.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd7937fe700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd79415c640 0x7fd794144bf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00cd70 tx=0x7fd78c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd77bfff700 1 -- 172.21.15.6:0/1442457005 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c003a20 con 0x7fd79415c640 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd77bfff700 1 -- 172.21.15.6:0/1442457005 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c003b80 con 0x7fd79415c640 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd7999ae700 1 -- 172.21.15.6:0/1442457005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7940d1710 con 0x7fd79415c640 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd77bfff700 1 -- 172.21.15.6:0/1442457005 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c004020 con 0x7fd79415c640 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.265+0000 7fd7999ae700 1 -- 172.21.15.6:0/1442457005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7940cd7c0 con 0x7fd79415c640 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.266+0000 7fd7999ae700 1 -- 172.21.15.6:0/1442457005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7940cdaa0 con 0x7fd79415c640 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.267+0000 7fd77bfff700 1 -- 172.21.15.6:0/1442457005 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd78c01c020 con 0x7fd79415c640 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.267+0000 7fd77bfff700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd7800682f0 0x7fd78006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.267+0000 7fd77bfff700 1 -- 172.21.15.6:0/1442457005 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fd78c012070 con 0x7fd79415c640 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.267+0000 7fd792ffd700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd7800682f0 0x7fd78006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:15.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.268+0000 7fd792ffd700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd7800682f0 0x7fd78006a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd7941f5650 tx=0x7fd784008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:15.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.272+0000 7fd77bfff700 1 -- 172.21.15.6:0/1442457005 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd78c052d30 con 0x7fd79415c640 2023-11-28T13:03:15.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:15.466+0000 7fd7999ae700 1 -- 172.21.15.6:0/1442457005 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd7940739f0 con 0x7fd79415c640 2023-11-28T13:03:16.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.110+0000 7fd77bfff700 1 -- 172.21.15.6:0/1442457005 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd78c056350 con 0x7fd79415c640 2023-11-28T13:03:16.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.113+0000 7fd7999ae700 1 -- 172.21.15.6:0/1442457005 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd7800682f0 msgr2=0x7fd78006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.113+0000 7fd7999ae700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd7800682f0 0x7fd78006a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd7941f5650 tx=0x7fd784008040 comp rx=0 tx=0).stop 2023-11-28T13:03:16.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.113+0000 7fd7999ae700 1 -- 172.21.15.6:0/1442457005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd79415c640 msgr2=0x7fd794144bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.113+0000 7fd7999ae700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd79415c640 0x7fd794144bf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00cd70 tx=0x7fd78c00a570 comp rx=0 tx=0).stop 2023-11-28T13:03:16.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.114+0000 7fd7999ae700 1 -- 172.21.15.6:0/1442457005 shutdown_connections 2023-11-28T13:03:16.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.114+0000 7fd7999ae700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd79415c640 0x7fd794144bf0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.114+0000 7fd7999ae700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd79406bf30 0x7fd7940b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.114+0000 7fd7999ae700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd7800682f0 0x7fd78006a7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.114+0000 7fd7999ae700 1 --2- 172.21.15.6:0/1442457005 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7940d2820 0x7fd7940c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.114+0000 7fd7999ae700 1 -- 172.21.15.6:0/1442457005 >> 172.21.15.6:0/1442457005 conn(0x7fd7940d2df0 msgr2=0x7fd7940b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:16.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.119+0000 7fd7999ae700 1 -- 172.21.15.6:0/1442457005 shutdown_connections 2023-11-28T13:03:16.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.121+0000 7fd7999ae700 1 -- 172.21.15.6:0/1442457005 wait complete. 2023-11-28T13:03:16.124 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' created 2023-11-28T13:03:16.143 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T13:03:16.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.273+0000 7fe48a6e0700 1 Processor -- start 2023-11-28T13:03:16.275 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.273+0000 7fe48a6e0700 1 -- start start 2023-11-28T13:03:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.274+0000 7fe48a6e0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe4840cb630 0x7fe4840b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.274+0000 7fe48a6e0700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe4840b2bc0 0x7fe4840ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.274+0000 7fe48a6e0700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4840b3000 con 0x7fe4840b62d0 2023-11-28T13:03:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.274+0000 7fe48a6e0700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4840cd2a0 con 0x7fe4840b2bc0 2023-11-28T13:03:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.274+0000 7fe48a6e0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4840d0400 con 0x7fe4840cb630 2023-11-28T13:03:16.277 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.274+0000 7fe488c7d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe4840b2bc0 0x7fe4840ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.274+0000 7fe483fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe4840b62d0 0x7fe48406f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49704/0 (socket says 172.21.15.6:49704) 2023-11-28T13:03:16.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.274+0000 7fe483fff700 1 -- 172.21.15.6:0/1964580473 learned_addr learned my addr 172.21.15.6:0/1964580473 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:16.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.274+0000 7fe4837fe700 1 --2- 172.21.15.6:0/1964580473 >> v2:172.21.15.61:3300/0 conn(0x7fe4840cb630 0x7fe4840b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.275+0000 7fe482ffd700 1 -- 172.21.15.6:0/1964580473 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922856009 0 0) 0x7fe4840b3000 con 0x7fe4840b62d0 2023-11-28T13:03:16.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.275+0000 7fe482ffd700 1 -- 172.21.15.6:0/1964580473 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4700035b0 con 0x7fe4840b62d0 2023-11-28T13:03:16.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.275+0000 7fe488c7d700 1 -- 172.21.15.6:0/1964580473 >> v2:172.21.15.61:3300/0 conn(0x7fe4840cb630 msgr2=0x7fe4840b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:16.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.275+0000 7fe488c7d700 1 --2- 172.21.15.6:0/1964580473 >> v2:172.21.15.61:3300/0 conn(0x7fe4840cb630 0x7fe4840b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.275+0000 7fe488c7d700 1 -- 172.21.15.6:0/1964580473 >> v1:172.21.15.6:6789/0 conn(0x7fe4840b62d0 legacy=0x7fe48406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.275+0000 7fe488c7d700 1 -- 172.21.15.6:0/1964580473 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4840d06e0 con 0x7fe4840b2bc0 2023-11-28T13:03:16.278 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.275+0000 7fe488c7d700 1 --2- 172.21.15.6:0/1964580473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840b2bc0 0x7fe4840ce7d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe4840cc8e0 tx=0x7fe47c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e8d34d076dce41 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.275+0000 7fe482ffd700 1 -- 172.21.15.6:0/1964580473 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c00e070 con 0x7fe4840b2bc0 2023-11-28T13:03:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.275+0000 7fe482ffd700 1 -- 172.21.15.6:0/1964580473 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c00bb20 con 0x7fe4840b2bc0 2023-11-28T13:03:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.275+0000 7fe482ffd700 1 -- 172.21.15.6:0/1964580473 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c00be90 con 0x7fe4840b2bc0 2023-11-28T13:03:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.276+0000 7fe48a6e0700 1 -- 172.21.15.6:0/1964580473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840b2bc0 msgr2=0x7fe4840ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.276+0000 7fe48a6e0700 1 --2- 172.21.15.6:0/1964580473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840b2bc0 0x7fe4840ce7d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe4840cc8e0 tx=0x7fe47c00b330 comp rx=0 tx=0).stop 2023-11-28T13:03:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.276+0000 7fe48a6e0700 1 -- 172.21.15.6:0/1964580473 shutdown_connections 2023-11-28T13:03:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.276+0000 7fe48a6e0700 1 --2- 172.21.15.6:0/1964580473 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840b2bc0 0x7fe4840ce7d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.276+0000 7fe48a6e0700 1 --2- 172.21.15.6:0/1964580473 >> v2:172.21.15.61:3300/0 conn(0x7fe4840cb630 0x7fe4840b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.276+0000 7fe48a6e0700 1 -- 172.21.15.6:0/1964580473 >> 172.21.15.6:0/1964580473 conn(0x7fe4840cd680 msgr2=0x7fe4840cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.276+0000 7fe48a6e0700 1 -- 172.21.15.6:0/1964580473 shutdown_connections 2023-11-28T13:03:16.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.276+0000 7fe48a6e0700 1 -- 172.21.15.6:0/1964580473 wait complete. 2023-11-28T13:03:16.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.277+0000 7fe48a6e0700 1 Processor -- start 2023-11-28T13:03:16.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.277+0000 7fe48a6e0700 1 -- start start 2023-11-28T13:03:16.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe48a6e0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe4840b2bc0 0x7fe4841f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe48a6e0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe4840b62d0 0x7fe484150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe48a6e0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840cb630 0x7fe484158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe48a6e0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840c8e30 con 0x7fe4840cb630 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe48a6e0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841f2650 con 0x7fe4840b2bc0 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe48a6e0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841f2930 con 0x7fe4840b62d0 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe488c7d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840cb630 0x7fe484158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe488c7d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840cb630 0x7fe484158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34160/0 (socket says 172.21.15.6:34160) 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe488c7d700 1 -- 172.21.15.6:0/4225339933 learned_addr learned my addr 172.21.15.6:0/4225339933 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe4837fe700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe4840b62d0 0x7fe484150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe483fff700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe4840b2bc0 0x7fe4841f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe488c7d700 1 -- 172.21.15.6:0/4225339933 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe4840b62d0 msgr2=0x7fe484150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe488c7d700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe4840b62d0 0x7fe484150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe488c7d700 1 -- 172.21.15.6:0/4225339933 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe4840b2bc0 msgr2=0x7fe4841f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe488c7d700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe4840b2bc0 0x7fe4841f2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe488c7d700 1 -- 172.21.15.6:0/4225339933 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c009a70 con 0x7fe4840cb630 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.278+0000 7fe488c7d700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840cb630 0x7fe484158990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe47c014040 tx=0x7fe47c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.279+0000 7fe480ff9700 1 -- 172.21.15.6:0/4225339933 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c004800 con 0x7fe4840cb630 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.279+0000 7fe480ff9700 1 -- 172.21.15.6:0/4225339933 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c020070 con 0x7fe4840cb630 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.279+0000 7fe48a6e0700 1 -- 172.21.15.6:0/4225339933 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4840b66a0 con 0x7fe4840cb630 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.279+0000 7fe480ff9700 1 -- 172.21.15.6:0/4225339933 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0276c0 con 0x7fe4840cb630 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.279+0000 7fe48a6e0700 1 -- 172.21.15.6:0/4225339933 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4840cffb0 con 0x7fe4840cb630 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.280+0000 7fe48a6e0700 1 -- 172.21.15.6:0/4225339933 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4840cce50 con 0x7fe4840cb630 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.280+0000 7fe480ff9700 1 -- 172.21.15.6:0/4225339933 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe47c012430 con 0x7fe4840cb630 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.281+0000 7fe480ff9700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4640682f0 0x7fe46406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.281+0000 7fe480ff9700 1 -- 172.21.15.6:0/4225339933 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fe47c022070 con 0x7fe4840cb630 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.281+0000 7fe483fff700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4640682f0 0x7fe46406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.281+0000 7fe483fff700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4640682f0 0x7fe46406a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe474007fd0 tx=0x7fe474009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:16.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.285+0000 7fe480ff9700 1 -- 172.21.15.6:0/4225339933 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe47c054720 con 0x7fe4840cb630 2023-11-28T13:03:16.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.528+0000 7fe48a6e0700 1 -- 172.21.15.6:0/4225339933 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe484067150 con 0x7fe4840cb630 2023-11-28T13:03:16.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.533+0000 7fe480ff9700 1 -- 172.21.15.6:0/4225339933 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v28) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe47c057d40 con 0x7fe4840cb630 2023-11-28T13:03:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.538+0000 7fe48a6e0700 1 -- 172.21.15.6:0/4225339933 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4640682f0 msgr2=0x7fe46406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.538+0000 7fe48a6e0700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4640682f0 0x7fe46406a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe474007fd0 tx=0x7fe474009040 comp rx=0 tx=0).stop 2023-11-28T13:03:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.538+0000 7fe48a6e0700 1 -- 172.21.15.6:0/4225339933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840cb630 msgr2=0x7fe484158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.539 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.538+0000 7fe48a6e0700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840cb630 0x7fe484158990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe47c014040 tx=0x7fe47c004ad0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.540 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.539+0000 7fe48a6e0700 1 -- 172.21.15.6:0/4225339933 shutdown_connections 2023-11-28T13:03:16.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.540+0000 7fe48a6e0700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4840cb630 0x7fe484158990 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.540+0000 7fe48a6e0700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe4840b2bc0 0x7fe4841f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.540+0000 7fe48a6e0700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe4640682f0 0x7fe46406a7a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.540+0000 7fe48a6e0700 1 --2- 172.21.15.6:0/4225339933 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe4840b62d0 0x7fe484150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.541 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.540+0000 7fe48a6e0700 1 -- 172.21.15.6:0/4225339933 >> 172.21.15.6:0/4225339933 conn(0x7fe4840cd680 msgr2=0x7fe4840aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.541+0000 7fe48a6e0700 1 -- 172.21.15.6:0/4225339933 shutdown_connections 2023-11-28T13:03:16.542 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.541+0000 7fe48a6e0700 1 -- 172.21.15.6:0/4225339933 wait complete. 2023-11-28T13:03:16.546 INFO:teuthology.orchestra.run.smithi006.stderr:new fs with metadata pool 10 and data pool 11 2023-11-28T13:03:16.564 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:03:16.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f342b0db700 1 Processor -- start 2023-11-28T13:03:16.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f342b0db700 1 -- start start 2023-11-28T13:03:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f342b0db700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f342415c990 0x7f34240cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f342b0db700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3424161090 0x7f34240ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f342b0db700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f34240caeb0 0x7f34240cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f342b0db700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34240ce340 con 0x7f342415c990 2023-11-28T13:03:16.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f342b0db700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34240cebe0 con 0x7f34240caeb0 2023-11-28T13:03:16.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f342b0db700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240cb3f0 con 0x7f3424161090 2023-11-28T13:03:16.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3428e77700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f342415c990 0x7f34240cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3428e77700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f342415c990 0x7f34240cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34164/0 (socket says 172.21.15.6:34164) 2023-11-28T13:03:16.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3428e77700 1 -- 172.21.15.6:0/4130667605 learned_addr learned my addr 172.21.15.6:0/4130667605 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:16.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3423fff700 1 --2- 172.21.15.6:0/4130667605 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3424161090 0x7f34240ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3429678700 1 --2- 172.21.15.6:0/4130667605 >> v2:172.21.15.61:3300/0 conn(0x7f34240caeb0 0x7f34240cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3428e77700 1 -- 172.21.15.6:0/4130667605 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3424161090 msgr2=0x7f34240ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:16.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3428e77700 1 --2- 172.21.15.6:0/4130667605 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3424161090 0x7f34240ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.694 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3428e77700 1 -- 172.21.15.6:0/4130667605 >> v2:172.21.15.61:3300/0 conn(0x7f34240caeb0 msgr2=0x7f34240cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:16.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3428e77700 1 --2- 172.21.15.6:0/4130667605 >> v2:172.21.15.61:3300/0 conn(0x7f34240caeb0 0x7f34240cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3428e77700 1 -- 172.21.15.6:0/4130667605 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34240ce590 con 0x7f342415c990 2023-11-28T13:03:16.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.689+0000 7f3428e77700 1 --2- 172.21.15.6:0/4130667605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 0x7f34240cf070 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f34240ca080 tx=0x7f3418009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d62c567e9c6ee6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:16.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f34237fe700 1 -- 172.21.15.6:0/4130667605 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341800b870 con 0x7f342415c990 2023-11-28T13:03:16.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f34237fe700 1 -- 172.21.15.6:0/4130667605 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341800b9d0 con 0x7f342415c990 2023-11-28T13:03:16.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f34237fe700 1 -- 172.21.15.6:0/4130667605 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341800bd60 con 0x7f342415c990 2023-11-28T13:03:16.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f342b0db700 1 -- 172.21.15.6:0/4130667605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 msgr2=0x7f34240cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f342b0db700 1 --2- 172.21.15.6:0/4130667605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 0x7f34240cf070 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f34240ca080 tx=0x7f3418009bd0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f342b0db700 1 -- 172.21.15.6:0/4130667605 shutdown_connections 2023-11-28T13:03:16.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f342b0db700 1 --2- 172.21.15.6:0/4130667605 >> v2:172.21.15.61:3300/0 conn(0x7f34240caeb0 0x7f34240cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f342b0db700 1 --2- 172.21.15.6:0/4130667605 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 0x7f34240cf070 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f342b0db700 1 --2- 172.21.15.6:0/4130667605 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3424161090 0x7f34240ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f342b0db700 1 -- 172.21.15.6:0/4130667605 >> 172.21.15.6:0/4130667605 conn(0x7f34240b1650 msgr2=0x7f34240b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:16.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f342b0db700 1 -- 172.21.15.6:0/4130667605 shutdown_connections 2023-11-28T13:03:16.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.690+0000 7f342b0db700 1 -- 172.21.15.6:0/4130667605 wait complete. 2023-11-28T13:03:16.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.692+0000 7f342b0db700 1 Processor -- start 2023-11-28T13:03:16.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.692+0000 7f342b0db700 1 -- start start 2023-11-28T13:03:16.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f342b0db700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34240caeb0 0x7f3424151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f342b0db700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 0x7f34240ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f342b0db700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3424161090 0x7f34240abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f342b0db700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240c4410 con 0x7f342415c990 2023-11-28T13:03:16.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f342b0db700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342415b4d0 con 0x7f34240caeb0 2023-11-28T13:03:16.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f342b0db700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240b4f90 con 0x7f3424161090 2023-11-28T13:03:16.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3423fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 0x7f34240ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3423fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 0x7f34240ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34172/0 (socket says 172.21.15.6:34172) 2023-11-28T13:03:16.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3423fff700 1 -- 172.21.15.6:0/526031917 learned_addr learned my addr 172.21.15.6:0/526031917 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:16.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3428e77700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34240caeb0 0x7f3424151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3429678700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3424161090 0x7f34240abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3423fff700 1 -- 172.21.15.6:0/526031917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3424161090 msgr2=0x7f34240abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3423fff700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3424161090 0x7f34240abe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3423fff700 1 -- 172.21.15.6:0/526031917 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34240caeb0 msgr2=0x7f3424151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3423fff700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34240caeb0 0x7f3424151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3423fff700 1 -- 172.21.15.6:0/526031917 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3418009550 con 0x7f342415c990 2023-11-28T13:03:16.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.693+0000 7f3423fff700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 0x7f34240ab8c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f341000d7d0 tx=0x7f341000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:16.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.694+0000 7f34217fa700 1 -- 172.21.15.6:0/526031917 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34100125f0 con 0x7f342415c990 2023-11-28T13:03:16.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.694+0000 7f34217fa700 1 -- 172.21.15.6:0/526031917 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410012790 con 0x7f342415c990 2023-11-28T13:03:16.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.694+0000 7f342b0db700 1 -- 172.21.15.6:0/526031917 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34240cf5b0 con 0x7f342415c990 2023-11-28T13:03:16.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.694+0000 7f34217fa700 1 -- 172.21.15.6:0/526031917 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410012b40 con 0x7f342415c990 2023-11-28T13:03:16.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.694+0000 7f342b0db700 1 -- 172.21.15.6:0/526031917 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34240ce870 con 0x7f342415c990 2023-11-28T13:03:16.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.695+0000 7f342b0db700 1 -- 172.21.15.6:0/526031917 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34240cf740 con 0x7f342415c990 2023-11-28T13:03:16.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.695+0000 7f34217fa700 1 -- 172.21.15.6:0/526031917 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3410012ca0 con 0x7f342415c990 2023-11-28T13:03:16.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.696+0000 7f34217fa700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34080682f0 0x7f340806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:16.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.696+0000 7f3428e77700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34080682f0 0x7f340806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:16.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.696+0000 7f34217fa700 1 -- 172.21.15.6:0/526031917 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3410086010 con 0x7f342415c990 2023-11-28T13:03:16.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.696+0000 7f3428e77700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34080682f0 0x7f340806a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3418000c00 tx=0x7f3418008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:16.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.701+0000 7f34217fa700 1 -- 172.21.15.6:0/526031917 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f34100526a0 con 0x7f342415c990 2023-11-28T13:03:16.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.896+0000 7f342b0db700 1 -- 172.21.15.6:0/526031917 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34240675a0 con 0x7f342415c990 2023-11-28T13:03:16.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.897+0000 7f34217fa700 1 -- 172.21.15.6:0/526031917 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18042 (secure 0 0 0) 0x7f3410055cc0 con 0x7f342415c990 2023-11-28T13:03:16.899 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:16.899 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":65,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:03:16.529792+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:03:14.665009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T13:03:15.497374+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:03:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.899+0000 7f342b0db700 1 -- 172.21.15.6:0/526031917 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34080682f0 msgr2=0x7f340806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.899+0000 7f342b0db700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34080682f0 0x7f340806a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3418000c00 tx=0x7f3418008030 comp rx=0 tx=0).stop 2023-11-28T13:03:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.899+0000 7f342b0db700 1 -- 172.21.15.6:0/526031917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 msgr2=0x7f34240ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.899+0000 7f342b0db700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 0x7f34240ab8c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f341000d7d0 tx=0x7f341000dae0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.899+0000 7f342b0db700 1 -- 172.21.15.6:0/526031917 shutdown_connections 2023-11-28T13:03:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.899+0000 7f342b0db700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f34080682f0 0x7f340806a7a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.899+0000 7f342b0db700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3424161090 0x7f34240abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.899+0000 7f342b0db700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34240caeb0 0x7f3424151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.899+0000 7f342b0db700 1 --2- 172.21.15.6:0/526031917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f342415c990 0x7f34240ab8c0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.899+0000 7f342b0db700 1 -- 172.21.15.6:0/526031917 >> 172.21.15.6:0/526031917 conn(0x7f34240b1650 msgr2=0x7f342406c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.900+0000 7f342b0db700 1 -- 172.21.15.6:0/526031917 shutdown_connections 2023-11-28T13:03:16.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:16.900+0000 7f342b0db700 1 -- 172.21.15.6:0/526031917 wait complete. 2023-11-28T13:03:16.913 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:03:17.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.037+0000 7fac6e102700 1 Processor -- start 2023-11-28T13:03:17.039 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.037+0000 7fac6e102700 1 -- start start 2023-11-28T13:03:17.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.038+0000 7fac6e102700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6806e400 0x7fac6815db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.038+0000 7fac6e102700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fac680ce890 0x7fac680ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.038+0000 7fac6e102700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6815e0b0 con 0x7fac68159590 2023-11-28T13:03:17.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.038+0000 7fac6e102700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac680cf250 con 0x7fac680ce890 2023-11-28T13:03:17.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.038+0000 7fac6e102700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6815d120 con 0x7fac6806e400 2023-11-28T13:03:17.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.038+0000 7fac67fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fac680ce890 0x7fac680ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.038+0000 7fac677fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fac68159590 0x7fac680ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49730/0 (socket says 172.21.15.6:49730) 2023-11-28T13:03:17.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.038+0000 7fac677fe700 1 -- 172.21.15.6:0/3817065181 learned_addr learned my addr 172.21.15.6:0/3817065181 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:17.041 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac66ffd700 1 --2- 172.21.15.6:0/3817065181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6806e400 0x7fac6815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac67fff700 1 -- 172.21.15.6:0/3817065181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6806e400 msgr2=0x7fac6815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac67fff700 1 --2- 172.21.15.6:0/3817065181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6806e400 0x7fac6815db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac67fff700 1 -- 172.21.15.6:0/3817065181 >> v1:172.21.15.6:6789/0 conn(0x7fac68159590 legacy=0x7fac680ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac667fc700 1 -- 172.21.15.6:0/3817065181 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329230788 0 0) 0x7fac58001110 con 0x7fac68159590 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac67fff700 1 -- 172.21.15.6:0/3817065181 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6815d6e0 con 0x7fac680ce890 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac67fff700 1 --2- 172.21.15.6:0/3817065181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 0x7fac680ced10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fac60009d90 tx=0x7fac6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415fc45ee72c26e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac667fc700 1 -- 172.21.15.6:0/3817065181 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6000e070 con 0x7fac680ce890 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac667fc700 1 -- 172.21.15.6:0/3817065181 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6000bbd0 con 0x7fac680ce890 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac667fc700 1 -- 172.21.15.6:0/3817065181 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60012430 con 0x7fac680ce890 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac6e102700 1 -- 172.21.15.6:0/3817065181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 msgr2=0x7fac680ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.039+0000 7fac6e102700 1 --2- 172.21.15.6:0/3817065181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 0x7fac680ced10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fac60009d90 tx=0x7fac6000b330 comp rx=0 tx=0).stop 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.040+0000 7fac6e102700 1 -- 172.21.15.6:0/3817065181 shutdown_connections 2023-11-28T13:03:17.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.040+0000 7fac6e102700 1 --2- 172.21.15.6:0/3817065181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 0x7fac680ced10 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.040+0000 7fac6e102700 1 --2- 172.21.15.6:0/3817065181 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6806e400 0x7fac6815db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.040+0000 7fac6e102700 1 -- 172.21.15.6:0/3817065181 >> 172.21.15.6:0/3817065181 conn(0x7fac680b1650 msgr2=0x7fac680b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:17.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.040+0000 7fac6e102700 1 -- 172.21.15.6:0/3817065181 shutdown_connections 2023-11-28T13:03:17.043 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.040+0000 7fac6e102700 1 -- 172.21.15.6:0/3817065181 wait complete. 2023-11-28T13:03:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.041+0000 7fac6e102700 1 Processor -- start 2023-11-28T13:03:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.041+0000 7fac6e102700 1 -- start start 2023-11-28T13:03:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac6e102700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6806e400 0x7fac68154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.047 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac6e102700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 0x7fac680b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac6e102700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac68159590 0x7fac680b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac6e102700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac68159960 con 0x7fac680ce890 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac6e102700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac68154da0 con 0x7fac68159590 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac6e102700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac681f0a00 con 0x7fac6806e400 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac66ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 0x7fac680b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac66ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 0x7fac680b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34186/0 (socket says 172.21.15.6:34186) 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac66ffd700 1 -- 172.21.15.6:0/3949786503 learned_addr learned my addr 172.21.15.6:0/3949786503 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac677fe700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6806e400 0x7fac68154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac67fff700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac68159590 0x7fac680b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac66ffd700 1 -- 172.21.15.6:0/3949786503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6806e400 msgr2=0x7fac68154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac66ffd700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6806e400 0x7fac68154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac66ffd700 1 -- 172.21.15.6:0/3949786503 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac68159590 msgr2=0x7fac680b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac66ffd700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac68159590 0x7fac680b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.048 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac66ffd700 1 -- 172.21.15.6:0/3949786503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac60009a70 con 0x7fac680ce890 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac67fff700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac68159590 0x7fac680b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac66ffd700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 0x7fac680b4960 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fac5000e810 tx=0x7fac5000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac5ffff700 1 -- 172.21.15.6:0/3949786503 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50013610 con 0x7fac680ce890 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac5ffff700 1 -- 172.21.15.6:0/3949786503 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac50013770 con 0x7fac680ce890 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac5ffff700 1 -- 172.21.15.6:0/3949786503 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50013b20 con 0x7fac680ce890 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.042+0000 7fac6e102700 1 -- 172.21.15.6:0/3949786503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac680af240 con 0x7fac680ce890 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.043+0000 7fac6e102700 1 -- 172.21.15.6:0/3949786503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac680aef60 con 0x7fac680ce890 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.043+0000 7fac6e102700 1 -- 172.21.15.6:0/3949786503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6815d370 con 0x7fac680ce890 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.044+0000 7fac5ffff700 1 -- 172.21.15.6:0/3949786503 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fac50023070 con 0x7fac680ce890 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.044+0000 7fac5ffff700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac480682f0 0x7fac4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.044+0000 7fac5ffff700 1 -- 172.21.15.6:0/3949786503 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fac50085b20 con 0x7fac680ce890 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.044+0000 7fac677fe700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac480682f0 0x7fac4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.049 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.045+0000 7fac677fe700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac480682f0 0x7fac4806a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fac58000f60 tx=0x7fac58007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:17.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.050+0000 7fac5ffff700 1 -- 172.21.15.6:0/3949786503 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fac500520d0 con 0x7fac680ce890 2023-11-28T13:03:17.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.245+0000 7fac6e102700 1 -- 172.21.15.6:0/3949786503 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac680675a0 con 0x7fac680ce890 2023-11-28T13:03:17.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.247+0000 7fac5ffff700 1 -- 172.21.15.6:0/3949786503 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18042 (secure 0 0 0) 0x7fac500556f0 con 0x7fac680ce890 2023-11-28T13:03:17.250 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:17.250 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":65,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:03:16.529792+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:03:14.665009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T13:03:15.497374+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:03:17.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 -- 172.21.15.6:0/3949786503 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac480682f0 msgr2=0x7fac4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:17.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac480682f0 0x7fac4806a7a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fac58000f60 tx=0x7fac58007e70 comp rx=0 tx=0).stop 2023-11-28T13:03:17.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 -- 172.21.15.6:0/3949786503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 msgr2=0x7fac680b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:17.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 0x7fac680b4960 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fac5000e810 tx=0x7fac5000eb20 comp rx=0 tx=0).stop 2023-11-28T13:03:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 -- 172.21.15.6:0/3949786503 shutdown_connections 2023-11-28T13:03:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac68159590 0x7fac680b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac480682f0 0x7fac4806a7a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac6806e400 0x7fac68154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 --2- 172.21.15.6:0/3949786503 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac680ce890 0x7fac680b4960 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 -- 172.21.15.6:0/3949786503 >> 172.21.15.6:0/3949786503 conn(0x7fac680b1650 msgr2=0x7fac680af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 -- 172.21.15.6:0/3949786503 shutdown_connections 2023-11-28T13:03:17.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.249+0000 7fac6e102700 1 -- 172.21.15.6:0/3949786503 wait complete. 2023-11-28T13:03:17.263 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T13:03:17.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.390+0000 7f48adb9f700 1 Processor -- start 2023-11-28T13:03:17.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.390+0000 7f48adb9f700 1 -- start start 2023-11-28T13:03:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f48a80b62d0 0x7f48a80cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f48a806f8b0 0x7f48a80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a80b2fe0 con 0x7f48a80b2bc0 2023-11-28T13:03:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a80cd2a0 con 0x7f48a806f8b0 2023-11-28T13:03:17.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a80d0400 con 0x7f48a80b62d0 2023-11-28T13:03:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a6ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f48a806f8b0 0x7f48a80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a6ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f48a806f8b0 0x7f48a80b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34194/0 (socket says 172.21.15.6:34194) 2023-11-28T13:03:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a6ffd700 1 -- 172.21.15.6:0/2796148236 learned_addr learned my addr 172.21.15.6:0/2796148236 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a77fe700 1 --2- 172.21.15.6:0/2796148236 >> v2:172.21.15.61:3300/0 conn(0x7f48a80b62d0 0x7f48a80cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a6ffd700 1 -- 172.21.15.6:0/2796148236 >> v2:172.21.15.61:3300/0 conn(0x7f48a80b62d0 msgr2=0x7f48a80cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a6ffd700 1 --2- 172.21.15.6:0/2796148236 >> v2:172.21.15.61:3300/0 conn(0x7f48a80b62d0 0x7f48a80cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a6ffd700 1 -- 172.21.15.6:0/2796148236 >> v1:172.21.15.61:6789/0 conn(0x7f48a80b2bc0 legacy=0x7f48a80ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a6ffd700 1 -- 172.21.15.6:0/2796148236 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a80d06e0 con 0x7f48a806f8b0 2023-11-28T13:03:17.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a6ffd700 1 --2- 172.21.15.6:0/2796148236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a806f8b0 0x7f48a80b2680 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f48a80720d0 tx=0x7f4898009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42db275578e033 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a67fc700 1 -- 172.21.15.6:0/2796148236 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898004230 con 0x7f48a806f8b0 2023-11-28T13:03:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a67fc700 1 -- 172.21.15.6:0/2796148236 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898004390 con 0x7f48a806f8b0 2023-11-28T13:03:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48a67fc700 1 -- 172.21.15.6:0/2796148236 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48980036a0 con 0x7f48a806f8b0 2023-11-28T13:03:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 -- 172.21.15.6:0/2796148236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a806f8b0 msgr2=0x7f48a80b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 --2- 172.21.15.6:0/2796148236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a806f8b0 0x7f48a80b2680 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f48a80720d0 tx=0x7f4898009bd0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 -- 172.21.15.6:0/2796148236 shutdown_connections 2023-11-28T13:03:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 --2- 172.21.15.6:0/2796148236 >> v2:172.21.15.61:3300/0 conn(0x7f48a80b62d0 0x7f48a80cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 --2- 172.21.15.6:0/2796148236 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a806f8b0 0x7f48a80b2680 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 -- 172.21.15.6:0/2796148236 >> 172.21.15.6:0/2796148236 conn(0x7f48a80cd680 msgr2=0x7f48a80cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:17.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 -- 172.21.15.6:0/2796148236 shutdown_connections 2023-11-28T13:03:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.391+0000 7f48adb9f700 1 -- 172.21.15.6:0/2796148236 wait complete. 2023-11-28T13:03:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.393+0000 7f48adb9f700 1 Processor -- start 2023-11-28T13:03:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.393+0000 7f48adb9f700 1 -- start start 2023-11-28T13:03:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48adb9f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48a806f8b0 0x7f48a8159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48adb9f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48a80b2bc0 0x7f48a81574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48adb9f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a80b62d0 0x7f48a80c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48adb9f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a815c590 con 0x7f48a80b62d0 2023-11-28T13:03:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48adb9f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a815bc60 con 0x7f48a80b2bc0 2023-11-28T13:03:17.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48adb9f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a815bf40 con 0x7f48a806f8b0 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a80b62d0 0x7f48a80c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a80b62d0 0x7f48a80c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34202/0 (socket says 172.21.15.6:34202) 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a7fff700 1 -- 172.21.15.6:0/3127155524 learned_addr learned my addr 172.21.15.6:0/3127155524 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a77fe700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48a806f8b0 0x7f48a8159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a6ffd700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48a80b2bc0 0x7f48a81574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a7fff700 1 -- 172.21.15.6:0/3127155524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48a806f8b0 msgr2=0x7f48a8159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a7fff700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48a806f8b0 0x7f48a8159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a7fff700 1 -- 172.21.15.6:0/3127155524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48a80b2bc0 msgr2=0x7f48a81574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a7fff700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48a80b2bc0 0x7f48a81574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a7fff700 1 -- 172.21.15.6:0/3127155524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898009550 con 0x7f48a80b62d0 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.394+0000 7f48a6ffd700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48a80b2bc0 0x7f48a81574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:17.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.395+0000 7f48a7fff700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a80b62d0 0x7f48a80c9320 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f48a000cc10 tx=0x7f48a000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.395+0000 7f489ffff700 1 -- 172.21.15.6:0/3127155524 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a000abb0 con 0x7f48a80b62d0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.395+0000 7f489ffff700 1 -- 172.21.15.6:0/3127155524 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a000ad10 con 0x7f48a80b62d0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.395+0000 7f48adb9f700 1 -- 172.21.15.6:0/3127155524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a806fd30 con 0x7f48a80b62d0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.395+0000 7f489ffff700 1 -- 172.21.15.6:0/3127155524 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0012660 con 0x7f48a80b62d0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.395+0000 7f48adb9f700 1 -- 172.21.15.6:0/3127155524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a80cffb0 con 0x7f48a80b62d0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.396+0000 7f48adb9f700 1 -- 172.21.15.6:0/3127155524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a80b66a0 con 0x7f48a80b62d0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.396+0000 7f489ffff700 1 -- 172.21.15.6:0/3127155524 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f48a0022070 con 0x7f48a80b62d0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.397+0000 7f489ffff700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48880682f0 0x7f488806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.397+0000 7f48a77fe700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48880682f0 0x7f488806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.397+0000 7f489ffff700 1 -- 172.21.15.6:0/3127155524 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f48a0086410 con 0x7f48a80b62d0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.397+0000 7f48a77fe700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48880682f0 0x7f488806a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f489000b9b0 tx=0x7f4890008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:17.502 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.402+0000 7f489ffff700 1 -- 172.21.15.6:0/3127155524 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f48a0052b30 con 0x7f48a80b62d0 2023-11-28T13:03:17.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:17.643+0000 7f48adb9f700 1 -- 172.21.15.6:0/3127155524 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f48a80739f0 con 0x7f48a80b62d0 2023-11-28T13:03:18.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.552+0000 7f489ffff700 1 -- 172.21.15.6:0/3127155524 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f48a00160b0 con 0x7f48a80b62d0 2023-11-28T13:03:18.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 -- 172.21.15.6:0/3127155524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48880682f0 msgr2=0x7f488806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48880682f0 0x7f488806a7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f489000b9b0 tx=0x7f4890008040 comp rx=0 tx=0).stop 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 -- 172.21.15.6:0/3127155524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a80b62d0 msgr2=0x7f48a80c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a80b62d0 0x7f48a80c9320 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f48a000cc10 tx=0x7f48a000a060 comp rx=0 tx=0).stop 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 -- 172.21.15.6:0/3127155524 shutdown_connections 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48a80b62d0 0x7f48a80c9320 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48880682f0 0x7f488806a7a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48a806f8b0 0x7f48a8159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 --2- 172.21.15.6:0/3127155524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f48a80b2bc0 0x7f48a81574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 -- 172.21.15.6:0/3127155524 >> 172.21.15.6:0/3127155524 conn(0x7f48a80cd680 msgr2=0x7f48a80b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 -- 172.21.15.6:0/3127155524 shutdown_connections 2023-11-28T13:03:18.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.555+0000 7f48adb9f700 1 -- 172.21.15.6:0/3127155524 wait complete. 2023-11-28T13:03:18.570 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T13:03:18.571 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T13:03:18.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.705+0000 7ff8019ab700 1 Processor -- start 2023-11-28T13:03:18.705 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.705+0000 7ff8019ab700 1 -- start start 2023-11-28T13:03:18.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.705+0000 7ff8019ab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7fc0d2820 0x7ff7fc0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:18.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.705+0000 7ff8019ab700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff7fc06bf30 0x7ff7fc06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:18.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.705+0000 7ff8019ab700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7fc06c890 con 0x7ff7fc15c640 2023-11-28T13:03:18.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.705+0000 7ff8019ab700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7fc15ca10 con 0x7ff7fc06bf30 2023-11-28T13:03:18.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.705+0000 7ff8019ab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0d1470 con 0x7ff7fc0d2820 2023-11-28T13:03:18.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7faffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff7fc15c640 0x7ff7fc0cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49758/0 (socket says 172.21.15.6:49758) 2023-11-28T13:03:18.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7fb7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff7fc06bf30 0x7ff7fc06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7faffd700 1 -- 172.21.15.6:0/142982018 learned_addr learned my addr 172.21.15.6:0/142982018 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7fa7fc700 1 --2- 172.21.15.6:0/142982018 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7fc0d2820 0x7ff7fc0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7fb7fe700 1 -- 172.21.15.6:0/142982018 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7fc0d2820 msgr2=0x7ff7fc0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7fb7fe700 1 --2- 172.21.15.6:0/142982018 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7fc0d2820 0x7ff7fc0b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7fb7fe700 1 -- 172.21.15.6:0/142982018 >> v1:172.21.15.6:6789/0 conn(0x7ff7fc15c640 legacy=0x7ff7fc0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7fb7fe700 1 -- 172.21.15.6:0/142982018 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc0cd4e0 con 0x7ff7fc06bf30 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7f9ffb700 1 -- 172.21.15.6:0/142982018 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607574355 0 0) 0x7ff7fc06c890 con 0x7ff7fc15c640 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7fb7fe700 1 --2- 172.21.15.6:0/142982018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 0x7ff7fc06c350 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009d90 tx=0x7ff7f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912d0191fed28aa9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7f9ffb700 1 -- 172.21.15.6:0/142982018 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f400e070 con 0x7ff7fc06bf30 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7f9ffb700 1 -- 172.21.15.6:0/142982018 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f400bbd0 con 0x7ff7fc06bf30 2023-11-28T13:03:18.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.706+0000 7ff7f9ffb700 1 -- 172.21.15.6:0/142982018 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4012430 con 0x7ff7fc06bf30 2023-11-28T13:03:18.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.707+0000 7ff8019ab700 1 -- 172.21.15.6:0/142982018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 msgr2=0x7ff7fc06c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:18.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.707+0000 7ff8019ab700 1 --2- 172.21.15.6:0/142982018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 0x7ff7fc06c350 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009d90 tx=0x7ff7f400b330 comp rx=0 tx=0).stop 2023-11-28T13:03:18.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.707+0000 7ff8019ab700 1 -- 172.21.15.6:0/142982018 shutdown_connections 2023-11-28T13:03:18.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.707+0000 7ff8019ab700 1 --2- 172.21.15.6:0/142982018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 0x7ff7fc06c350 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:18.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.707+0000 7ff8019ab700 1 --2- 172.21.15.6:0/142982018 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7fc0d2820 0x7ff7fc0b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:18.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.707+0000 7ff8019ab700 1 -- 172.21.15.6:0/142982018 >> 172.21.15.6:0/142982018 conn(0x7ff7fc0d2df0 msgr2=0x7ff7fc0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:18.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.707+0000 7ff8019ab700 1 -- 172.21.15.6:0/142982018 shutdown_connections 2023-11-28T13:03:18.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.707+0000 7ff8019ab700 1 -- 172.21.15.6:0/142982018 wait complete. 2023-11-28T13:03:18.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.708+0000 7ff8019ab700 1 Processor -- start 2023-11-28T13:03:18.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.708+0000 7ff8019ab700 1 -- start start 2023-11-28T13:03:18.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff8019ab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 0x7ff7fc1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:18.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff8019ab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7fc0d2820 0x7ff7fc0b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff8019ab700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7fc15c640 0x7ff7fc155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff8019ab700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0b5ce0 con 0x7ff7fc06bf30 2023-11-28T13:03:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff8019ab700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0b7930 con 0x7ff7fc15c640 2023-11-28T13:03:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff8019ab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1f5ca0 con 0x7ff7fc0d2820 2023-11-28T13:03:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff7faffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 0x7ff7fc1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff7faffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 0x7ff7fc1f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34212/0 (socket says 172.21.15.6:34212) 2023-11-28T13:03:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff7faffd700 1 -- 172.21.15.6:0/430595233 learned_addr learned my addr 172.21.15.6:0/430595233 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff7fa7fc700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7fc0d2820 0x7ff7fc0b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff7fb7fe700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7fc15c640 0x7ff7fc155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:18.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff7faffd700 1 -- 172.21.15.6:0/430595233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7fc0d2820 msgr2=0x7ff7fc0b7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff7faffd700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7fc0d2820 0x7ff7fc0b7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff7faffd700 1 -- 172.21.15.6:0/430595233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7fc15c640 msgr2=0x7ff7fc155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff7faffd700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7fc15c640 0x7ff7fc155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.709+0000 7ff7faffd700 1 -- 172.21.15.6:0/430595233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f4009a70 con 0x7ff7fc06bf30 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.710+0000 7ff7faffd700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 0x7ff7fc1f56c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0011e0 tx=0x7ff7ec00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.710+0000 7ff7e3fff700 1 -- 172.21.15.6:0/430595233 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec004c80 con 0x7ff7fc06bf30 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.710+0000 7ff7e3fff700 1 -- 172.21.15.6:0/430595233 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec004e00 con 0x7ff7fc06bf30 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.710+0000 7ff8019ab700 1 -- 172.21.15.6:0/430595233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7fc0d1710 con 0x7ff7fc06bf30 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.710+0000 7ff7e3fff700 1 -- 172.21.15.6:0/430595233 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec007310 con 0x7ff7fc06bf30 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.710+0000 7ff8019ab700 1 -- 172.21.15.6:0/430595233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc0cd7c0 con 0x7ff7fc06bf30 2023-11-28T13:03:18.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.711+0000 7ff8019ab700 1 -- 172.21.15.6:0/430595233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7fc0cdaa0 con 0x7ff7fc06bf30 2023-11-28T13:03:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.711+0000 7ff7e3fff700 1 -- 172.21.15.6:0/430595233 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff7ec01a020 con 0x7ff7fc06bf30 2023-11-28T13:03:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.711+0000 7ff7e3fff700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7e80682f0 0x7ff7e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.712+0000 7ff7e3fff700 1 -- 172.21.15.6:0/430595233 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff7ec085920 con 0x7ff7fc06bf30 2023-11-28T13:03:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.712+0000 7ff7fa7fc700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7e80682f0 0x7ff7e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.712+0000 7ff7fa7fc700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7e80682f0 0x7ff7e806a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc06a950 tx=0x7ff7f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:18.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.716+0000 7ff7e3fff700 1 -- 172.21.15.6:0/430595233 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff7ec051fe0 con 0x7ff7fc06bf30 2023-11-28T13:03:18.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:18.955+0000 7ff8019ab700 1 -- 172.21.15.6:0/430595233 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff7fc0739f0 con 0x7ff7fc06bf30 2023-11-28T13:03:19.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.559+0000 7ff7e3fff700 1 -- 172.21.15.6:0/430595233 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff7ec055600 con 0x7ff7fc06bf30 2023-11-28T13:03:19.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 -- 172.21.15.6:0/430595233 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7e80682f0 msgr2=0x7ff7e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:19.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7e80682f0 0x7ff7e806a7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc06a950 tx=0x7ff7f0008040 comp rx=0 tx=0).stop 2023-11-28T13:03:19.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 -- 172.21.15.6:0/430595233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 msgr2=0x7ff7fc1f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:19.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 0x7ff7fc1f56c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0011e0 tx=0x7ff7ec00ca30 comp rx=0 tx=0).stop 2023-11-28T13:03:19.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 -- 172.21.15.6:0/430595233 shutdown_connections 2023-11-28T13:03:19.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7fc15c640 0x7ff7fc155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7fc06bf30 0x7ff7fc1f56c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7e80682f0 0x7ff7e806a7a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 --2- 172.21.15.6:0/430595233 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7fc0d2820 0x7ff7fc0b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 -- 172.21.15.6:0/430595233 >> 172.21.15.6:0/430595233 conn(0x7ff7fc0d2df0 msgr2=0x7ff7fc0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:19.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 -- 172.21.15.6:0/430595233 shutdown_connections 2023-11-28T13:03:19.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.562+0000 7ff8019ab700 1 -- 172.21.15.6:0/430595233 wait complete. 2023-11-28T13:03:19.577 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T13:03:19.578 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:19.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.709+0000 7fe37753b700 1 Processor -- start 2023-11-28T13:03:19.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.709+0000 7fe37753b700 1 -- start start 2023-11-28T13:03:19.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.710+0000 7fe37753b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe370159590 0x7fe37006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:19.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.710+0000 7fe37753b700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37015d120 con 0x7fe37015db70 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.710+0000 7fe37753b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37015d6e0 con 0x7fe3700ce890 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.710+0000 7fe37753b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700aef60 con 0x7fe370159590 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.710+0000 7fe374ad6700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe37015db70 0x7fe3700ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49776/0 (socket says 172.21.15.6:49776) 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.710+0000 7fe374ad6700 1 -- 172.21.15.6:0/474050829 learned_addr learned my addr 172.21.15.6:0/474050829 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.710+0000 7fe3752d7700 1 --2- 172.21.15.6:0/474050829 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe370159590 0x7fe37006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.710+0000 7fe367fff700 1 -- 172.21.15.6:0/474050829 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585215790 0 0) 0x7fe37015d120 con 0x7fe37015db70 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.710+0000 7fe367fff700 1 -- 172.21.15.6:0/474050829 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe35c0035b0 con 0x7fe37015db70 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.711+0000 7fe367fff700 1 -- 172.21.15.6:0/474050829 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476200536 0 0) 0x7fe37015d6e0 con 0x7fe3700ce890 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.711+0000 7fe367fff700 1 -- 172.21.15.6:0/474050829 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe37015d120 con 0x7fe3700ce890 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.711+0000 7fe367fff700 1 -- 172.21.15.6:0/474050829 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1603249158 0 0) 0x7fe35c0035b0 con 0x7fe37015db70 2023-11-28T13:03:19.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.711+0000 7fe3752d7700 1 -- 172.21.15.6:0/474050829 >> v1:172.21.15.61:6789/0 conn(0x7fe3700ce890 legacy=0x7fe3700caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:19.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.711+0000 7fe3752d7700 1 -- 172.21.15.6:0/474050829 >> v1:172.21.15.6:6789/0 conn(0x7fe37015db70 legacy=0x7fe3700ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:19.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.711+0000 7fe3752d7700 1 -- 172.21.15.6:0/474050829 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3700af240 con 0x7fe370159590 2023-11-28T13:03:19.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.711+0000 7fe3752d7700 1 --2- 172.21.15.6:0/474050829 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe370159590 0x7fe37006e400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe360009870 tx=0x7fe360009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce8c87ac9e0fe1b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:19.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.711+0000 7fe367fff700 1 -- 172.21.15.6:0/474050829 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c001670 con 0x7fe370159590 2023-11-28T13:03:19.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.711+0000 7fe367fff700 1 -- 172.21.15.6:0/474050829 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe360004030 con 0x7fe370159590 2023-11-28T13:03:19.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.711+0000 7fe367fff700 1 -- 172.21.15.6:0/474050829 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600043a0 con 0x7fe370159590 2023-11-28T13:03:19.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.712+0000 7fe37753b700 1 -- 172.21.15.6:0/474050829 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe370159590 msgr2=0x7fe37006e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:19.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.712+0000 7fe37753b700 1 --2- 172.21.15.6:0/474050829 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe370159590 0x7fe37006e400 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe360009870 tx=0x7fe360009b80 comp rx=0 tx=0).stop 2023-11-28T13:03:19.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.712+0000 7fe37753b700 1 -- 172.21.15.6:0/474050829 shutdown_connections 2023-11-28T13:03:19.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.712+0000 7fe37753b700 1 --2- 172.21.15.6:0/474050829 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe370159590 0x7fe37006e400 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.712+0000 7fe37753b700 1 -- 172.21.15.6:0/474050829 >> 172.21.15.6:0/474050829 conn(0x7fe3700b1650 msgr2=0x7fe3700b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:19.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.712+0000 7fe37753b700 1 -- 172.21.15.6:0/474050829 shutdown_connections 2023-11-28T13:03:19.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.712+0000 7fe37753b700 1 -- 172.21.15.6:0/474050829 wait complete. 2023-11-28T13:03:19.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.713+0000 7fe37753b700 1 Processor -- start 2023-11-28T13:03:19.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.713+0000 7fe37753b700 1 -- start start 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe37753b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3700ce890 0x7fe3700c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe37753b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe370159590 0x7fe3701526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe37753b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe37015db70 0x7fe3700b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe37753b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700c8650 con 0x7fe370159590 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe37753b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700c7cf0 con 0x7fe3700ce890 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe37753b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700b4d60 con 0x7fe37015db70 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe374ad6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe370159590 0x7fe3701526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe374ad6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe370159590 0x7fe3701526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34228/0 (socket says 172.21.15.6:34228) 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe374ad6700 1 -- 172.21.15.6:0/2363385887 learned_addr learned my addr 172.21.15.6:0/2363385887 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe3752d7700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3700ce890 0x7fe3700c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe375ad8700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe37015db70 0x7fe3700b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:19.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe374ad6700 1 -- 172.21.15.6:0/2363385887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe37015db70 msgr2=0x7fe3700b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe374ad6700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe37015db70 0x7fe3700b4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe374ad6700 1 -- 172.21.15.6:0/2363385887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3700ce890 msgr2=0x7fe3700c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe374ad6700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3700ce890 0x7fe3700c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe374ad6700 1 -- 172.21.15.6:0/2363385887 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3600094a0 con 0x7fe370159590 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe3752d7700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3700ce890 0x7fe3700c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.714+0000 7fe374ad6700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe370159590 0x7fe3701526d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00c500 tx=0x7fe36c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.715+0000 7fe365ffb700 1 -- 172.21.15.6:0/2363385887 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c004c80 con 0x7fe370159590 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.715+0000 7fe365ffb700 1 -- 172.21.15.6:0/2363385887 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36c004e00 con 0x7fe370159590 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.715+0000 7fe365ffb700 1 -- 172.21.15.6:0/2363385887 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c007310 con 0x7fe370159590 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.715+0000 7fe37753b700 1 -- 172.21.15.6:0/2363385887 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37015dfe0 con 0x7fe370159590 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.715+0000 7fe37753b700 1 -- 172.21.15.6:0/2363385887 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37015d370 con 0x7fe370159590 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.716+0000 7fe37753b700 1 -- 172.21.15.6:0/2363385887 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe370159960 con 0x7fe370159590 2023-11-28T13:03:19.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.716+0000 7fe365ffb700 1 -- 172.21.15.6:0/2363385887 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe36c01a020 con 0x7fe370159590 2023-11-28T13:03:19.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.717+0000 7fe365ffb700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3540682f0 0x7fe35406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:19.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.717+0000 7fe3752d7700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3540682f0 0x7fe35406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:19.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.717+0000 7fe365ffb700 1 -- 172.21.15.6:0/2363385887 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe36c085b80 con 0x7fe370159590 2023-11-28T13:03:19.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.717+0000 7fe3752d7700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3540682f0 0x7fe35406a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe3600090b0 tx=0x7fe360008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:19.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.721+0000 7fe365ffb700 1 -- 172.21.15.6:0/2363385887 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe36c052150 con 0x7fe370159590 2023-11-28T13:03:19.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.955+0000 7fe37753b700 1 -- 172.21.15.6:0/2363385887 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3700c8790 con 0x7fe370159590 2023-11-28T13:03:19.957 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.956+0000 7fe365ffb700 1 -- 172.21.15.6:0/2363385887 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4641 (secure 0 0 0) 0x7fe3700c8790 con 0x7fe370159590 2023-11-28T13:03:19.958 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:19.958 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1198233336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1198233336},{"type":"v1","addr":"172.21.15.6:6836","nonce":1198233336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/235204249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":235204249},{"type":"v1","addr":"172.21.15.6:6837","nonce":235204249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2675481701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2675481701},{"type":"v1","addr":"172.21.15.61:6833","nonce":2675481701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:03:16.530114+0000","modified":"2023-11-28T13:03:19.551191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6835/3076579711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":3076579711},{"type":"v1","addr":"172.21.15.61:6835","nonce":3076579711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:03:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 -- 172.21.15.6:0/2363385887 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3540682f0 msgr2=0x7fe35406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3540682f0 0x7fe35406a7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe3600090b0 tx=0x7fe360008030 comp rx=0 tx=0).stop 2023-11-28T13:03:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 -- 172.21.15.6:0/2363385887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe370159590 msgr2=0x7fe3701526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe370159590 0x7fe3701526d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00c500 tx=0x7fe36c00c8c0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 -- 172.21.15.6:0/2363385887 shutdown_connections 2023-11-28T13:03:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe3540682f0 0x7fe35406a7a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe37015db70 0x7fe3700b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe3700ce890 0x7fe3700c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 --2- 172.21.15.6:0/2363385887 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe370159590 0x7fe3701526d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 -- 172.21.15.6:0/2363385887 >> 172.21.15.6:0/2363385887 conn(0x7fe3700b1650 msgr2=0x7fe3700b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 -- 172.21.15.6:0/2363385887 shutdown_connections 2023-11-28T13:03:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:19.958+0000 7fe37753b700 1 -- 172.21.15.6:0/2363385887 wait complete. 2023-11-28T13:03:19.960 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 32 2023-11-28T13:03:19.971 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:03:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.095+0000 7f09d2246700 1 Processor -- start 2023-11-28T13:03:20.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.096+0000 7f09d2246700 1 -- start start 2023-11-28T13:03:20.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.096+0000 7f09d2246700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f09cc15c990 0x7f09cc0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.096+0000 7f09d2246700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f09cc161090 0x7f09cc0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.096+0000 7f09d2246700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09cc0ce340 con 0x7f09cc0caeb0 2023-11-28T13:03:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.096+0000 7f09d2246700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc0cebe0 con 0x7f09cc161090 2023-11-28T13:03:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.096+0000 7f09d2246700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc0cb3f0 con 0x7f09cc15c990 2023-11-28T13:03:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.096+0000 7f09caffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f09cc161090 0x7f09cc0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.096+0000 7f09caffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f09cc161090 0x7f09cc0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34236/0 (socket says 172.21.15.6:34236) 2023-11-28T13:03:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09caffd700 1 -- 172.21.15.6:0/541836472 learned_addr learned my addr 172.21.15.6:0/541836472 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09cbfff700 1 --1- 172.21.15.6:0/541836472 >> v1:172.21.15.6:6789/0 conn(0x7f09cc0caeb0 0x7f09cc0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49790/0 (socket says 172.21.15.6:49790) 2023-11-28T13:03:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09cb7fe700 1 --2- 172.21.15.6:0/541836472 >> v2:172.21.15.61:3300/0 conn(0x7f09cc15c990 0x7f09cc0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09caffd700 1 -- 172.21.15.6:0/541836472 >> v2:172.21.15.61:3300/0 conn(0x7f09cc15c990 msgr2=0x7f09cc0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:20.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09caffd700 1 --2- 172.21.15.6:0/541836472 >> v2:172.21.15.61:3300/0 conn(0x7f09cc15c990 0x7f09cc0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09caffd700 1 -- 172.21.15.6:0/541836472 >> v1:172.21.15.6:6789/0 conn(0x7f09cc0caeb0 legacy=0x7f09cc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09caffd700 1 -- 172.21.15.6:0/541836472 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc0ce590 con 0x7f09cc161090 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09caffd700 1 --2- 172.21.15.6:0/541836472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc161090 0x7f09cc0ca970 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f09cc06ea90 tx=0x7f09c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25089c029c530982 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09ca7fc700 1 -- 172.21.15.6:0/541836472 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000b870 con 0x7f09cc161090 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09ca7fc700 1 -- 172.21.15.6:0/541836472 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c000b9d0 con 0x7f09cc161090 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.097+0000 7f09ca7fc700 1 -- 172.21.15.6:0/541836472 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000bd60 con 0x7f09cc161090 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.098+0000 7f09d2246700 1 -- 172.21.15.6:0/541836472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc161090 msgr2=0x7f09cc0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.098+0000 7f09d2246700 1 --2- 172.21.15.6:0/541836472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc161090 0x7f09cc0ca970 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f09cc06ea90 tx=0x7f09c0009bd0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.098+0000 7f09d2246700 1 -- 172.21.15.6:0/541836472 shutdown_connections 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.098+0000 7f09d2246700 1 --2- 172.21.15.6:0/541836472 >> v2:172.21.15.61:3300/0 conn(0x7f09cc15c990 0x7f09cc0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.098+0000 7f09d2246700 1 --2- 172.21.15.6:0/541836472 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc161090 0x7f09cc0ca970 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.098+0000 7f09d2246700 1 -- 172.21.15.6:0/541836472 >> 172.21.15.6:0/541836472 conn(0x7f09cc0b1650 msgr2=0x7f09cc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:20.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.098+0000 7f09d2246700 1 -- 172.21.15.6:0/541836472 shutdown_connections 2023-11-28T13:03:20.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.098+0000 7f09d2246700 1 -- 172.21.15.6:0/541836472 wait complete. 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.099+0000 7f09d2246700 1 Processor -- start 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.099+0000 7f09d2246700 1 -- start start 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09d2246700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09cc0caeb0 0x7f09cc0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09d2246700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc15c990 0x7f09cc0ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09d2246700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09cc161090 0x7f09cc1f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09d2246700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0ae5f0 con 0x7f09cc15c990 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09d2246700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0ae8c0 con 0x7f09cc161090 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09d2246700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0c41a0 con 0x7f09cc0caeb0 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09caffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc15c990 0x7f09cc0ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09caffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc15c990 0x7f09cc0ae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34242/0 (socket says 172.21.15.6:34242) 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09caffd700 1 -- 172.21.15.6:0/4271181356 learned_addr learned my addr 172.21.15.6:0/4271181356 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:20.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09cb7fe700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09cc0caeb0 0x7f09cc0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09cbfff700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09cc161090 0x7f09cc1f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09caffd700 1 -- 172.21.15.6:0/4271181356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09cc0caeb0 msgr2=0x7f09cc0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09caffd700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09cc0caeb0 0x7f09cc0abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09caffd700 1 -- 172.21.15.6:0/4271181356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09cc161090 msgr2=0x7f09cc1f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09caffd700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09cc161090 0x7f09cc1f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.100+0000 7f09caffd700 1 -- 172.21.15.6:0/4271181356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0009550 con 0x7f09cc15c990 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.101+0000 7f09caffd700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc15c990 0x7f09cc0ae0b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009c10 tx=0x7f09c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.101+0000 7f09b3fff700 1 -- 172.21.15.6:0/4271181356 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00036a0 con 0x7f09cc15c990 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.101+0000 7f09b3fff700 1 -- 172.21.15.6:0/4271181356 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c0003800 con 0x7f09cc15c990 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.101+0000 7f09d2246700 1 -- 172.21.15.6:0/4271181356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc0cf5b0 con 0x7f09cc15c990 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.101+0000 7f09b3fff700 1 -- 172.21.15.6:0/4271181356 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0003b70 con 0x7f09cc15c990 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.101+0000 7f09d2246700 1 -- 172.21.15.6:0/4271181356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc0ce870 con 0x7f09cc15c990 2023-11-28T13:03:20.105 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.102+0000 7f09b1ffb700 1 -- 172.21.15.6:0/4271181356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a40052f0 con 0x7f09cc15c990 2023-11-28T13:03:20.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.102+0000 7f09b3fff700 1 -- 172.21.15.6:0/4271181356 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f09c000f050 con 0x7f09cc15c990 2023-11-28T13:03:20.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.103+0000 7f09b3fff700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f09b80682f0 0x7f09b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.103+0000 7f09b3fff700 1 -- 172.21.15.6:0/4271181356 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f09c00872a0 con 0x7f09cc15c990 2023-11-28T13:03:20.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.103+0000 7f09cb7fe700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f09b80682f0 0x7f09b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.106 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.103+0000 7f09cb7fe700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f09b80682f0 0x7f09b806a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009930 tx=0x7f09bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:20.107 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.107+0000 7f09b3fff700 1 -- 172.21.15.6:0/4271181356 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f09c00539c0 con 0x7f09cc15c990 2023-11-28T13:03:20.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.302+0000 7f09b1ffb700 1 -- 172.21.15.6:0/4271181356 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09a40055a0 con 0x7f09cc15c990 2023-11-28T13:03:20.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.303+0000 7f09b3fff700 1 -- 172.21.15.6:0/4271181356 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+18029 (secure 0 0 0) 0x7f09c0056fe0 con 0x7f09cc15c990 2023-11-28T13:03:20.304 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:20.304 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":66,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:03:17.531838+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:03:14.665009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T13:03:15.497374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.305+0000 7f09b1ffb700 1 -- 172.21.15.6:0/4271181356 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f09b80682f0 msgr2=0x7f09b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.305+0000 7f09b1ffb700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f09b80682f0 0x7f09b806a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009930 tx=0x7f09bc008040 comp rx=0 tx=0).stop 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.305+0000 7f09b1ffb700 1 -- 172.21.15.6:0/4271181356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc15c990 msgr2=0x7f09cc0ae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.305+0000 7f09b1ffb700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc15c990 0x7f09cc0ae0b0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009c10 tx=0x7f09c0004750 comp rx=0 tx=0).stop 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.305+0000 7f09b1ffb700 1 -- 172.21.15.6:0/4271181356 shutdown_connections 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.305+0000 7f09b1ffb700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f09cc161090 0x7f09cc1f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.305+0000 7f09b1ffb700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f09b80682f0 0x7f09b806a7a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.305+0000 7f09b1ffb700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f09cc0caeb0 0x7f09cc0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.305+0000 7f09b1ffb700 1 --2- 172.21.15.6:0/4271181356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09cc15c990 0x7f09cc0ae0b0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.305+0000 7f09b1ffb700 1 -- 172.21.15.6:0/4271181356 >> 172.21.15.6:0/4271181356 conn(0x7f09cc0b1650 msgr2=0x7f09cc06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:20.306 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.306+0000 7f09b1ffb700 1 -- 172.21.15.6:0/4271181356 shutdown_connections 2023-11-28T13:03:20.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.306+0000 7f09b1ffb700 1 -- 172.21.15.6:0/4271181356 wait complete. 2023-11-28T13:03:20.318 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T13:03:20.319 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:03:20.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.443+0000 7fd4957b8700 1 Processor -- start 2023-11-28T13:03:20.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.443+0000 7fd4957b8700 1 -- start start 2023-11-28T13:03:20.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.444+0000 7fd4957b8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd4900cc310 0x7fd4900cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.444+0000 7fd4957b8700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4900ce340 con 0x7fd49015c990 2023-11-28T13:03:20.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.444+0000 7fd4957b8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4900cebe0 con 0x7fd490161090 2023-11-28T13:03:20.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.444+0000 7fd4957b8700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4900cf780 con 0x7fd4900cc310 2023-11-28T13:03:20.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.444+0000 7fd48f7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd4900cc310 0x7fd4900cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.444+0000 7fd48effd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd49015c990 0x7fd4900cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49800/0 (socket says 172.21.15.6:49800) 2023-11-28T13:03:20.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.444+0000 7fd48effd700 1 -- 172.21.15.6:0/1235004267 learned_addr learned my addr 172.21.15.6:0/1235004267 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:20.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.445+0000 7fd48f7fe700 1 -- 172.21.15.6:0/1235004267 >> v1:172.21.15.61:6789/0 conn(0x7fd490161090 legacy=0x7fd4900ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.445+0000 7fd48f7fe700 1 -- 172.21.15.6:0/1235004267 >> v1:172.21.15.6:6789/0 conn(0x7fd49015c990 legacy=0x7fd4900cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.445+0000 7fd48f7fe700 1 -- 172.21.15.6:0/1235004267 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4900cccb0 con 0x7fd4900cc310 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.445+0000 7fd48dffb700 1 -- 172.21.15.6:0/1235004267 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910776833 0 0) 0x7fd4900ce340 con 0x7fd49015c990 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.445+0000 7fd48f7fe700 1 --2- 172.21.15.6:0/1235004267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4900cc310 0x7fd4900cc770 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d90 tx=0x7fd48800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856d98f4a6f5a895 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.445+0000 7fd48dffb700 1 -- 172.21.15.6:0/1235004267 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800e070 con 0x7fd4900cc310 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.445+0000 7fd48dffb700 1 -- 172.21.15.6:0/1235004267 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800bb90 con 0x7fd4900cc310 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.445+0000 7fd48dffb700 1 -- 172.21.15.6:0/1235004267 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488003a20 con 0x7fd4900cc310 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.445+0000 7fd4957b8700 1 -- 172.21.15.6:0/1235004267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4900cc310 msgr2=0x7fd4900cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.445+0000 7fd4957b8700 1 --2- 172.21.15.6:0/1235004267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4900cc310 0x7fd4900cc770 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d90 tx=0x7fd48800b330 comp rx=0 tx=0).stop 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.446+0000 7fd4957b8700 1 -- 172.21.15.6:0/1235004267 shutdown_connections 2023-11-28T13:03:20.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.446+0000 7fd4957b8700 1 --2- 172.21.15.6:0/1235004267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd4900cc310 0x7fd4900cc770 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.446+0000 7fd4957b8700 1 -- 172.21.15.6:0/1235004267 >> 172.21.15.6:0/1235004267 conn(0x7fd4900b1650 msgr2=0x7fd4900b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:20.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.446+0000 7fd4957b8700 1 -- 172.21.15.6:0/1235004267 shutdown_connections 2023-11-28T13:03:20.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.446+0000 7fd4957b8700 1 -- 172.21.15.6:0/1235004267 wait complete. 2023-11-28T13:03:20.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.447+0000 7fd4957b8700 1 Processor -- start 2023-11-28T13:03:20.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.447+0000 7fd4957b8700 1 -- start start 2023-11-28T13:03:20.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.448+0000 7fd4957b8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4900cc310 0x7fd4901e8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.448+0000 7fd4957b8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd49015c990 0x7fd4900c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.448+0000 7fd4957b8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd490161090 0x7fd4900c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.448+0000 7fd4957b8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900c4170 con 0x7fd490161090 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.448+0000 7fd4957b8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900c91f0 con 0x7fd49015c990 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.448+0000 7fd4957b8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900c94d0 con 0x7fd4900cc310 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.448+0000 7fd48f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd490161090 0x7fd4900c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd490161090 0x7fd4900c7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34258/0 (socket says 172.21.15.6:34258) 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48f7fe700 1 -- 172.21.15.6:0/3746706163 learned_addr learned my addr 172.21.15.6:0/3746706163 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48effd700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4900cc310 0x7fd4901e8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48e7fc700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd49015c990 0x7fd4900c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48f7fe700 1 -- 172.21.15.6:0/3746706163 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4900cc310 msgr2=0x7fd4901e8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48f7fe700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4900cc310 0x7fd4901e8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48f7fe700 1 -- 172.21.15.6:0/3746706163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd49015c990 msgr2=0x7fd4900c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48f7fe700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd49015c990 0x7fd4900c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48f7fe700 1 -- 172.21.15.6:0/3746706163 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd488009a70 con 0x7fd490161090 2023-11-28T13:03:20.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48e7fc700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd49015c990 0x7fd4900c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:20.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd48f7fe700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd490161090 0x7fd4900c7400 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd49006eac0 tx=0x7fd488004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:20.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd4777fe700 1 -- 172.21.15.6:0/3746706163 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488012630 con 0x7fd490161090 2023-11-28T13:03:20.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd4777fe700 1 -- 172.21.15.6:0/3746706163 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4880127b0 con 0x7fd490161090 2023-11-28T13:03:20.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd4957b8700 1 -- 172.21.15.6:0/3746706163 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4900ce590 con 0x7fd490161090 2023-11-28T13:03:20.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd4777fe700 1 -- 172.21.15.6:0/3746706163 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488012b60 con 0x7fd490161090 2023-11-28T13:03:20.452 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.449+0000 7fd4957b8700 1 -- 172.21.15.6:0/3746706163 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4900cb3f0 con 0x7fd490161090 2023-11-28T13:03:20.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.450+0000 7fd4957b8700 1 -- 172.21.15.6:0/3746706163 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4900ce870 con 0x7fd490161090 2023-11-28T13:03:20.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.451+0000 7fd4777fe700 1 -- 172.21.15.6:0/3746706163 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd488012cc0 con 0x7fd490161090 2023-11-28T13:03:20.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.451+0000 7fd4777fe700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd47c0682f0 0x7fd47c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.453 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.451+0000 7fd4777fe700 1 -- 172.21.15.6:0/3746706163 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd488088560 con 0x7fd490161090 2023-11-28T13:03:20.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.451+0000 7fd48effd700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd47c0682f0 0x7fd47c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.454 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.452+0000 7fd48effd700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd47c0682f0 0x7fd47c06a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd484000f60 tx=0x7fd484009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:20.455 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.455+0000 7fd4777fe700 1 -- 172.21.15.6:0/3746706163 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd488054d30 con 0x7fd490161090 2023-11-28T13:03:20.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.649+0000 7fd4957b8700 1 -- 172.21.15.6:0/3746706163 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4900c9360 con 0x7fd47c0682f0 2023-11-28T13:03:20.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.657+0000 7fd4777fe700 1 -- 172.21.15.6:0/3746706163 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297188 (secure 0 0 0) 0x7fd4900c9360 con 0x7fd47c0682f0 2023-11-28T13:03:20.660 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 -- 172.21.15.6:0/3746706163 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd47c0682f0 msgr2=0x7fd47c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd47c0682f0 0x7fd47c06a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd484000f60 tx=0x7fd484009040 comp rx=0 tx=0).stop 2023-11-28T13:03:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 -- 172.21.15.6:0/3746706163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd490161090 msgr2=0x7fd4900c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd490161090 0x7fd4900c7400 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd49006eac0 tx=0x7fd488004ad0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 -- 172.21.15.6:0/3746706163 shutdown_connections 2023-11-28T13:03:20.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd490161090 0x7fd4900c7400 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd47c0682f0 0x7fd47c06a7a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd4900cc310 0x7fd4901e8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 --2- 172.21.15.6:0/3746706163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd49015c990 0x7fd4900c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 -- 172.21.15.6:0/3746706163 >> 172.21.15.6:0/3746706163 conn(0x7fd4900b1650 msgr2=0x7fd490160250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 -- 172.21.15.6:0/3746706163 shutdown_connections 2023-11-28T13:03:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.660+0000 7fd4957b8700 1 -- 172.21.15.6:0/3746706163 wait complete. 2023-11-28T13:03:20.664 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:03:20.673 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2023-11-28T13:03:18.981262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78368,"kb_used_data":7008,"kb_used_omap":0,"kb_used_meta":71296,"kb_avail":754896352,"statfs":{"total":773094113280,"available":773013864448,"internally_reserved":0,"allocated":7176192,"data_stored":3959785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73007104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000917"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:03:11.252066+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:03:11.252066+0000","last_peered":"2023-11-28T13:03:11.252066+0000","last_clean":"2023-11-28T13:03:11.252066+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:03:11.252066+0000","last_undegraded":"2023-11-28T13:03:11.252066+0000","last_fullsized":"2023-11-28T13:03:11.252066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":51,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:03:11.236241+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:03:11.236241+0000","last_peered":"2023-11-28T13:03:11.236241+0000","last_clean":"2023-11-28T13:03:11.236241+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:03:11.236241+0000","last_undegraded":"2023-11-28T13:03:11.236241+0000","last_fullsized":"2023-11-28T13:03:11.236241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:03:11.251845+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:03:11.251845+0000","last_peered":"2023-11-28T13:03:11.251845+0000","last_clean":"2023-11-28T13:03:11.251845+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:03:11.251845+0000","last_undegraded":"2023-11-28T13:03:11.251845+0000","last_fullsized":"2023-11-28T13:03:11.251845+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":206,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:03:11.237389+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:03:11.237389+0000","last_peered":"2023-11-28T13:03:11.237389+0000","last_clean":"2023-11-28T13:03:11.237389+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:03:11.237389+0000","last_undegraded":"2023-11-28T13:03:11.237389+0000","last_fullsized":"2023-11-28T13:03:11.237389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:03:11.237312+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:03:11.237312+0000","last_peered":"2023-11-28T13:03:11.237312+0000","last_clean":"2023-11-28T13:03:11.237312+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:03:11.237312+0000","last_undegraded":"2023-11-28T13:03:11.237312+0000","last_fullsized":"2023-11-28T13:03:11.237312+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:03:11.237313+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:03:11.237313+0000","last_peered":"2023-11-28T13:03:11.237313+0000","last_clean":"2023-11-28T13:03:11.237313+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:03:11.237313+0000","last_undegraded":"2023-11-28T13:03:11.237313+0000","last_fullsized":"2023-11-28T13:03:11.237313+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:03:11.251730+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:03:11.251730+0000","last_peered":"2023-11-28T13:03:11.251730+0000","last_clean":"2023-11-28T13:03:11.251730+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:03:11.251730+0000","last_undegraded":"2023-11-28T13:03:11.251730+0000","last_fullsized":"2023-11-28T13:03:11.251730+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:03:11.251703+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:03:11.251703+0000","last_peered":"2023-11-28T13:03:11.251703+0000","last_clean":"2023-11-28T13:03:11.251703+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:03:11.251703+0000","last_undegraded":"2023-11-28T13:03:11.251703+0000","last_fullsized":"2023-11-28T13:03:11.251703+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:16.104884+0000","last_change":"2023-11-28T13:03:16.104884+0000","last_active":"2023-11-28T13:03:16.104884+0000","last_peered":"2023-11-28T13:03:16.104884+0000","last_clean":"2023-11-28T13:03:16.104884+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:16.104884+0000","last_undegraded":"2023-11-28T13:03:16.104884+0000","last_fullsized":"2023-11-28T13:03:16.104884+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T13:03:15.098635+0000","last_change":"2023-11-28T13:03:15.098635+0000","last_active":"2023-11-28T13:03:15.098635+0000","last_peered":"2023-11-28T13:03:15.098635+0000","last_clean":"2023-11-28T13:03:15.098635+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T13:03:15.098635+0000","last_undegraded":"2023-11-28T13:03:15.098635+0000","last_fullsized":"2023-11-28T13:03:15.098635+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-28T13:03:11.251682+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:03:11.251682+0000","last_peered":"2023-11-28T13:03:11.251682+0000","last_clean":"2023-11-28T13:03:11.251682+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:03:11.251682+0000","last_undegraded":"2023-11-28T13:03:11.251682+0000","last_fullsized":"2023-11-28T13:03:11.251682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10664,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361176,"statfs":{"total":96636764160,"available":96625844224,"internally_reserved":0,"allocated":1343488,"data_stored":937998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.45700000000000002,"15min":0.442},"min":{"1min":0.32200000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.502,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69899999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.60399999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.36499999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.52300000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.68799999999999994}]},{"osd":2,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.52000000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.40799999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.621}]},{"osd":3,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.77100000000000002,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.65600000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.751},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65200000000000002,"15min":0.65500000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.79000000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:02:18 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.46500000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.79000000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85299999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9380,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362460,"statfs":{"total":96636764160,"available":96627159040,"internally_reserved":0,"allocated":749568,"data_stored":347630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.33900000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.49399999999999999,"15min":0.48199999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72799999999999998,"5min":0.87,"15min":0.87},"last":0.68799999999999994}]},{"osd":2,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.375}]},{"osd":3,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42099999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55900000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.38900000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.74299999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.71099999999999997}]},{"osd":7,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.626},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.77100000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10532,"kb_used_data":1308,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361308,"statfs":{"total":96636764160,"available":96625979392,"internally_reserved":0,"allocated":1339392,"data_stored":936993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.313,"15min":0.309},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.318},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.56799999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.39900000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35099999999999998,"15min":0.34799999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38200000000000001,"15min":0.38},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.373}]},{"osd":3,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38400000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.82899999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.56399999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46500000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.72899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.626},"min":{"1min":0.47299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":1.157,"15min":1.157},"last":0.60499999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.443},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86799999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.56200000000000006}]},{"osd":7,"last update":"Tue Nov 28 13:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.52500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9696,"kb_used_data":728,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362144,"statfs":{"total":96636764160,"available":96626835456,"internally_reserved":0,"allocated":745472,"data_stored":346644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.313},{"interface":"front","average":{"1min":0.37,"5min":0.34499999999999997,"15min":0.34100000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.36799999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.33500000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.33800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42499999999999999,"15min":0.42799999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80100000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.49299999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91900000000000004,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.504},{"interface":"front","average":{"1min":0.63,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.61599999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.46999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9380,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362460,"statfs":{"total":96636764160,"available":96627159040,"internally_reserved":0,"allocated":749568,"data_stored":347630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.438,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.311},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36599999999999999,"15min":0.36699999999999999},"min":{"1min":0.27700000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.25800000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51400000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38100000000000001},"min":{"1min":0.27400000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.56100000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.371,"15min":0.36899999999999999},"min":{"1min":0.27400000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52000000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.61599999999999999,"5min":0.748,"15min":0.748},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73299999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.499,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79300000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.621,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84099999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66200000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81599999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79600000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86699999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.63},{"interface":"front","average":{"1min":0.629,"5min":0.64100000000000001,"15min":0.64300000000000002},"min":{"1min":0.48199999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9636,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362204,"statfs":{"total":96636764160,"available":96626896896,"internally_reserved":0,"allocated":749568,"data_stored":347630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.504,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.47599999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.255,"5min":0.246,"15min":0.246},"max":{"1min":0.45200000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39800000000000002,"15min":0.40400000000000003},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58399999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.40500000000000003}]},{"osd":2,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.56399999999999995,"15min":0.59599999999999997},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54700000000000004,"5min":5.0540000000000003,"15min":5.0540000000000003},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.45200000000000001,"15min":0.46300000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.53700000000000003,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.44800000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85799999999999998,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.626,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84399999999999997,"5min":1.2,"15min":1.2},"last":0.54000000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":1.169,"15min":1.169},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81999999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.49399999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81499999999999995,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.627},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":1.23,"15min":1.23},"last":0.57699999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77800000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85199999999999998,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9636,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362204,"statfs":{"total":96636764160,"available":96626896896,"internally_reserved":0,"allocated":749568,"data_stored":347630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.45600000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.41999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.495,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53900000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.49099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.503},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.58699999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73799999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55600000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.624},{"interface":"front","average":{"1min":0.628,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81499999999999995,"5min":1.117,"15min":1.117},"last":0.60199999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9444,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362396,"statfs":{"total":96636764160,"available":96627093504,"internally_reserved":0,"allocated":749568,"data_stored":347630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71899999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.312},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42699999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77500000000000002,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.501,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59799999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.59499999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57999999999999996,"15min":0.58399999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.625,"15min":0.64000000000000001},"min":{"1min":0.35399999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.502},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.64000000000000001,"15min":0.66000000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.75700000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.45100000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.55700000000000005,"15min":0.56599999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.69499999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.67900000000000005,"15min":0.69699999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.751,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.66400000000000003,"15min":0.67700000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83299999999999996,"5min":1.052,"15min":1.052},"last":0.51600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:03:20.680 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:03:20.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.837+0000 7f0178e60700 1 Processor -- start 2023-11-28T13:03:20.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.837+0000 7f0178e60700 1 -- start start 2023-11-28T13:03:20.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.838+0000 7f0178e60700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0174161090 0x7f01740ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.838+0000 7f0178e60700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01740ce340 con 0x7f01740caeb0 2023-11-28T13:03:20.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.838+0000 7f0178e60700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01740cebe0 con 0x7f017415c990 2023-11-28T13:03:20.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.838+0000 7f0178e60700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01740cf780 con 0x7f0174161090 2023-11-28T13:03:20.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.838+0000 7f0172d9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f01740caeb0 0x7f01740cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49818/0 (socket says 172.21.15.6:49818) 2023-11-28T13:03:20.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.838+0000 7f0172d9d700 1 -- 172.21.15.6:0/1030379894 learned_addr learned my addr 172.21.15.6:0/1030379894 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:20.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f0171d9b700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0174161090 0x7f01740ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917681157 0 0) 0x7f01740ce340 con 0x7f01740caeb0 2023-11-28T13:03:20.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f015c0035b0 con 0x7f01740caeb0 2023-11-28T13:03:20.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3071923292 0 0) 0x7f015c0035b0 con 0x7f01740caeb0 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0174161090 msgr2=0x7f01740ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0174161090 0x7f01740ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 >> v1:172.21.15.61:6789/0 conn(0x7f017415c990 legacy=0x7f01740cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01740cb3f0 con 0x7f01740caeb0 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f016c0019f0 con 0x7f01740caeb0 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 >> v1:172.21.15.6:6789/0 conn(0x7f01740caeb0 legacy=0x7f01740cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c003a30 0x7f015c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f015c004850 0x7f015c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0174161090 0x7f015c005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01740cf780 con 0x7f015c003a30 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c00bb40 con 0x7f015c004850 2023-11-28T13:03:20.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c00bd00 con 0x7f0174161090 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017259c700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c003a30 0x7f015c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f0171d9b700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f015c004850 0x7f015c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f0172d9d700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0174161090 0x7f015c005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017259c700 1 -- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0174161090 msgr2=0x7f015c005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017259c700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0174161090 0x7f015c005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017259c700 1 -- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f015c004850 msgr2=0x7f015c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017259c700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f015c004850 0x7f015c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.839+0000 7f017259c700 1 -- 172.21.15.6:0/1030379894 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c0035b0 con 0x7f015c003a30 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f017259c700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c003a30 0x7f015c003e00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f01680018d0 tx=0x7f015c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7b8f83b42b2bd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168012620 con 0x7f015c003a30 2023-11-28T13:03:20.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01680127a0 con 0x7f015c003a30 2023-11-28T13:03:20.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f017159a700 1 -- 172.21.15.6:0/1030379894 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168012b50 con 0x7f015c003a30 2023-11-28T13:03:20.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f0178e60700 1 -- 172.21.15.6:0/1030379894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c003a30 msgr2=0x7f015c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f0178e60700 1 --2- 172.21.15.6:0/1030379894 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f015c003a30 0x7f015c003e00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f01680018d0 tx=0x7f015c005880 comp rx=0 tx=0).stop 2023-11-28T13:03:20.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f0172d9d700 1 -- 172.21.15.6:0/1030379894 reap_dead start 2023-11-28T13:03:20.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f0178e60700 1 -- 172.21.15.6:0/1030379894 shutdown_connections 2023-11-28T13:03:20.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f0178e60700 1 -- 172.21.15.6:0/1030379894 >> 172.21.15.6:0/1030379894 conn(0x7f01740b1650 msgr2=0x7f01740b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:20.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f0178e60700 1 -- 172.21.15.6:0/1030379894 shutdown_connections 2023-11-28T13:03:20.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.840+0000 7f0178e60700 1 -- 172.21.15.6:0/1030379894 wait complete. 2023-11-28T13:03:20.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.842+0000 7f0178e60700 1 Processor -- start 2023-11-28T13:03:20.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.842+0000 7f0178e60700 1 -- start start 2023-11-28T13:03:20.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.843+0000 7f0178e60700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f015c003a30 0x7f017415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.843+0000 7f0178e60700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01740c52c0 0x7f01740c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.843+0000 7f0178e60700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01740c5c70 0x7f01741f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.843+0000 7f0178e60700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01740b3db0 con 0x7f01740c52c0 2023-11-28T13:03:20.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.843+0000 7f0178e60700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01740c6110 con 0x7f01740c5c70 2023-11-28T13:03:20.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.843+0000 7f0178e60700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01740c41b0 con 0x7f015c003a30 2023-11-28T13:03:20.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.843+0000 7f0171d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01740c52c0 0x7f01740c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.847 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.843+0000 7f0171d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01740c52c0 0x7f01740c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34276/0 (socket says 172.21.15.6:34276) 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.843+0000 7f0171d9b700 1 -- 172.21.15.6:0/2175663035 learned_addr learned my addr 172.21.15.6:0/2175663035 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f017259c700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f015c003a30 0x7f017415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0171d9b700 1 -- 172.21.15.6:0/2175663035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f015c003a30 msgr2=0x7f017415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0172d9d700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01740c5c70 0x7f01741f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0171d9b700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f015c003a30 0x7f017415a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0171d9b700 1 -- 172.21.15.6:0/2175663035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01740c5c70 msgr2=0x7f01741f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0171d9b700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01740c5c70 0x7f01741f53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0171d9b700 1 -- 172.21.15.6:0/2175663035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01740ce870 con 0x7f01740c52c0 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0171d9b700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01740c52c0 0x7f01740c5730 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0164011940 tx=0x7f0164011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0162ffd700 1 -- 172.21.15.6:0/2175663035 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164004240 con 0x7f01740c52c0 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0162ffd700 1 -- 172.21.15.6:0/2175663035 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01640043a0 con 0x7f01740c52c0 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0178e60700 1 -- 172.21.15.6:0/2175663035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01740ce590 con 0x7f01740c52c0 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0162ffd700 1 -- 172.21.15.6:0/2175663035 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016400f950 con 0x7f01740c52c0 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.844+0000 7f0178e60700 1 -- 172.21.15.6:0/2175663035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01740cca20 con 0x7f01740c52c0 2023-11-28T13:03:20.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.845+0000 7f0178e60700 1 -- 172.21.15.6:0/2175663035 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01740675a0 con 0x7f01740c52c0 2023-11-28T13:03:20.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.846+0000 7f0162ffd700 1 -- 172.21.15.6:0/2175663035 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0164025070 con 0x7f01740c52c0 2023-11-28T13:03:20.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.846+0000 7f0162ffd700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01540682f0 0x7f015406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:20.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.846+0000 7f0162ffd700 1 -- 172.21.15.6:0/2175663035 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f01640860e0 con 0x7f01740c52c0 2023-11-28T13:03:20.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.846+0000 7f017259c700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01540682f0 0x7f015406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:20.849 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.846+0000 7f017259c700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01540682f0 0x7f015406a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0168014040 tx=0x7f0168005040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:20.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:20.850+0000 7f0162ffd700 1 -- 172.21.15.6:0/2175663035 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0164052800 con 0x7f01740c52c0 2023-11-28T13:03:21.042 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.041+0000 7f0178e60700 1 -- 172.21.15.6:0/2175663035 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01740cdcc0 con 0x7f01540682f0 2023-11-28T13:03:21.050 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.050+0000 7f0162ffd700 1 -- 172.21.15.6:0/2175663035 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312023 (secure 0 0 0) 0x7f01740cdcc0 con 0x7f01540682f0 2023-11-28T13:03:21.054 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:21.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 -- 172.21.15.6:0/2175663035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01540682f0 msgr2=0x7f015406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01540682f0 0x7f015406a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0168014040 tx=0x7f0168005040 comp rx=0 tx=0).stop 2023-11-28T13:03:21.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 -- 172.21.15.6:0/2175663035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01740c52c0 msgr2=0x7f01740c5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01740c52c0 0x7f01740c5730 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0164011940 tx=0x7f0164011d00 comp rx=0 tx=0).stop 2023-11-28T13:03:21.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 -- 172.21.15.6:0/2175663035 shutdown_connections 2023-11-28T13:03:21.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f01740c5c70 0x7f01741f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f01540682f0 0x7f015406a7a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f015c003a30 0x7f017415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 --2- 172.21.15.6:0/2175663035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01740c52c0 0x7f01740c5730 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 -- 172.21.15.6:0/2175663035 >> 172.21.15.6:0/2175663035 conn(0x7f01740b1650 msgr2=0x7f017415bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:21.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 -- 172.21.15.6:0/2175663035 shutdown_connections 2023-11-28T13:03:21.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.053+0000 7f0178e60700 1 -- 172.21.15.6:0/2175663035 wait complete. 2023-11-28T13:03:21.058 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:03:21.066 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2023-11-28T13:03:20.981635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":80112,"kb_used_data":7408,"kb_used_omap":0,"kb_used_meta":72640,"kb_avail":754894608,"statfs":{"total":773094113280,"available":773012078592,"internally_reserved":0,"allocated":7585792,"data_stored":4159425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74383360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.876407"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546634+0000","last_change":"2023-11-28T13:03:16.556121+0000","last_active":"2023-11-28T13:03:17.546634+0000","last_peered":"2023-11-28T13:03:17.546634+0000","last_clean":"2023-11-28T13:03:17.546634+0000","last_became_active":"2023-11-28T13:03:16.555688+0000","last_became_peered":"2023-11-28T13:03:16.555688+0000","last_unstale":"2023-11-28T13:03:17.546634+0000","last_undegraded":"2023-11-28T13:03:17.546634+0000","last_fullsized":"2023-11-28T13:03:17.546634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:24:24.843178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.855292+0000","last_change":"2023-11-28T13:03:16.562462+0000","last_active":"2023-11-28T13:03:18.855292+0000","last_peered":"2023-11-28T13:03:18.855292+0000","last_clean":"2023-11-28T13:03:18.855292+0000","last_became_active":"2023-11-28T13:03:16.562140+0000","last_became_peered":"2023-11-28T13:03:16.562140+0000","last_unstale":"2023-11-28T13:03:18.855292+0000","last_undegraded":"2023-11-28T13:03:18.855292+0000","last_fullsized":"2023-11-28T13:03:18.855292+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:51:33.574685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854586+0000","last_change":"2023-11-28T13:03:16.543800+0000","last_active":"2023-11-28T13:03:18.854586+0000","last_peered":"2023-11-28T13:03:18.854586+0000","last_clean":"2023-11-28T13:03:18.854586+0000","last_became_active":"2023-11-28T13:03:16.543552+0000","last_became_peered":"2023-11-28T13:03:16.543552+0000","last_unstale":"2023-11-28T13:03:18.854586+0000","last_undegraded":"2023-11-28T13:03:18.854586+0000","last_fullsized":"2023-11-28T13:03:18.854586+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:03.451223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854069+0000","last_change":"2023-11-28T13:03:16.555950+0000","last_active":"2023-11-28T13:03:18.854069+0000","last_peered":"2023-11-28T13:03:18.854069+0000","last_clean":"2023-11-28T13:03:18.854069+0000","last_became_active":"2023-11-28T13:03:16.555731+0000","last_became_peered":"2023-11-28T13:03:16.555731+0000","last_unstale":"2023-11-28T13:03:18.854069+0000","last_undegraded":"2023-11-28T13:03:18.854069+0000","last_fullsized":"2023-11-28T13:03:18.854069+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:22:31.390479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543141+0000","last_change":"2023-11-28T13:03:16.541618+0000","last_active":"2023-11-28T13:03:17.543141+0000","last_peered":"2023-11-28T13:03:17.543141+0000","last_clean":"2023-11-28T13:03:17.543141+0000","last_became_active":"2023-11-28T13:03:16.541404+0000","last_became_peered":"2023-11-28T13:03:16.541404+0000","last_unstale":"2023-11-28T13:03:17.543141+0000","last_undegraded":"2023-11-28T13:03:17.543141+0000","last_fullsized":"2023-11-28T13:03:17.543141+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:29:50.141486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854526+0000","last_change":"2023-11-28T13:03:16.561439+0000","last_active":"2023-11-28T13:03:18.854526+0000","last_peered":"2023-11-28T13:03:18.854526+0000","last_clean":"2023-11-28T13:03:18.854526+0000","last_became_active":"2023-11-28T13:03:16.561122+0000","last_became_peered":"2023-11-28T13:03:16.561122+0000","last_unstale":"2023-11-28T13:03:18.854526+0000","last_undegraded":"2023-11-28T13:03:18.854526+0000","last_fullsized":"2023-11-28T13:03:18.854526+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:07:20.686852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500229+0000","last_change":"2023-11-28T13:03:16.567868+0000","last_active":"2023-11-28T13:03:18.500229+0000","last_peered":"2023-11-28T13:03:18.500229+0000","last_clean":"2023-11-28T13:03:18.500229+0000","last_became_active":"2023-11-28T13:03:16.567551+0000","last_became_peered":"2023-11-28T13:03:16.567551+0000","last_unstale":"2023-11-28T13:03:18.500229+0000","last_undegraded":"2023-11-28T13:03:18.500229+0000","last_fullsized":"2023-11-28T13:03:18.500229+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:20.125179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854594+0000","last_change":"2023-11-28T13:03:16.550979+0000","last_active":"2023-11-28T13:03:18.854594+0000","last_peered":"2023-11-28T13:03:18.854594+0000","last_clean":"2023-11-28T13:03:18.854594+0000","last_became_active":"2023-11-28T13:03:16.550554+0000","last_became_peered":"2023-11-28T13:03:16.550554+0000","last_unstale":"2023-11-28T13:03:18.854594+0000","last_undegraded":"2023-11-28T13:03:18.854594+0000","last_fullsized":"2023-11-28T13:03:18.854594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:30.078114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543327+0000","last_change":"2023-11-28T13:03:16.549456+0000","last_active":"2023-11-28T13:03:17.543327+0000","last_peered":"2023-11-28T13:03:17.543327+0000","last_clean":"2023-11-28T13:03:17.543327+0000","last_became_active":"2023-11-28T13:03:16.548542+0000","last_became_peered":"2023-11-28T13:03:16.548542+0000","last_unstale":"2023-11-28T13:03:17.543327+0000","last_undegraded":"2023-11-28T13:03:17.543327+0000","last_fullsized":"2023-11-28T13:03:17.543327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:09.375531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.248943+0000","last_change":"2023-11-28T13:03:16.560146+0000","last_active":"2023-11-28T13:03:18.248943+0000","last_peered":"2023-11-28T13:03:18.248943+0000","last_clean":"2023-11-28T13:03:18.248943+0000","last_became_active":"2023-11-28T13:03:16.559851+0000","last_became_peered":"2023-11-28T13:03:16.559851+0000","last_unstale":"2023-11-28T13:03:18.248943+0000","last_undegraded":"2023-11-28T13:03:18.248943+0000","last_fullsized":"2023-11-28T13:03:18.248943+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:12:19.747468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546589+0000","last_change":"2023-11-28T13:03:16.553600+0000","last_active":"2023-11-28T13:03:17.546589+0000","last_peered":"2023-11-28T13:03:17.546589+0000","last_clean":"2023-11-28T13:03:17.546589+0000","last_became_active":"2023-11-28T13:03:16.552975+0000","last_became_peered":"2023-11-28T13:03:16.552975+0000","last_unstale":"2023-11-28T13:03:17.546589+0000","last_undegraded":"2023-11-28T13:03:17.546589+0000","last_fullsized":"2023-11-28T13:03:17.546589+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:01.764812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854173+0000","last_change":"2023-11-28T13:03:16.553435+0000","last_active":"2023-11-28T13:03:18.854173+0000","last_peered":"2023-11-28T13:03:18.854173+0000","last_clean":"2023-11-28T13:03:18.854173+0000","last_became_active":"2023-11-28T13:03:16.553155+0000","last_became_peered":"2023-11-28T13:03:16.553155+0000","last_unstale":"2023-11-28T13:03:18.854173+0000","last_undegraded":"2023-11-28T13:03:18.854173+0000","last_fullsized":"2023-11-28T13:03:18.854173+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:14:45.187571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.541562+0000","last_change":"2023-11-28T13:03:16.550263+0000","last_active":"2023-11-28T13:03:17.541562+0000","last_peered":"2023-11-28T13:03:17.541562+0000","last_clean":"2023-11-28T13:03:17.541562+0000","last_became_active":"2023-11-28T13:03:16.549991+0000","last_became_peered":"2023-11-28T13:03:16.549991+0000","last_unstale":"2023-11-28T13:03:17.541562+0000","last_undegraded":"2023-11-28T13:03:17.541562+0000","last_fullsized":"2023-11-28T13:03:17.541562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:23.649426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543248+0000","last_change":"2023-11-28T13:03:16.565986+0000","last_active":"2023-11-28T13:03:17.543248+0000","last_peered":"2023-11-28T13:03:17.543248+0000","last_clean":"2023-11-28T13:03:17.543248+0000","last_became_active":"2023-11-28T13:03:16.565679+0000","last_became_peered":"2023-11-28T13:03:16.565679+0000","last_unstale":"2023-11-28T13:03:17.543248+0000","last_undegraded":"2023-11-28T13:03:17.543248+0000","last_fullsized":"2023-11-28T13:03:17.543248+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:52.434197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.541432+0000","last_change":"2023-11-28T13:03:16.542267+0000","last_active":"2023-11-28T13:03:17.541432+0000","last_peered":"2023-11-28T13:03:17.541432+0000","last_clean":"2023-11-28T13:03:17.541432+0000","last_became_active":"2023-11-28T13:03:16.540566+0000","last_became_peered":"2023-11-28T13:03:16.540566+0000","last_unstale":"2023-11-28T13:03:17.541432+0000","last_undegraded":"2023-11-28T13:03:17.541432+0000","last_fullsized":"2023-11-28T13:03:17.541432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:48.649554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.248500+0000","last_change":"2023-11-28T13:03:16.556486+0000","last_active":"2023-11-28T13:03:18.248500+0000","last_peered":"2023-11-28T13:03:18.248500+0000","last_clean":"2023-11-28T13:03:18.248500+0000","last_became_active":"2023-11-28T13:03:16.553177+0000","last_became_peered":"2023-11-28T13:03:16.553177+0000","last_unstale":"2023-11-28T13:03:18.248500+0000","last_undegraded":"2023-11-28T13:03:18.248500+0000","last_fullsized":"2023-11-28T13:03:18.248500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:12:05.335714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543445+0000","last_change":"2023-11-28T13:03:16.549455+0000","last_active":"2023-11-28T13:03:17.543445+0000","last_peered":"2023-11-28T13:03:17.543445+0000","last_clean":"2023-11-28T13:03:17.543445+0000","last_became_active":"2023-11-28T13:03:16.544998+0000","last_became_peered":"2023-11-28T13:03:16.544998+0000","last_unstale":"2023-11-28T13:03:17.543445+0000","last_undegraded":"2023-11-28T13:03:17.543445+0000","last_fullsized":"2023-11-28T13:03:17.543445+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:49.601765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.853947+0000","last_change":"2023-11-28T13:03:16.550452+0000","last_active":"2023-11-28T13:03:18.853947+0000","last_peered":"2023-11-28T13:03:18.853947+0000","last_clean":"2023-11-28T13:03:18.853947+0000","last_became_active":"2023-11-28T13:03:16.550256+0000","last_became_peered":"2023-11-28T13:03:16.550256+0000","last_unstale":"2023-11-28T13:03:18.853947+0000","last_undegraded":"2023-11-28T13:03:18.853947+0000","last_fullsized":"2023-11-28T13:03:18.853947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:55:51.571308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854990+0000","last_change":"2023-11-28T13:03:16.554887+0000","last_active":"2023-11-28T13:03:18.854990+0000","last_peered":"2023-11-28T13:03:18.854990+0000","last_clean":"2023-11-28T13:03:18.854990+0000","last_became_active":"2023-11-28T13:03:16.550583+0000","last_became_peered":"2023-11-28T13:03:16.550583+0000","last_unstale":"2023-11-28T13:03:18.854990+0000","last_undegraded":"2023-11-28T13:03:18.854990+0000","last_fullsized":"2023-11-28T13:03:18.854990+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:43.075281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.541673+0000","last_change":"2023-11-28T13:03:16.564862+0000","last_active":"2023-11-28T13:03:17.541673+0000","last_peered":"2023-11-28T13:03:17.541673+0000","last_clean":"2023-11-28T13:03:17.541673+0000","last_became_active":"2023-11-28T13:03:16.564599+0000","last_became_peered":"2023-11-28T13:03:16.564599+0000","last_unstale":"2023-11-28T13:03:17.541673+0000","last_undegraded":"2023-11-28T13:03:17.541673+0000","last_fullsized":"2023-11-28T13:03:17.541673+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:18:03.451223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.248615+0000","last_change":"2023-11-28T13:03:16.559036+0000","last_active":"2023-11-28T13:03:18.248615+0000","last_peered":"2023-11-28T13:03:18.248615+0000","last_clean":"2023-11-28T13:03:18.248615+0000","last_became_active":"2023-11-28T13:03:16.558684+0000","last_became_peered":"2023-11-28T13:03:16.558684+0000","last_unstale":"2023-11-28T13:03:18.248615+0000","last_undegraded":"2023-11-28T13:03:18.248615+0000","last_fullsized":"2023-11-28T13:03:18.248615+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:06:18.402809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.248668+0000","last_change":"2023-11-28T13:03:16.558331+0000","last_active":"2023-11-28T13:03:18.248668+0000","last_peered":"2023-11-28T13:03:18.248668+0000","last_clean":"2023-11-28T13:03:18.248668+0000","last_became_active":"2023-11-28T13:03:16.558035+0000","last_became_peered":"2023-11-28T13:03:16.558035+0000","last_unstale":"2023-11-28T13:03:18.248668+0000","last_undegraded":"2023-11-28T13:03:18.248668+0000","last_fullsized":"2023-11-28T13:03:18.248668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:18.695260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546274+0000","last_change":"2023-11-28T13:03:16.568186+0000","last_active":"2023-11-28T13:03:17.546274+0000","last_peered":"2023-11-28T13:03:17.546274+0000","last_clean":"2023-11-28T13:03:17.546274+0000","last_became_active":"2023-11-28T13:03:16.567882+0000","last_became_peered":"2023-11-28T13:03:16.567882+0000","last_unstale":"2023-11-28T13:03:17.546274+0000","last_undegraded":"2023-11-28T13:03:17.546274+0000","last_fullsized":"2023-11-28T13:03:17.546274+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:06:18.402809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546150+0000","last_change":"2023-11-28T13:03:16.554431+0000","last_active":"2023-11-28T13:03:17.546150+0000","last_peered":"2023-11-28T13:03:17.546150+0000","last_clean":"2023-11-28T13:03:17.546150+0000","last_became_active":"2023-11-28T13:03:16.554125+0000","last_became_peered":"2023-11-28T13:03:16.554125+0000","last_unstale":"2023-11-28T13:03:17.546150+0000","last_undegraded":"2023-11-28T13:03:17.546150+0000","last_fullsized":"2023-11-28T13:03:17.546150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:53.764225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.249019+0000","last_change":"2023-11-28T13:03:16.561373+0000","last_active":"2023-11-28T13:03:18.249019+0000","last_peered":"2023-11-28T13:03:18.249019+0000","last_clean":"2023-11-28T13:03:18.249019+0000","last_became_active":"2023-11-28T13:03:16.561132+0000","last_became_peered":"2023-11-28T13:03:16.561132+0000","last_unstale":"2023-11-28T13:03:18.249019+0000","last_undegraded":"2023-11-28T13:03:18.249019+0000","last_fullsized":"2023-11-28T13:03:18.249019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:58:50.626353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500635+0000","last_change":"2023-11-28T13:03:16.567220+0000","last_active":"2023-11-28T13:03:18.500635+0000","last_peered":"2023-11-28T13:03:18.500635+0000","last_clean":"2023-11-28T13:03:18.500635+0000","last_became_active":"2023-11-28T13:03:16.565411+0000","last_became_peered":"2023-11-28T13:03:16.565411+0000","last_unstale":"2023-11-28T13:03:18.500635+0000","last_undegraded":"2023-11-28T13:03:18.500635+0000","last_fullsized":"2023-11-28T13:03:18.500635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:53.764225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854684+0000","last_change":"2023-11-28T13:03:16.560136+0000","last_active":"2023-11-28T13:03:18.854684+0000","last_peered":"2023-11-28T13:03:18.854684+0000","last_clean":"2023-11-28T13:03:18.854684+0000","last_became_active":"2023-11-28T13:03:16.559874+0000","last_became_peered":"2023-11-28T13:03:16.559874+0000","last_unstale":"2023-11-28T13:03:18.854684+0000","last_undegraded":"2023-11-28T13:03:18.854684+0000","last_fullsized":"2023-11-28T13:03:18.854684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:55:44.213577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.248841+0000","last_change":"2023-11-28T13:03:16.557831+0000","last_active":"2023-11-28T13:03:18.248841+0000","last_peered":"2023-11-28T13:03:18.248841+0000","last_clean":"2023-11-28T13:03:18.248841+0000","last_became_active":"2023-11-28T13:03:16.557328+0000","last_became_peered":"2023-11-28T13:03:16.557328+0000","last_unstale":"2023-11-28T13:03:18.248841+0000","last_undegraded":"2023-11-28T13:03:18.248841+0000","last_fullsized":"2023-11-28T13:03:18.248841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:49.182277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854818+0000","last_change":"2023-11-28T13:03:16.568087+0000","last_active":"2023-11-28T13:03:18.854818+0000","last_peered":"2023-11-28T13:03:18.854818+0000","last_clean":"2023-11-28T13:03:18.854818+0000","last_became_active":"2023-11-28T13:03:16.567805+0000","last_became_peered":"2023-11-28T13:03:16.567805+0000","last_unstale":"2023-11-28T13:03:18.854818+0000","last_undegraded":"2023-11-28T13:03:18.854818+0000","last_fullsized":"2023-11-28T13:03:18.854818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:04.916385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500403+0000","last_change":"2023-11-28T13:03:16.565124+0000","last_active":"2023-11-28T13:03:18.500403+0000","last_peered":"2023-11-28T13:03:18.500403+0000","last_clean":"2023-11-28T13:03:18.500403+0000","last_became_active":"2023-11-28T13:03:16.564425+0000","last_became_peered":"2023-11-28T13:03:16.564425+0000","last_unstale":"2023-11-28T13:03:18.500403+0000","last_undegraded":"2023-11-28T13:03:18.500403+0000","last_fullsized":"2023-11-28T13:03:18.500403+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:12:19.747468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500441+0000","last_change":"2023-11-28T13:03:16.567135+0000","last_active":"2023-11-28T13:03:18.500441+0000","last_peered":"2023-11-28T13:03:18.500441+0000","last_clean":"2023-11-28T13:03:18.500441+0000","last_became_active":"2023-11-28T13:03:16.565648+0000","last_became_peered":"2023-11-28T13:03:16.565648+0000","last_unstale":"2023-11-28T13:03:18.500441+0000","last_undegraded":"2023-11-28T13:03:18.500441+0000","last_fullsized":"2023-11-28T13:03:18.500441+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:34:13.288278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854605+0000","last_change":"2023-11-28T13:03:16.555273+0000","last_active":"2023-11-28T13:03:18.854605+0000","last_peered":"2023-11-28T13:03:18.854605+0000","last_clean":"2023-11-28T13:03:18.854605+0000","last_became_active":"2023-11-28T13:03:16.554790+0000","last_became_peered":"2023-11-28T13:03:16.554790+0000","last_unstale":"2023-11-28T13:03:18.854605+0000","last_undegraded":"2023-11-28T13:03:18.854605+0000","last_fullsized":"2023-11-28T13:03:18.854605+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:36.659086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546498+0000","last_change":"2023-11-28T13:03:16.560030+0000","last_active":"2023-11-28T13:03:17.546498+0000","last_peered":"2023-11-28T13:03:17.546498+0000","last_clean":"2023-11-28T13:03:17.546498+0000","last_became_active":"2023-11-28T13:03:16.559751+0000","last_became_peered":"2023-11-28T13:03:16.559751+0000","last_unstale":"2023-11-28T13:03:17.546498+0000","last_undegraded":"2023-11-28T13:03:17.546498+0000","last_fullsized":"2023-11-28T13:03:17.546498+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:49.182277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"65'4","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546713+0000","last_change":"2023-11-28T13:03:16.121834+0000","last_active":"2023-11-28T13:03:17.546713+0000","last_peered":"2023-11-28T13:03:17.546713+0000","last_clean":"2023-11-28T13:03:17.546713+0000","last_became_active":"2023-11-28T13:03:16.121535+0000","last_became_peered":"2023-11-28T13:03:16.121535+0000","last_unstale":"2023-11-28T13:03:17.546713+0000","last_undegraded":"2023-11-28T13:03:17.546713+0000","last_fullsized":"2023-11-28T13:03:17.546713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:29.071865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543466+0000","last_change":"2023-11-28T13:03:16.548808+0000","last_active":"2023-11-28T13:03:17.543466+0000","last_peered":"2023-11-28T13:03:17.543466+0000","last_clean":"2023-11-28T13:03:17.543466+0000","last_became_active":"2023-11-28T13:03:16.544549+0000","last_became_peered":"2023-11-28T13:03:16.544549+0000","last_unstale":"2023-11-28T13:03:17.543466+0000","last_undegraded":"2023-11-28T13:03:17.543466+0000","last_fullsized":"2023-11-28T13:03:17.543466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:33.963505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"65'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.541720+0000","last_change":"2023-11-28T13:03:16.126787+0000","last_active":"2023-11-28T13:03:17.541720+0000","last_peered":"2023-11-28T13:03:17.541720+0000","last_clean":"2023-11-28T13:03:17.541720+0000","last_became_active":"2023-11-28T13:03:16.126422+0000","last_became_peered":"2023-11-28T13:03:16.126422+0000","last_unstale":"2023-11-28T13:03:17.541720+0000","last_undegraded":"2023-11-28T13:03:17.541720+0000","last_fullsized":"2023-11-28T13:03:17.541720+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:06.960570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854881+0000","last_change":"2023-11-28T13:03:16.562558+0000","last_active":"2023-11-28T13:03:18.854881+0000","last_peered":"2023-11-28T13:03:18.854881+0000","last_clean":"2023-11-28T13:03:18.854881+0000","last_became_active":"2023-11-28T13:03:16.561841+0000","last_became_peered":"2023-11-28T13:03:16.561841+0000","last_unstale":"2023-11-28T13:03:18.854881+0000","last_undegraded":"2023-11-28T13:03:18.854881+0000","last_fullsized":"2023-11-28T13:03:18.854881+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:20.981142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"65'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.248928+0000","last_change":"2023-11-28T13:03:16.132461+0000","last_active":"2023-11-28T13:03:18.248928+0000","last_peered":"2023-11-28T13:03:18.248928+0000","last_clean":"2023-11-28T13:03:18.248928+0000","last_became_active":"2023-11-28T13:03:16.131886+0000","last_became_peered":"2023-11-28T13:03:16.131886+0000","last_unstale":"2023-11-28T13:03:18.248928+0000","last_undegraded":"2023-11-28T13:03:18.248928+0000","last_fullsized":"2023-11-28T13:03:18.248928+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:18.442477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546687+0000","last_change":"2023-11-28T13:03:16.553830+0000","last_active":"2023-11-28T13:03:17.546687+0000","last_peered":"2023-11-28T13:03:17.546687+0000","last_clean":"2023-11-28T13:03:17.546687+0000","last_became_active":"2023-11-28T13:03:16.553308+0000","last_became_peered":"2023-11-28T13:03:16.553308+0000","last_unstale":"2023-11-28T13:03:17.546687+0000","last_undegraded":"2023-11-28T13:03:17.546687+0000","last_fullsized":"2023-11-28T13:03:17.546687+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:20.125179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"65'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.855088+0000","last_change":"2023-11-28T13:03:16.130379+0000","last_active":"2023-11-28T13:03:18.855088+0000","last_peered":"2023-11-28T13:03:18.855088+0000","last_clean":"2023-11-28T13:03:18.855088+0000","last_became_active":"2023-11-28T13:03:16.130068+0000","last_became_peered":"2023-11-28T13:03:16.130068+0000","last_unstale":"2023-11-28T13:03:18.855088+0000","last_undegraded":"2023-11-28T13:03:18.855088+0000","last_fullsized":"2023-11-28T13:03:18.855088+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:25.671895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500473+0000","last_change":"2023-11-28T13:03:16.565370+0000","last_active":"2023-11-28T13:03:18.500473+0000","last_peered":"2023-11-28T13:03:18.500473+0000","last_clean":"2023-11-28T13:03:18.500473+0000","last_became_active":"2023-11-28T13:03:16.564763+0000","last_became_peered":"2023-11-28T13:03:16.564763+0000","last_unstale":"2023-11-28T13:03:18.500473+0000","last_undegraded":"2023-11-28T13:03:18.500473+0000","last_fullsized":"2023-11-28T13:03:18.500473+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:49.182277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"65'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854914+0000","last_change":"2023-11-28T13:03:16.124491+0000","last_active":"2023-11-28T13:03:18.854914+0000","last_peered":"2023-11-28T13:03:18.854914+0000","last_clean":"2023-11-28T13:03:18.854914+0000","last_became_active":"2023-11-28T13:03:16.124045+0000","last_became_peered":"2023-11-28T13:03:16.124045+0000","last_unstale":"2023-11-28T13:03:18.854914+0000","last_undegraded":"2023-11-28T13:03:18.854914+0000","last_fullsized":"2023-11-28T13:03:18.854914+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:51.829397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.248671+0000","last_change":"2023-11-28T13:03:16.562519+0000","last_active":"2023-11-28T13:03:18.248671+0000","last_peered":"2023-11-28T13:03:18.248671+0000","last_clean":"2023-11-28T13:03:18.248671+0000","last_became_active":"2023-11-28T13:03:16.561853+0000","last_became_peered":"2023-11-28T13:03:16.561853+0000","last_unstale":"2023-11-28T13:03:18.248671+0000","last_undegraded":"2023-11-28T13:03:18.248671+0000","last_fullsized":"2023-11-28T13:03:18.248671+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:35:53.764225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"65'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854624+0000","last_change":"2023-11-28T13:03:16.133220+0000","last_active":"2023-11-28T13:03:18.854624+0000","last_peered":"2023-11-28T13:03:18.854624+0000","last_clean":"2023-11-28T13:03:18.854624+0000","last_became_active":"2023-11-28T13:03:16.132948+0000","last_became_peered":"2023-11-28T13:03:16.132948+0000","last_unstale":"2023-11-28T13:03:18.854624+0000","last_undegraded":"2023-11-28T13:03:18.854624+0000","last_fullsized":"2023-11-28T13:03:18.854624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:35:51.266942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.542991+0000","last_change":"2023-11-28T13:03:16.553950+0000","last_active":"2023-11-28T13:03:17.542991+0000","last_peered":"2023-11-28T13:03:17.542991+0000","last_clean":"2023-11-28T13:03:17.542991+0000","last_became_active":"2023-11-28T13:03:16.553711+0000","last_became_peered":"2023-11-28T13:03:16.553711+0000","last_unstale":"2023-11-28T13:03:17.542991+0000","last_undegraded":"2023-11-28T13:03:17.542991+0000","last_fullsized":"2023-11-28T13:03:17.542991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:44:43.842707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.853968+0000","last_change":"2023-11-28T13:03:16.125151+0000","last_active":"2023-11-28T13:03:18.853968+0000","last_peered":"2023-11-28T13:03:18.853968+0000","last_clean":"2023-11-28T13:03:18.853968+0000","last_became_active":"2023-11-28T13:03:16.124718+0000","last_became_peered":"2023-11-28T13:03:16.124718+0000","last_unstale":"2023-11-28T13:03:18.853968+0000","last_undegraded":"2023-11-28T13:03:18.853968+0000","last_fullsized":"2023-11-28T13:03:18.853968+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:33:05.543800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.541484+0000","last_change":"2023-11-28T13:03:16.542429+0000","last_active":"2023-11-28T13:03:17.541484+0000","last_peered":"2023-11-28T13:03:17.541484+0000","last_clean":"2023-11-28T13:03:17.541484+0000","last_became_active":"2023-11-28T13:03:16.541999+0000","last_became_peered":"2023-11-28T13:03:16.541999+0000","last_unstale":"2023-11-28T13:03:17.541484+0000","last_undegraded":"2023-11-28T13:03:17.541484+0000","last_fullsized":"2023-11-28T13:03:17.541484+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:55:55.387588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543015+0000","last_change":"2023-11-28T13:03:16.122223+0000","last_active":"2023-11-28T13:03:17.543015+0000","last_peered":"2023-11-28T13:03:17.543015+0000","last_clean":"2023-11-28T13:03:17.543015+0000","last_became_active":"2023-11-28T13:03:16.121924+0000","last_became_peered":"2023-11-28T13:03:16.121924+0000","last_unstale":"2023-11-28T13:03:17.543015+0000","last_undegraded":"2023-11-28T13:03:17.543015+0000","last_fullsized":"2023-11-28T13:03:17.543015+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:05:02.821061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500712+0000","last_change":"2023-11-28T13:03:16.565684+0000","last_active":"2023-11-28T13:03:18.500712+0000","last_peered":"2023-11-28T13:03:18.500712+0000","last_clean":"2023-11-28T13:03:18.500712+0000","last_became_active":"2023-11-28T13:03:16.565049+0000","last_became_peered":"2023-11-28T13:03:16.565049+0000","last_unstale":"2023-11-28T13:03:18.500712+0000","last_undegraded":"2023-11-28T13:03:18.500712+0000","last_fullsized":"2023-11-28T13:03:18.500712+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:18.695260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.249012+0000","last_change":"2023-11-28T13:03:16.123742+0000","last_active":"2023-11-28T13:03:18.249012+0000","last_peered":"2023-11-28T13:03:18.249012+0000","last_clean":"2023-11-28T13:03:18.249012+0000","last_became_active":"2023-11-28T13:03:16.123251+0000","last_became_peered":"2023-11-28T13:03:16.123251+0000","last_unstale":"2023-11-28T13:03:18.249012+0000","last_undegraded":"2023-11-28T13:03:18.249012+0000","last_fullsized":"2023-11-28T13:03:18.249012+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:04:11.870117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546716+0000","last_change":"2023-11-28T13:03:16.557158+0000","last_active":"2023-11-28T13:03:17.546716+0000","last_peered":"2023-11-28T13:03:17.546716+0000","last_clean":"2023-11-28T13:03:17.546716+0000","last_became_active":"2023-11-28T13:03:16.556661+0000","last_became_peered":"2023-11-28T13:03:16.556661+0000","last_unstale":"2023-11-28T13:03:17.546716+0000","last_undegraded":"2023-11-28T13:03:17.546716+0000","last_fullsized":"2023-11-28T13:03:17.546716+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:12:19.747468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.855146+0000","last_change":"2023-11-28T13:03:16.131787+0000","last_active":"2023-11-28T13:03:18.855146+0000","last_peered":"2023-11-28T13:03:18.855146+0000","last_clean":"2023-11-28T13:03:18.855146+0000","last_became_active":"2023-11-28T13:03:16.131522+0000","last_became_peered":"2023-11-28T13:03:16.131522+0000","last_unstale":"2023-11-28T13:03:18.855146+0000","last_undegraded":"2023-11-28T13:03:18.855146+0000","last_fullsized":"2023-11-28T13:03:18.855146+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:40.138783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854960+0000","last_change":"2023-11-28T13:03:16.562735+0000","last_active":"2023-11-28T13:03:18.854960+0000","last_peered":"2023-11-28T13:03:18.854960+0000","last_clean":"2023-11-28T13:03:18.854960+0000","last_became_active":"2023-11-28T13:03:16.562228+0000","last_became_peered":"2023-11-28T13:03:16.562228+0000","last_unstale":"2023-11-28T13:03:18.854960+0000","last_undegraded":"2023-11-28T13:03:18.854960+0000","last_fullsized":"2023-11-28T13:03:18.854960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:53:48.035997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"65'3","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854721+0000","last_change":"2023-11-28T13:03:16.128454+0000","last_active":"2023-11-28T13:03:18.854721+0000","last_peered":"2023-11-28T13:03:18.854721+0000","last_clean":"2023-11-28T13:03:18.854721+0000","last_became_active":"2023-11-28T13:03:16.128063+0000","last_became_peered":"2023-11-28T13:03:16.128063+0000","last_unstale":"2023-11-28T13:03:18.854721+0000","last_undegraded":"2023-11-28T13:03:18.854721+0000","last_fullsized":"2023-11-28T13:03:18.854721+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:22:06.960570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543249+0000","last_change":"2023-11-28T13:03:16.548230+0000","last_active":"2023-11-28T13:03:17.543249+0000","last_peered":"2023-11-28T13:03:17.543249+0000","last_clean":"2023-11-28T13:03:17.543249+0000","last_became_active":"2023-11-28T13:03:16.547954+0000","last_became_peered":"2023-11-28T13:03:16.547954+0000","last_unstale":"2023-11-28T13:03:17.543249+0000","last_undegraded":"2023-11-28T13:03:17.543249+0000","last_fullsized":"2023-11-28T13:03:17.543249+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:44:26.678144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"65'6","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854356+0000","last_change":"2023-11-28T13:03:16.131951+0000","last_active":"2023-11-28T13:03:18.854356+0000","last_peered":"2023-11-28T13:03:18.854356+0000","last_clean":"2023-11-28T13:03:18.854356+0000","last_became_active":"2023-11-28T13:03:16.131512+0000","last_became_peered":"2023-11-28T13:03:16.131512+0000","last_unstale":"2023-11-28T13:03:18.854356+0000","last_undegraded":"2023-11-28T13:03:18.854356+0000","last_fullsized":"2023-11-28T13:03:18.854356+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:36.336883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546222+0000","last_change":"2023-11-28T13:03:16.556213+0000","last_active":"2023-11-28T13:03:17.546222+0000","last_peered":"2023-11-28T13:03:17.546222+0000","last_clean":"2023-11-28T13:03:17.546222+0000","last_became_active":"2023-11-28T13:03:16.555915+0000","last_became_peered":"2023-11-28T13:03:16.555915+0000","last_unstale":"2023-11-28T13:03:17.546222+0000","last_undegraded":"2023-11-28T13:03:17.546222+0000","last_fullsized":"2023-11-28T13:03:17.546222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:10:57.017619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"65'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.853928+0000","last_change":"2023-11-28T13:03:16.129100+0000","last_active":"2023-11-28T13:03:18.853928+0000","last_peered":"2023-11-28T13:03:18.853928+0000","last_clean":"2023-11-28T13:03:18.853928+0000","last_became_active":"2023-11-28T13:03:16.128596+0000","last_became_peered":"2023-11-28T13:03:16.128596+0000","last_unstale":"2023-11-28T13:03:18.853928+0000","last_undegraded":"2023-11-28T13:03:18.853928+0000","last_fullsized":"2023-11-28T13:03:18.853928+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:24:55.613056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500505+0000","last_change":"2023-11-28T13:03:16.564785+0000","last_active":"2023-11-28T13:03:18.500505+0000","last_peered":"2023-11-28T13:03:18.500505+0000","last_clean":"2023-11-28T13:03:18.500505+0000","last_became_active":"2023-11-28T13:03:16.564494+0000","last_became_peered":"2023-11-28T13:03:16.564494+0000","last_unstale":"2023-11-28T13:03:18.500505+0000","last_undegraded":"2023-11-28T13:03:18.500505+0000","last_fullsized":"2023-11-28T13:03:18.500505+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:12:05.335714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854675+0000","last_change":"2023-11-28T13:03:16.133090+0000","last_active":"2023-11-28T13:03:18.854675+0000","last_peered":"2023-11-28T13:03:18.854675+0000","last_clean":"2023-11-28T13:03:18.854675+0000","last_became_active":"2023-11-28T13:03:16.132775+0000","last_became_peered":"2023-11-28T13:03:16.132775+0000","last_unstale":"2023-11-28T13:03:18.854675+0000","last_undegraded":"2023-11-28T13:03:18.854675+0000","last_fullsized":"2023-11-28T13:03:18.854675+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:08:50.291315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.541331+0000","last_change":"2023-11-28T13:03:16.558877+0000","last_active":"2023-11-28T13:03:17.541331+0000","last_peered":"2023-11-28T13:03:17.541331+0000","last_clean":"2023-11-28T13:03:17.541331+0000","last_became_active":"2023-11-28T13:03:16.558666+0000","last_became_peered":"2023-11-28T13:03:16.558666+0000","last_unstale":"2023-11-28T13:03:17.541331+0000","last_undegraded":"2023-11-28T13:03:17.541331+0000","last_fullsized":"2023-11-28T13:03:17.541331+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:54.233219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500322+0000","last_change":"2023-11-28T13:03:16.127864+0000","last_active":"2023-11-28T13:03:18.500322+0000","last_peered":"2023-11-28T13:03:18.500322+0000","last_clean":"2023-11-28T13:03:18.500322+0000","last_became_active":"2023-11-28T13:03:16.127613+0000","last_became_peered":"2023-11-28T13:03:16.127613+0000","last_unstale":"2023-11-28T13:03:18.500322+0000","last_undegraded":"2023-11-28T13:03:18.500322+0000","last_fullsized":"2023-11-28T13:03:18.500322+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:09.149739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543445+0000","last_change":"2023-11-28T13:03:16.547415+0000","last_active":"2023-11-28T13:03:17.543445+0000","last_peered":"2023-11-28T13:03:17.543445+0000","last_clean":"2023-11-28T13:03:17.543445+0000","last_became_active":"2023-11-28T13:03:16.544639+0000","last_became_peered":"2023-11-28T13:03:16.544639+0000","last_unstale":"2023-11-28T13:03:17.543445+0000","last_undegraded":"2023-11-28T13:03:17.543445+0000","last_fullsized":"2023-11-28T13:03:17.543445+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:34:09.688688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"65'8","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.248575+0000","last_change":"2023-11-28T13:03:16.129682+0000","last_active":"2023-11-28T13:03:18.248575+0000","last_peered":"2023-11-28T13:03:18.248575+0000","last_clean":"2023-11-28T13:03:18.248575+0000","last_became_active":"2023-11-28T13:03:16.129420+0000","last_became_peered":"2023-11-28T13:03:16.129420+0000","last_unstale":"2023-11-28T13:03:18.248575+0000","last_undegraded":"2023-11-28T13:03:18.248575+0000","last_fullsized":"2023-11-28T13:03:18.248575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:43:25.171850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.541615+0000","last_change":"2023-11-28T13:03:16.542228+0000","last_active":"2023-11-28T13:03:17.541615+0000","last_peered":"2023-11-28T13:03:17.541615+0000","last_clean":"2023-11-28T13:03:17.541615+0000","last_became_active":"2023-11-28T13:03:16.541950+0000","last_became_peered":"2023-11-28T13:03:16.541950+0000","last_unstale":"2023-11-28T13:03:17.541615+0000","last_undegraded":"2023-11-28T13:03:17.541615+0000","last_fullsized":"2023-11-28T13:03:17.541615+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:26:04.916385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"65'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.855261+0000","last_change":"2023-11-28T13:03:16.132985+0000","last_active":"2023-11-28T13:03:18.855261+0000","last_peered":"2023-11-28T13:03:18.855261+0000","last_clean":"2023-11-28T13:03:18.855261+0000","last_became_active":"2023-11-28T13:03:16.132748+0000","last_became_peered":"2023-11-28T13:03:16.132748+0000","last_unstale":"2023-11-28T13:03:18.855261+0000","last_undegraded":"2023-11-28T13:03:18.855261+0000","last_fullsized":"2023-11-28T13:03:18.855261+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:08:28.373675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500676+0000","last_change":"2023-11-28T13:03:16.567175+0000","last_active":"2023-11-28T13:03:18.500676+0000","last_peered":"2023-11-28T13:03:18.500676+0000","last_clean":"2023-11-28T13:03:18.500676+0000","last_became_active":"2023-11-28T13:03:16.565430+0000","last_became_peered":"2023-11-28T13:03:16.565430+0000","last_unstale":"2023-11-28T13:03:18.500676+0000","last_undegraded":"2023-11-28T13:03:18.500676+0000","last_fullsized":"2023-11-28T13:03:18.500676+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:06:18.402809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500750+0000","last_change":"2023-11-28T13:03:16.128015+0000","last_active":"2023-11-28T13:03:18.500750+0000","last_peered":"2023-11-28T13:03:18.500750+0000","last_clean":"2023-11-28T13:03:18.500750+0000","last_became_active":"2023-11-28T13:03:16.127670+0000","last_became_peered":"2023-11-28T13:03:16.127670+0000","last_unstale":"2023-11-28T13:03:18.500750+0000","last_undegraded":"2023-11-28T13:03:18.500750+0000","last_fullsized":"2023-11-28T13:03:18.500750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:03:18.442477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546438+0000","last_change":"2023-11-28T13:03:16.550658+0000","last_active":"2023-11-28T13:03:17.546438+0000","last_peered":"2023-11-28T13:03:17.546438+0000","last_clean":"2023-11-28T13:03:17.546438+0000","last_became_active":"2023-11-28T13:03:16.550360+0000","last_became_peered":"2023-11-28T13:03:16.550360+0000","last_unstale":"2023-11-28T13:03:17.546438+0000","last_undegraded":"2023-11-28T13:03:17.546438+0000","last_fullsized":"2023-11-28T13:03:17.546438+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:18.695260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543434+0000","last_change":"2023-11-28T13:03:16.123808+0000","last_active":"2023-11-28T13:03:17.543434+0000","last_peered":"2023-11-28T13:03:17.543434+0000","last_clean":"2023-11-28T13:03:17.543434+0000","last_became_active":"2023-11-28T13:03:16.123521+0000","last_became_peered":"2023-11-28T13:03:16.123521+0000","last_unstale":"2023-11-28T13:03:17.543434+0000","last_undegraded":"2023-11-28T13:03:17.543434+0000","last_fullsized":"2023-11-28T13:03:17.543434+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:25:49.901368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546754+0000","last_change":"2023-11-28T13:03:16.568322+0000","last_active":"2023-11-28T13:03:17.546754+0000","last_peered":"2023-11-28T13:03:17.546754+0000","last_clean":"2023-11-28T13:03:17.546754+0000","last_became_active":"2023-11-28T13:03:16.567910+0000","last_became_peered":"2023-11-28T13:03:16.567910+0000","last_unstale":"2023-11-28T13:03:17.546754+0000","last_undegraded":"2023-11-28T13:03:17.546754+0000","last_fullsized":"2023-11-28T13:03:17.546754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:12:05.335714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.541758+0000","last_change":"2023-11-28T13:03:16.125854+0000","last_active":"2023-11-28T13:03:17.541758+0000","last_peered":"2023-11-28T13:03:17.541758+0000","last_clean":"2023-11-28T13:03:17.541758+0000","last_became_active":"2023-11-28T13:03:16.125635+0000","last_became_peered":"2023-11-28T13:03:16.125635+0000","last_unstale":"2023-11-28T13:03:17.541758+0000","last_undegraded":"2023-11-28T13:03:17.541758+0000","last_fullsized":"2023-11-28T13:03:17.541758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:28:25.714458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543346+0000","last_change":"2023-11-28T13:03:16.544913+0000","last_active":"2023-11-28T13:03:17.543346+0000","last_peered":"2023-11-28T13:03:17.543346+0000","last_clean":"2023-11-28T13:03:17.543346+0000","last_became_active":"2023-11-28T13:03:16.544615+0000","last_became_peered":"2023-11-28T13:03:16.544615+0000","last_unstale":"2023-11-28T13:03:17.543346+0000","last_undegraded":"2023-11-28T13:03:17.543346+0000","last_fullsized":"2023-11-28T13:03:17.543346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:55:12.204868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543134+0000","last_change":"2023-11-28T13:03:16.124237+0000","last_active":"2023-11-28T13:03:17.543134+0000","last_peered":"2023-11-28T13:03:17.543134+0000","last_clean":"2023-11-28T13:03:17.543134+0000","last_became_active":"2023-11-28T13:03:16.123934+0000","last_became_peered":"2023-11-28T13:03:16.123934+0000","last_unstale":"2023-11-28T13:03:17.543134+0000","last_undegraded":"2023-11-28T13:03:17.543134+0000","last_fullsized":"2023-11-28T13:03:17.543134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:13:52.425934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854767+0000","last_change":"2023-11-28T13:03:16.560996+0000","last_active":"2023-11-28T13:03:18.854767+0000","last_peered":"2023-11-28T13:03:18.854767+0000","last_clean":"2023-11-28T13:03:18.854767+0000","last_became_active":"2023-11-28T13:03:16.560738+0000","last_became_peered":"2023-11-28T13:03:16.560738+0000","last_unstale":"2023-11-28T13:03:18.854767+0000","last_undegraded":"2023-11-28T13:03:18.854767+0000","last_fullsized":"2023-11-28T13:03:18.854767+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:09:40.832345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.853880+0000","last_change":"2023-11-28T13:03:16.124942+0000","last_active":"2023-11-28T13:03:18.853880+0000","last_peered":"2023-11-28T13:03:18.853880+0000","last_clean":"2023-11-28T13:03:18.853880+0000","last_became_active":"2023-11-28T13:03:16.124654+0000","last_became_peered":"2023-11-28T13:03:16.124654+0000","last_unstale":"2023-11-28T13:03:18.853880+0000","last_undegraded":"2023-11-28T13:03:18.853880+0000","last_fullsized":"2023-11-28T13:03:18.853880+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:59:35.501873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.853783+0000","last_change":"2023-11-28T13:03:16.551085+0000","last_active":"2023-11-28T13:03:18.853783+0000","last_peered":"2023-11-28T13:03:18.853783+0000","last_clean":"2023-11-28T13:03:18.853783+0000","last_became_active":"2023-11-28T13:03:16.550794+0000","last_became_peered":"2023-11-28T13:03:16.550794+0000","last_unstale":"2023-11-28T13:03:18.853783+0000","last_undegraded":"2023-11-28T13:03:18.853783+0000","last_fullsized":"2023-11-28T13:03:18.853783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:17:35.632742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.248507+0000","last_change":"2023-11-28T13:03:16.129590+0000","last_active":"2023-11-28T13:03:18.248507+0000","last_peered":"2023-11-28T13:03:18.248507+0000","last_clean":"2023-11-28T13:03:18.248507+0000","last_became_active":"2023-11-28T13:03:16.129282+0000","last_became_peered":"2023-11-28T13:03:16.129282+0000","last_unstale":"2023-11-28T13:03:18.248507+0000","last_undegraded":"2023-11-28T13:03:18.248507+0000","last_fullsized":"2023-11-28T13:03:18.248507+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:11:35.652837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854491+0000","last_change":"2023-11-28T13:03:16.555649+0000","last_active":"2023-11-28T13:03:18.854491+0000","last_peered":"2023-11-28T13:03:18.854491+0000","last_clean":"2023-11-28T13:03:18.854491+0000","last_became_active":"2023-11-28T13:03:16.555352+0000","last_became_peered":"2023-11-28T13:03:16.555352+0000","last_unstale":"2023-11-28T13:03:18.854491+0000","last_undegraded":"2023-11-28T13:03:18.854491+0000","last_fullsized":"2023-11-28T13:03:18.854491+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:53:55.017764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854352+0000","last_change":"2023-11-28T13:03:16.130240+0000","last_active":"2023-11-28T13:03:18.854352+0000","last_peered":"2023-11-28T13:03:18.854352+0000","last_clean":"2023-11-28T13:03:18.854352+0000","last_became_active":"2023-11-28T13:03:16.129959+0000","last_became_peered":"2023-11-28T13:03:16.129959+0000","last_unstale":"2023-11-28T13:03:18.854352+0000","last_undegraded":"2023-11-28T13:03:18.854352+0000","last_fullsized":"2023-11-28T13:03:18.854352+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:32.957257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546637+0000","last_change":"2023-11-28T13:03:16.556938+0000","last_active":"2023-11-28T13:03:17.546637+0000","last_peered":"2023-11-28T13:03:17.546637+0000","last_clean":"2023-11-28T13:03:17.546637+0000","last_became_active":"2023-11-28T13:03:16.556332+0000","last_became_peered":"2023-11-28T13:03:16.556332+0000","last_unstale":"2023-11-28T13:03:17.546637+0000","last_undegraded":"2023-11-28T13:03:17.546637+0000","last_fullsized":"2023-11-28T13:03:17.546637+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:34:13.288278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"65'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.249022+0000","last_change":"2023-11-28T13:03:16.132322+0000","last_active":"2023-11-28T13:03:18.249022+0000","last_peered":"2023-11-28T13:03:18.249022+0000","last_clean":"2023-11-28T13:03:18.249022+0000","last_became_active":"2023-11-28T13:03:16.132042+0000","last_became_peered":"2023-11-28T13:03:16.132042+0000","last_unstale":"2023-11-28T13:03:18.249022+0000","last_undegraded":"2023-11-28T13:03:18.249022+0000","last_fullsized":"2023-11-28T13:03:18.249022+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:46:09.149739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:03:16.547537+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:03:16.547537+0000","last_peered":"2023-11-28T13:03:16.547537+0000","last_clean":"2023-11-28T13:03:16.547537+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:03:16.547537+0000","last_undegraded":"2023-11-28T13:03:16.547537+0000","last_fullsized":"2023-11-28T13:03:16.547537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.249039+0000","last_change":"2023-11-28T13:03:16.556975+0000","last_active":"2023-11-28T13:03:18.249039+0000","last_peered":"2023-11-28T13:03:18.249039+0000","last_clean":"2023-11-28T13:03:18.249039+0000","last_became_active":"2023-11-28T13:03:16.556449+0000","last_became_peered":"2023-11-28T13:03:16.556449+0000","last_unstale":"2023-11-28T13:03:18.249039+0000","last_undegraded":"2023-11-28T13:03:18.249039+0000","last_fullsized":"2023-11-28T13:03:18.249039+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:34:13.288278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"65'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.855176+0000","last_change":"2023-11-28T13:03:16.133178+0000","last_active":"2023-11-28T13:03:18.855176+0000","last_peered":"2023-11-28T13:03:18.855176+0000","last_clean":"2023-11-28T13:03:18.855176+0000","last_became_active":"2023-11-28T13:03:16.132809+0000","last_became_peered":"2023-11-28T13:03:16.132809+0000","last_unstale":"2023-11-28T13:03:18.855176+0000","last_undegraded":"2023-11-28T13:03:18.855176+0000","last_fullsized":"2023-11-28T13:03:18.855176+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:40:18.263873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":55,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:03:16.541606+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:03:16.541606+0000","last_peered":"2023-11-28T13:03:16.541606+0000","last_clean":"2023-11-28T13:03:16.541606+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:03:16.541606+0000","last_undegraded":"2023-11-28T13:03:16.541606+0000","last_fullsized":"2023-11-28T13:03:16.541606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854268+0000","last_change":"2023-11-28T13:03:16.558422+0000","last_active":"2023-11-28T13:03:18.854268+0000","last_peered":"2023-11-28T13:03:18.854268+0000","last_clean":"2023-11-28T13:03:18.854268+0000","last_became_active":"2023-11-28T13:03:16.558209+0000","last_became_peered":"2023-11-28T13:03:16.558209+0000","last_unstale":"2023-11-28T13:03:18.854268+0000","last_undegraded":"2023-11-28T13:03:18.854268+0000","last_fullsized":"2023-11-28T13:03:18.854268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:23:54.233219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.541342+0000","last_change":"2023-11-28T13:03:16.126999+0000","last_active":"2023-11-28T13:03:17.541342+0000","last_peered":"2023-11-28T13:03:17.541342+0000","last_clean":"2023-11-28T13:03:17.541342+0000","last_became_active":"2023-11-28T13:03:16.126710+0000","last_became_peered":"2023-11-28T13:03:16.126710+0000","last_unstale":"2023-11-28T13:03:17.541342+0000","last_undegraded":"2023-11-28T13:03:17.541342+0000","last_fullsized":"2023-11-28T13:03:17.541342+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:41:51.829397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:03:16.539469+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:03:16.539469+0000","last_peered":"2023-11-28T13:03:16.539469+0000","last_clean":"2023-11-28T13:03:16.539469+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:03:16.539469+0000","last_undegraded":"2023-11-28T13:03:16.539469+0000","last_fullsized":"2023-11-28T13:03:16.539469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":210,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:03:16.539197+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:03:16.539197+0000","last_peered":"2023-11-28T13:03:16.539197+0000","last_clean":"2023-11-28T13:03:16.539197+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:03:16.539197+0000","last_undegraded":"2023-11-28T13:03:16.539197+0000","last_fullsized":"2023-11-28T13:03:16.539197+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854302+0000","last_change":"2023-11-28T13:03:16.559624+0000","last_active":"2023-11-28T13:03:18.854302+0000","last_peered":"2023-11-28T13:03:18.854302+0000","last_clean":"2023-11-28T13:03:18.854302+0000","last_became_active":"2023-11-28T13:03:16.559379+0000","last_became_peered":"2023-11-28T13:03:16.559379+0000","last_unstale":"2023-11-28T13:03:18.854302+0000","last_undegraded":"2023-11-28T13:03:18.854302+0000","last_fullsized":"2023-11-28T13:03:18.854302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:48.649554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.500384+0000","last_change":"2023-11-28T13:03:16.127004+0000","last_active":"2023-11-28T13:03:18.500384+0000","last_peered":"2023-11-28T13:03:18.500384+0000","last_clean":"2023-11-28T13:03:18.500384+0000","last_became_active":"2023-11-28T13:03:16.126737+0000","last_became_peered":"2023-11-28T13:03:16.126737+0000","last_unstale":"2023-11-28T13:03:18.500384+0000","last_undegraded":"2023-11-28T13:03:18.500384+0000","last_fullsized":"2023-11-28T13:03:18.500384+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:04:11.870117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":61,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:03:16.538643+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:03:16.538643+0000","last_peered":"2023-11-28T13:03:16.538643+0000","last_clean":"2023-11-28T13:03:16.538643+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:03:16.538643+0000","last_undegraded":"2023-11-28T13:03:16.538643+0000","last_fullsized":"2023-11-28T13:03:16.538643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854408+0000","last_change":"2023-11-28T13:03:16.559623+0000","last_active":"2023-11-28T13:03:18.854408+0000","last_peered":"2023-11-28T13:03:18.854408+0000","last_clean":"2023-11-28T13:03:18.854408+0000","last_became_active":"2023-11-28T13:03:16.559349+0000","last_became_peered":"2023-11-28T13:03:16.559349+0000","last_unstale":"2023-11-28T13:03:18.854408+0000","last_undegraded":"2023-11-28T13:03:18.854408+0000","last_fullsized":"2023-11-28T13:03:18.854408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:13:23.649426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"65'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854806+0000","last_change":"2023-11-28T13:03:16.124091+0000","last_active":"2023-11-28T13:03:18.854806+0000","last_peered":"2023-11-28T13:03:18.854806+0000","last_clean":"2023-11-28T13:03:18.854806+0000","last_became_active":"2023-11-28T13:03:16.123873+0000","last_became_peered":"2023-11-28T13:03:16.123873+0000","last_unstale":"2023-11-28T13:03:18.854806+0000","last_undegraded":"2023-11-28T13:03:18.854806+0000","last_fullsized":"2023-11-28T13:03:18.854806+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:48.595517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":61,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:03:16.539082+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:03:16.539082+0000","last_peered":"2023-11-28T13:03:16.539082+0000","last_clean":"2023-11-28T13:03:16.539082+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:03:16.539082+0000","last_undegraded":"2023-11-28T13:03:16.539082+0000","last_fullsized":"2023-11-28T13:03:16.539082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854818+0000","last_change":"2023-11-28T13:03:16.548829+0000","last_active":"2023-11-28T13:03:18.854818+0000","last_peered":"2023-11-28T13:03:18.854818+0000","last_clean":"2023-11-28T13:03:18.854818+0000","last_became_active":"2023-11-28T13:03:16.548579+0000","last_became_peered":"2023-11-28T13:03:16.548579+0000","last_unstale":"2023-11-28T13:03:18.854818+0000","last_undegraded":"2023-11-28T13:03:18.854818+0000","last_fullsized":"2023-11-28T13:03:18.854818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:55:55.387588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854734+0000","last_change":"2023-11-28T13:03:16.132501+0000","last_active":"2023-11-28T13:03:18.854734+0000","last_peered":"2023-11-28T13:03:18.854734+0000","last_clean":"2023-11-28T13:03:18.854734+0000","last_became_active":"2023-11-28T13:03:16.131877+0000","last_became_peered":"2023-11-28T13:03:16.131877+0000","last_unstale":"2023-11-28T13:03:18.854734+0000","last_undegraded":"2023-11-28T13:03:18.854734+0000","last_fullsized":"2023-11-28T13:03:18.854734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:44:42.836459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:03:16.558160+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:03:16.558160+0000","last_peered":"2023-11-28T13:03:16.558160+0000","last_clean":"2023-11-28T13:03:16.558160+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:03:16.558160+0000","last_undegraded":"2023-11-28T13:03:16.558160+0000","last_fullsized":"2023-11-28T13:03:16.558160+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.854267+0000","last_change":"2023-11-28T13:03:16.553098+0000","last_active":"2023-11-28T13:03:18.854267+0000","last_peered":"2023-11-28T13:03:18.854267+0000","last_clean":"2023-11-28T13:03:18.854267+0000","last_became_active":"2023-11-28T13:03:16.552811+0000","last_became_peered":"2023-11-28T13:03:16.552811+0000","last_unstale":"2023-11-28T13:03:18.854267+0000","last_undegraded":"2023-11-28T13:03:18.854267+0000","last_fullsized":"2023-11-28T13:03:18.854267+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:26:09.684739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"65'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:18.855080+0000","last_change":"2023-11-28T13:03:16.130438+0000","last_active":"2023-11-28T13:03:18.855080+0000","last_peered":"2023-11-28T13:03:18.855080+0000","last_clean":"2023-11-28T13:03:18.855080+0000","last_became_active":"2023-11-28T13:03:16.130110+0000","last_became_peered":"2023-11-28T13:03:16.130110+0000","last_unstale":"2023-11-28T13:03:18.855080+0000","last_undegraded":"2023-11-28T13:03:18.855080+0000","last_fullsized":"2023-11-28T13:03:18.855080+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:23:08.369283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:03:16.552747+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:03:16.552747+0000","last_peered":"2023-11-28T13:03:16.552747+0000","last_clean":"2023-11-28T13:03:16.552747+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:03:16.552747+0000","last_undegraded":"2023-11-28T13:03:16.552747+0000","last_fullsized":"2023-11-28T13:03:16.552747+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.543368+0000","last_change":"2023-11-28T13:03:16.560024+0000","last_active":"2023-11-28T13:03:17.543368+0000","last_peered":"2023-11-28T13:03:17.543368+0000","last_clean":"2023-11-28T13:03:17.543368+0000","last_became_active":"2023-11-28T13:03:16.559850+0000","last_became_peered":"2023-11-28T13:03:16.559850+0000","last_unstale":"2023-11-28T13:03:17.543368+0000","last_undegraded":"2023-11-28T13:03:17.543368+0000","last_fullsized":"2023-11-28T13:03:17.543368+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:16.104884+0000","last_clean_scrub_stamp":"2023-11-28T13:03:16.104884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:41.145031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T13:03:17.546537+0000","last_change":"2023-11-28T13:03:16.126583+0000","last_active":"2023-11-28T13:03:17.546537+0000","last_peered":"2023-11-28T13:03:17.546537+0000","last_clean":"2023-11-28T13:03:17.546537+0000","last_became_active":"2023-11-28T13:03:16.126258+0000","last_became_peered":"2023-11-28T13:03:16.126258+0000","last_unstale":"2023-11-28T13:03:17.546537+0000","last_undegraded":"2023-11-28T13:03:17.546537+0000","last_fullsized":"2023-11-28T13:03:17.546537+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:03:15.098635+0000","last_clean_scrub_stamp":"2023-11-28T13:03:15.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:42.069033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T13:03:16.553390+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:03:16.553390+0000","last_peered":"2023-11-28T13:03:16.553390+0000","last_clean":"2023-11-28T13:03:16.553390+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:03:16.553390+0000","last_undegraded":"2023-11-28T13:03:16.553390+0000","last_fullsized":"2023-11-28T13:03:16.553390+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10912,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360928,"statfs":{"total":96636764160,"available":96625590272,"internally_reserved":0,"allocated":1400832,"data_stored":966457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.48399999999999999,"15min":0.45100000000000001},"min":{"1min":0.39200000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.80800000000000005,"5min":0.877,"15min":0.877},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.53100000000000003,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.5110000000000001,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.68100000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.51500000000000001,"15min":0.48699999999999999},"min":{"1min":0.42099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.55200000000000005,"15min":0.52800000000000002},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.51200000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54400000000000004,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.53300000000000003,"15min":0.503},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.61699999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.42699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57899999999999996,"15min":0.57199999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.76000000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.85199999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.64200000000000002,"15min":0.60199999999999998},"min":{"1min":0.34499999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":7.9939999999999998,"5min":7.9939999999999998,"15min":7.9939999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.69599999999999995,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.7350000000000003,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.871}]},{"osd":6,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.133,"5min":4.133,"15min":4.133},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.70199999999999996,"15min":0.66300000000000003},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":8.3000000000000007,"5min":8.3000000000000007,"15min":8.3000000000000007},"last":0.89300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9680,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362160,"statfs":{"total":96636764160,"available":96626851840,"internally_reserved":0,"allocated":794624,"data_stored":368608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.33900000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.37},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.49399999999999999,"15min":0.48199999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.58599999999999997}]},{"osd":1,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.48499999999999999,"15min":0.47499999999999998},"min":{"1min":0.36699999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72799999999999998,"5min":0.87,"15min":0.87},"last":0.42899999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.51500000000000001,"15min":0.50900000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35499999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48199999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.501},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.38900000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.74299999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:02:17 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.626},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.622},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.77100000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10716,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":9344,"kb_avail":94361124,"statfs":{"total":96636764160,"available":96625790976,"internally_reserved":0,"allocated":1396736,"data_stored":960027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9568256},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.315,"15min":0.31},"min":{"1min":0.245,"5min":0.223,"15min":0.223},"max":{"1min":0.53500000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.26900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.47099999999999997,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35099999999999998,"15min":0.34799999999999998},"min":{"1min":0.252,"5min":0.25,"15min":0.25},"max":{"1min":0.57699999999999996,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.38400000000000001,"15min":0.38100000000000001},"min":{"1min":0.29099999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55600000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.309}]},{"osd":3,"last update":"Tue Nov 28 13:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.33600000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.59699999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.378},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.373,"15min":0.37},"min":{"1min":0.25900000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.63700000000000001,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40600000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70899999999999996,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.501},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.46500000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.71999999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61899999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.41299999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73799999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.74199999999999999,"5min":1.157,"15min":1.157},"last":0.48099999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59199999999999997,"15min":0.59699999999999998},"min":{"1min":0.41799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65500000000000003,"15min":0.66500000000000004},"min":{"1min":0.503,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60699999999999998,"15min":0.61299999999999999},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76000000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.745,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.49099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9872,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361968,"statfs":{"total":96636764160,"available":96626655232,"internally_reserved":0,"allocated":794624,"data_stored":374503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33700000000000002,"15min":0.33800000000000002},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.34499999999999997,"15min":0.34100000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.32500000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.372,"15min":0.371},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40500000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.33800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42499999999999999,"15min":0.42799999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.46800000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.87,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.501}]},{"osd":5,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80100000000000005,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56000000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.91900000000000004,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.63400000000000001,"15min":0.63500000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:02:16 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9632,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362208,"statfs":{"total":96636764160,"available":96626900992,"internally_reserved":0,"allocated":811008,"data_stored":377037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.438,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.36599999999999999,"15min":0.36699999999999999},"min":{"1min":0.27700000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56799999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.33200000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51400000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38300000000000001,"15min":0.38100000000000001},"min":{"1min":0.27400000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.56100000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.28699999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.371,"15min":0.36899999999999999},"min":{"1min":0.27400000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52000000000000002,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.317},{"interface":"front","average":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.61599999999999999,"5min":0.748,"15min":0.748},"last":0.376}]},{"osd":4,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73299999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.499,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79300000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.621,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84099999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.622,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63700000000000001,"15min":0.63900000000000001},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81599999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.433},{"interface":"front","average":{"1min":0.624,"5min":0.623,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79600000000000004,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45000000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.86699999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.64100000000000001,"15min":0.64300000000000002},"min":{"1min":0.48199999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.85099999999999998,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.50600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9808,"kb_used_data":776,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362032,"statfs":{"total":96636764160,"available":96626720768,"internally_reserved":0,"allocated":794624,"data_stored":368628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.504,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.35199999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.36099999999999999,"15min":0.36399999999999999},"min":{"1min":0.255,"5min":0.246,"15min":0.246},"max":{"1min":0.45200000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.39800000000000002,"15min":0.40400000000000003},"min":{"1min":0.247,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.58399999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.56399999999999995,"15min":0.59599999999999997},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54700000000000004,"5min":5.0540000000000003,"15min":5.0540000000000003},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.45200000000000001,"15min":0.46300000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.53700000000000003,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85799999999999998,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.626,"15min":0.622},"min":{"1min":0.48199999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84399999999999997,"5min":1.2,"15min":1.2},"last":0.69299999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71999999999999997,"5min":1.169,"15min":1.169},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81999999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67900000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81499999999999995,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":1.23,"15min":1.23},"last":0.66500000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.63,"15min":0.63100000000000001},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77800000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65400000000000003,"15min":0.65500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85199999999999998,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.61199999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9868,"kb_used_data":772,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361972,"statfs":{"total":96636764160,"available":96626659328,"internally_reserved":0,"allocated":790528,"data_stored":368586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.45600000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.39700000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.53700000000000003,"15min":0.52700000000000002},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.749},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.50600000000000001,"15min":0.49399999999999999},"min":{"1min":0.41999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80500000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68000000000000005}]},{"osd":2,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.52900000000000003,"15min":0.51600000000000001},"min":{"1min":0.42899999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.495,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.78600000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.54200000000000004,"15min":0.52900000000000003},"min":{"1min":0.49099999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.71199999999999997}]},{"osd":5,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58299999999999996,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73799999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.439},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.46000000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.497},{"interface":"front","average":{"1min":0.628,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.42599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.81499999999999995,"5min":1.117,"15min":1.117},"last":0.52400000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.755,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9624,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":8832,"kb_avail":94362216,"statfs":{"total":96636764160,"available":96626909184,"internally_reserved":0,"allocated":802816,"data_stored":375579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9043968},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.50800000000000001,"15min":0.49099999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53500000000000003,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79200000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.438},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.57299999999999995}]},{"osd":2,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.39300000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.47399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.59999999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.35999999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.63600000000000001,"15min":0.64300000000000002},"min":{"1min":0.501,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80500000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.51200000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.63500000000000001},"min":{"1min":0.34100000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.73699999999999999,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61699999999999999,"15min":0.65300000000000002},"min":{"1min":0.39900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81100000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55000000000000004,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.67500000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.69199999999999995},"min":{"1min":0.437,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.68899999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77600000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65100000000000002,"15min":0.67300000000000004},"min":{"1min":0.48299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75900000000000001,"5min":1.052,"15min":1.052},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:03:21.071 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T13:03:21.071 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T13:03:21.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.239+0000 7ff68c495700 1 Processor -- start 2023-11-28T13:03:21.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.239+0000 7ff68c495700 1 -- start start 2023-11-28T13:03:21.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68c495700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff68415a640 0x7ff6840c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68c495700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff6840cd140 0x7ff68406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68c495700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6840ae730 con 0x7ff68406d140 2023-11-28T13:03:21.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68c495700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6840cb210 con 0x7ff68415a640 2023-11-28T13:03:21.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68c495700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68406e9e0 con 0x7ff6840cd140 2023-11-28T13:03:21.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68a231700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff68415a640 0x7ff6840c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68aa32700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff68406d140 0x7ff6840cb630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49838/0 (socket says 172.21.15.6:49838) 2023-11-28T13:03:21.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68aa32700 1 -- 172.21.15.6:0/2911776743 learned_addr learned my addr 172.21.15.6:0/2911776743 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:21.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff689a30700 1 --2- 172.21.15.6:0/2911776743 >> v2:172.21.15.61:3300/0 conn(0x7ff6840cd140 0x7ff68406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68a231700 1 -- 172.21.15.6:0/2911776743 >> v2:172.21.15.61:3300/0 conn(0x7ff6840cd140 msgr2=0x7ff68406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:21.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68a231700 1 --2- 172.21.15.6:0/2911776743 >> v2:172.21.15.61:3300/0 conn(0x7ff6840cd140 0x7ff68406b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68a231700 1 -- 172.21.15.6:0/2911776743 >> v1:172.21.15.6:6789/0 conn(0x7ff68406d140 legacy=0x7ff6840cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68a231700 1 -- 172.21.15.6:0/2911776743 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6840ae980 con 0x7ff68415a640 2023-11-28T13:03:21.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff68922f700 1 -- 172.21.15.6:0/2911776743 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565078836 0 0) 0x7ff6840ae730 con 0x7ff68406d140 2023-11-28T13:03:21.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.240+0000 7ff689a30700 1 --2- 172.21.15.6:0/2911776743 >> v2:172.21.15.61:3300/0 conn(0x7ff6840cd140 0x7ff68406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:21.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68a231700 1 --2- 172.21.15.6:0/2911776743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68415a640 0x7ff6840c3700 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff6840d0c50 tx=0x7ff674009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb848a27a72825c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68922f700 1 -- 172.21.15.6:0/2911776743 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674011830 con 0x7ff68415a640 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68922f700 1 -- 172.21.15.6:0/2911776743 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674011990 con 0x7ff68415a640 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68922f700 1 -- 172.21.15.6:0/2911776743 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674011d20 con 0x7ff68415a640 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68c495700 1 -- 172.21.15.6:0/2911776743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68415a640 msgr2=0x7ff6840c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68c495700 1 --2- 172.21.15.6:0/2911776743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68415a640 0x7ff6840c3700 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff6840d0c50 tx=0x7ff674009b20 comp rx=0 tx=0).stop 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68c495700 1 -- 172.21.15.6:0/2911776743 shutdown_connections 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68c495700 1 --2- 172.21.15.6:0/2911776743 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68415a640 0x7ff6840c3700 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68c495700 1 --2- 172.21.15.6:0/2911776743 >> v2:172.21.15.61:3300/0 conn(0x7ff6840cd140 0x7ff68406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68c495700 1 -- 172.21.15.6:0/2911776743 >> 172.21.15.6:0/2911776743 conn(0x7ff6840cc630 msgr2=0x7ff6840cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68c495700 1 -- 172.21.15.6:0/2911776743 shutdown_connections 2023-11-28T13:03:21.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.241+0000 7ff68c495700 1 -- 172.21.15.6:0/2911776743 wait complete. 2023-11-28T13:03:21.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68c495700 1 Processor -- start 2023-11-28T13:03:21.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68c495700 1 -- start start 2023-11-28T13:03:21.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68c495700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68406d140 0x7ff6841f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68c495700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6840cd140 0x7ff6840cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68c495700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff68415a640 0x7ff6840cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68c495700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68415aa10 con 0x7ff68406d140 2023-11-28T13:03:21.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68c495700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68415b220 con 0x7ff6840cd140 2023-11-28T13:03:21.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68c495700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68415ac60 con 0x7ff68415a640 2023-11-28T13:03:21.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68a231700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68406d140 0x7ff6841f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68a231700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68406d140 0x7ff6841f30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34288/0 (socket says 172.21.15.6:34288) 2023-11-28T13:03:21.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.243+0000 7ff68a231700 1 -- 172.21.15.6:0/143829751 learned_addr learned my addr 172.21.15.6:0/143829751 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:21.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff68aa32700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff68415a640 0x7ff6840cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff68a231700 1 -- 172.21.15.6:0/143829751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff68415a640 msgr2=0x7ff6840cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff68a231700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff68415a640 0x7ff6840cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff68a231700 1 -- 172.21.15.6:0/143829751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6840cd140 msgr2=0x7ff6840cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff689a30700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6840cd140 0x7ff6840cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff68a231700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6840cd140 0x7ff6840cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff68a231700 1 -- 172.21.15.6:0/143829751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740094a0 con 0x7ff68406d140 2023-11-28T13:03:21.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff689a30700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6840cd140 0x7ff6840cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:21.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff68a231700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68406d140 0x7ff6841f30c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff6740090b0 tx=0x7ff674004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff67affd700 1 -- 172.21.15.6:0/143829751 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740038a0 con 0x7ff68406d140 2023-11-28T13:03:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff67affd700 1 -- 172.21.15.6:0/143829751 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67401e070 con 0x7ff68406d140 2023-11-28T13:03:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff68c495700 1 -- 172.21.15.6:0/143829751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6840aef40 con 0x7ff68406d140 2023-11-28T13:03:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff67affd700 1 -- 172.21.15.6:0/143829751 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674003c20 con 0x7ff68406d140 2023-11-28T13:03:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.244+0000 7ff68c495700 1 -- 172.21.15.6:0/143829751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6840aec60 con 0x7ff68406d140 2023-11-28T13:03:21.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.245+0000 7ff68c495700 1 -- 172.21.15.6:0/143829751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6840af220 con 0x7ff68406d140 2023-11-28T13:03:21.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.245+0000 7ff67affd700 1 -- 172.21.15.6:0/143829751 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff67400f050 con 0x7ff68406d140 2023-11-28T13:03:21.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.246+0000 7ff67affd700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6680682f0 0x7ff66806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.246+0000 7ff67affd700 1 -- 172.21.15.6:0/143829751 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff674086ec0 con 0x7ff68406d140 2023-11-28T13:03:21.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.246+0000 7ff689a30700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6680682f0 0x7ff66806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.246+0000 7ff689a30700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6680682f0 0x7ff66806a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff6840ac330 tx=0x7ff680008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:21.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.250+0000 7ff67affd700 1 -- 172.21.15.6:0/143829751 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff6740535e0 con 0x7ff68406d140 2023-11-28T13:03:21.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.523+0000 7ff68c495700 1 -- 172.21.15.6:0/143829751 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7ff6840739f0 con 0x7ff68406d140 2023-11-28T13:03:21.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.527+0000 7ff67affd700 1 -- 172.21.15.6:0/143829751 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff6740150e0 con 0x7ff68406d140 2023-11-28T13:03:21.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 -- 172.21.15.6:0/143829751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6680682f0 msgr2=0x7ff66806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6680682f0 0x7ff66806a7a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff6840ac330 tx=0x7ff680008040 comp rx=0 tx=0).stop 2023-11-28T13:03:21.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 -- 172.21.15.6:0/143829751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68406d140 msgr2=0x7ff6841f30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68406d140 0x7ff6841f30c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff6740090b0 tx=0x7ff674004750 comp rx=0 tx=0).stop 2023-11-28T13:03:21.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 -- 172.21.15.6:0/143829751 shutdown_connections 2023-11-28T13:03:21.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff6680682f0 0x7ff66806a7a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff68415a640 0x7ff6840cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff68406d140 0x7ff6841f30c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 --2- 172.21.15.6:0/143829751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6840cd140 0x7ff6840cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 -- 172.21.15.6:0/143829751 >> 172.21.15.6:0/143829751 conn(0x7ff6840cc630 msgr2=0x7ff68406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:21.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 -- 172.21.15.6:0/143829751 shutdown_connections 2023-11-28T13:03:21.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.529+0000 7ff68c495700 1 -- 172.21.15.6:0/143829751 wait complete. 2023-11-28T13:03:21.531 INFO:teuthology.orchestra.run.smithi006.stderr:updated caps for client.0 2023-11-28T13:03:21.542 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:21.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.670+0000 7fd7bf8c5700 1 Processor -- start 2023-11-28T13:03:21.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.670+0000 7fd7bf8c5700 1 -- start start 2023-11-28T13:03:21.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.671+0000 7fd7bf8c5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b815d1e0 0x7fd7b815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.671+0000 7fd7bf8c5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd7b806a560 0x7fd7b806a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.671+0000 7fd7bf8c5700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b80cdec0 con 0x7fd7b80cceb0 2023-11-28T13:03:21.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.671+0000 7fd7bf8c5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b806af10 con 0x7fd7b806a560 2023-11-28T13:03:21.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.671+0000 7fd7bf8c5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80c9cd0 con 0x7fd7b815d1e0 2023-11-28T13:03:21.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.671+0000 7fd7bde62700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd7b80cceb0 0x7fd7b80cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49850/0 (socket says 172.21.15.6:49850) 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.671+0000 7fd7bde62700 1 -- 172.21.15.6:0/2771078727 learned_addr learned my addr 172.21.15.6:0/2771078727 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.671+0000 7fd7bce60700 1 --2- 172.21.15.6:0/2771078727 >> v2:172.21.15.61:3300/0 conn(0x7fd7b806a560 0x7fd7b806a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.671+0000 7fd7bd661700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b815d1e0 0x7fd7b815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419173349 0 0) 0x7fd7b80cdec0 con 0x7fd7b80cceb0 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a40035b0 con 0x7fd7b80cceb0 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656462859 0 0) 0x7fd7a40035b0 con 0x7fd7b80cceb0 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b815d1e0 msgr2=0x7fd7b815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b815d1e0 0x7fd7b815d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 >> v2:172.21.15.61:3300/0 conn(0x7fd7b806a560 msgr2=0x7fd7b806a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 --2- 172.21.15.6:0/2771078727 >> v2:172.21.15.61:3300/0 conn(0x7fd7b806a560 0x7fd7b806a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80c9fb0 con 0x7fd7b80cceb0 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fd7b00019f0 con 0x7fd7b80cceb0 2023-11-28T13:03:21.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 >> v1:172.21.15.6:6789/0 conn(0x7fd7b80cceb0 legacy=0x7fd7b80cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a4003a00 0x7fd7a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7a4004430 0x7fd7a40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b815d1e0 0x7fd7a4004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b806af10 con 0x7fd7a4003a00 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80c9cd0 con 0x7fd7a4004430 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a400ba10 con 0x7fd7b815d1e0 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7bd661700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a4003a00 0x7fd7a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7bde62700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b815d1e0 0x7fd7a4004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7bce60700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7a4004430 0x7fd7a40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7bd661700 1 -- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b815d1e0 msgr2=0x7fd7a4004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7bd661700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b815d1e0 0x7fd7a4004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7bd661700 1 -- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7a4004430 msgr2=0x7fd7a40048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:21.676 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7bd661700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7a4004430 0x7fd7a40048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7bd661700 1 -- 172.21.15.6:0/2771078727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40035b0 con 0x7fd7a4003a00 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.672+0000 7fd7bd661700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a4003a00 0x7fd7a4003e10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009b60 tx=0x7fd7a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabecccd3df77ab6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.673+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800e6c0 con 0x7fd7a4003a00 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.673+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a800e820 con 0x7fd7a4003a00 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.673+0000 7fd7affff700 1 -- 172.21.15.6:0/2771078727 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800eb90 con 0x7fd7a4003a00 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.673+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/2771078727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a4003a00 msgr2=0x7fd7a4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.673+0000 7fd7bf8c5700 1 --2- 172.21.15.6:0/2771078727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7a4003a00 0x7fd7a4003e10 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009b60 tx=0x7fd7a4005880 comp rx=0 tx=0).stop 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.673+0000 7fd7bde62700 1 -- 172.21.15.6:0/2771078727 reap_dead start 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.673+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/2771078727 shutdown_connections 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.673+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/2771078727 >> 172.21.15.6:0/2771078727 conn(0x7fd7b80b0dd0 msgr2=0x7fd7b80b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.673+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/2771078727 shutdown_connections 2023-11-28T13:03:21.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.673+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/2771078727 wait complete. 2023-11-28T13:03:21.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.674+0000 7fd7bf8c5700 1 Processor -- start 2023-11-28T13:03:21.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.674+0000 7fd7bf8c5700 1 -- start start 2023-11-28T13:03:21.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bf8c5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7a4003a00 0x7fd7b81473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bf8c5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b80c5820 0x7fd7b8158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bf8c5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80abcc0 0x7fd7b80ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bf8c5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b8159210 con 0x7fd7b80abcc0 2023-11-28T13:03:21.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bf8c5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80c5c90 con 0x7fd7a4003a00 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bf8c5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b8147a70 con 0x7fd7b80c5820 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bde62700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80abcc0 0x7fd7b80ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bde62700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80abcc0 0x7fd7b80ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34310/0 (socket says 172.21.15.6:34310) 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bde62700 1 -- 172.21.15.6:0/1737326281 learned_addr learned my addr 172.21.15.6:0/1737326281 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bd661700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7a4003a00 0x7fd7b81473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.675+0000 7fd7bce60700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b80c5820 0x7fd7b8158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7bde62700 1 -- 172.21.15.6:0/1737326281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b80c5820 msgr2=0x7fd7b8158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7bde62700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b80c5820 0x7fd7b8158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7bde62700 1 -- 172.21.15.6:0/1737326281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7a4003a00 msgr2=0x7fd7b81473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7bde62700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7a4003a00 0x7fd7b81473e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7bde62700 1 -- 172.21.15.6:0/1737326281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80ca570 con 0x7fd7b80abcc0 2023-11-28T13:03:21.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7bce60700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b80c5820 0x7fd7b8158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7bde62700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80abcc0 0x7fd7b80ac160 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80b1ce0 tx=0x7fd7b0007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7adffb700 1 -- 172.21.15.6:0/1737326281 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0017070 con 0x7fd7b80abcc0 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7adffb700 1 -- 172.21.15.6:0/1737326281 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b000ece0 con 0x7fd7b80abcc0 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/1737326281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b80ca290 con 0x7fd7b80abcc0 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7adffb700 1 -- 172.21.15.6:0/1737326281 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b000ee70 con 0x7fd7b80abcc0 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.676+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/1737326281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b80ca850 con 0x7fd7b80abcc0 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.677+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/1737326281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b80675a0 con 0x7fd7b80abcc0 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.677+0000 7fd7adffb700 1 -- 172.21.15.6:0/1737326281 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd7b0013420 con 0x7fd7b80abcc0 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.678+0000 7fd7adffb700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd79c0682f0 0x7fd79c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.678+0000 7fd7adffb700 1 -- 172.21.15.6:0/1737326281 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd7b0085d80 con 0x7fd7b80abcc0 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.678+0000 7fd7bd661700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd79c0682f0 0x7fd79c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:21.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.678+0000 7fd7bd661700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd79c0682f0 0x7fd79c06a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80090b0 tx=0x7fd7a8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:21.683 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.682+0000 7fd7adffb700 1 -- 172.21.15.6:0/1737326281 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7b0052350 con 0x7fd7b80abcc0 2023-11-28T13:03:21.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.919+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/1737326281 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b80b5260 con 0x7fd7b80abcc0 2023-11-28T13:03:21.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.920+0000 7fd7adffb700 1 -- 172.21.15.6:0/1737326281 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4641 (secure 0 0 0) 0x7fd7b0055970 con 0x7fd7b80abcc0 2023-11-28T13:03:21.920 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:21.920 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1198233336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1198233336},{"type":"v1","addr":"172.21.15.6:6836","nonce":1198233336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/235204249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":235204249},{"type":"v1","addr":"172.21.15.6:6837","nonce":235204249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2675481701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2675481701},{"type":"v1","addr":"172.21.15.61:6833","nonce":2675481701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:03:16.530114+0000","modified":"2023-11-28T13:03:19.551191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6835/3076579711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":3076579711},{"type":"v1","addr":"172.21.15.61:6835","nonce":3076579711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:03:21.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/1737326281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd79c0682f0 msgr2=0x7fd79c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd79c0682f0 0x7fd79c06a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80090b0 tx=0x7fd7a8008030 comp rx=0 tx=0).stop 2023-11-28T13:03:21.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/1737326281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80abcc0 msgr2=0x7fd7b80ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:21.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80abcc0 0x7fd7b80ac160 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80b1ce0 tx=0x7fd7b0007c90 comp rx=0 tx=0).stop 2023-11-28T13:03:21.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/1737326281 shutdown_connections 2023-11-28T13:03:21.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd7b80abcc0 0x7fd7b80ac160 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd7a4003a00 0x7fd7b81473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd79c0682f0 0x7fd79c06a7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 --2- 172.21.15.6:0/1737326281 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd7b80c5820 0x7fd7b8158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:21.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/1737326281 >> 172.21.15.6:0/1737326281 conn(0x7fd7b80b0dd0 msgr2=0x7fd7b815a810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:21.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/1737326281 shutdown_connections 2023-11-28T13:03:21.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:21.922+0000 7fd7bf8c5700 1 -- 172.21.15.6:0/1737326281 wait complete. 2023-11-28T13:03:21.924 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 32 2023-11-28T13:03:22.102 INFO:tasks.ceph.mds.b.smithi061.stderr:2023-11-28T13:03:22.102+0000 7fb5986df700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T13:03:22.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T13:03:16.530114+0000', 'modified': '2023-11-28T13:03:19.551191+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6471}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6471': {'gid': 6471, 'name': 'b', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.61:6835/3076579711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6834', 'nonce': 3076579711}, {'type': 'v1', 'addr': '172.21.15.61:6835', 'nonce': 3076579711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T13:03:22.338 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T13:03:22.338 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T13:03:22.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.473+0000 7f5791125700 1 Processor -- start 2023-11-28T13:03:22.473 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.473+0000 7f5791125700 1 -- start start 2023-11-28T13:03:22.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f5791125700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f578c06aa70 0x7f578c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f5791125700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f578c15c680 0x7f578c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f5791125700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578c06ed80 con 0x7f578c158a30 2023-11-28T13:03:22.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f5791125700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578c06f060 con 0x7f578c15c680 2023-11-28T13:03:22.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f5791125700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578c0cd950 con 0x7f578c06aa70 2023-11-28T13:03:22.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f578b59e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f578c15c680 0x7f578c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f578b59e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f578c15c680 0x7f578c15cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34318/0 (socket says 172.21.15.6:34318) 2023-11-28T13:03:22.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f578b59e700 1 -- 172.21.15.6:0/2906634781 learned_addr learned my addr 172.21.15.6:0/2906634781 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:22.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f578a59c700 1 --2- 172.21.15.6:0/2906634781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f578c06aa70 0x7f578c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f578b59e700 1 -- 172.21.15.6:0/2906634781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f578c06aa70 msgr2=0x7f578c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f578b59e700 1 --2- 172.21.15.6:0/2906634781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f578c06aa70 0x7f578c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f578b59e700 1 -- 172.21.15.6:0/2906634781 >> v1:172.21.15.61:6789/0 conn(0x7f578c158a30 legacy=0x7f578c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f578b59e700 1 -- 172.21.15.6:0/2906634781 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c0cdc30 con 0x7f578c15c680 2023-11-28T13:03:22.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f578b59e700 1 --2- 172.21.15.6:0/2906634781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c15c680 0x7f578c15cb00 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f578400d150 tx=0x7f578400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4840466f46a079cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:22.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.474+0000 7f5789d9b700 1 -- 172.21.15.6:0/2906634781 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784010070 con 0x7f578c15c680 2023-11-28T13:03:22.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.475+0000 7f5789d9b700 1 -- 172.21.15.6:0/2906634781 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578400dd80 con 0x7f578c15c680 2023-11-28T13:03:22.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.475+0000 7f5789d9b700 1 -- 172.21.15.6:0/2906634781 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578400b310 con 0x7f578c15c680 2023-11-28T13:03:22.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.475+0000 7f5791125700 1 -- 172.21.15.6:0/2906634781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c15c680 msgr2=0x7f578c15cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.475+0000 7f5791125700 1 --2- 172.21.15.6:0/2906634781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c15c680 0x7f578c15cb00 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f578400d150 tx=0x7f578400d460 comp rx=0 tx=0).stop 2023-11-28T13:03:22.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.475+0000 7f5791125700 1 -- 172.21.15.6:0/2906634781 shutdown_connections 2023-11-28T13:03:22.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.475+0000 7f5791125700 1 --2- 172.21.15.6:0/2906634781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c15c680 0x7f578c15cb00 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.475+0000 7f5791125700 1 --2- 172.21.15.6:0/2906634781 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f578c06aa70 0x7f578c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.475+0000 7f5791125700 1 -- 172.21.15.6:0/2906634781 >> 172.21.15.6:0/2906634781 conn(0x7f578c0b0dd0 msgr2=0x7f578c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:22.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.475+0000 7f5791125700 1 -- 172.21.15.6:0/2906634781 shutdown_connections 2023-11-28T13:03:22.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.475+0000 7f5791125700 1 -- 172.21.15.6:0/2906634781 wait complete. 2023-11-28T13:03:22.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f5791125700 1 Processor -- start 2023-11-28T13:03:22.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f5791125700 1 -- start start 2023-11-28T13:03:22.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f5791125700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f578c06aa70 0x7f578c1467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f5791125700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c158a30 0x7f578c0ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f5791125700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f578c15c680 0x7f578c153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f5791125700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c0c75b0 con 0x7f578c158a30 2023-11-28T13:03:22.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f5791125700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c151680 con 0x7f578c15c680 2023-11-28T13:03:22.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f5791125700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578c1513a0 con 0x7f578c06aa70 2023-11-28T13:03:22.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f578a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c158a30 0x7f578c0ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f578a59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c158a30 0x7f578c0ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34324/0 (socket says 172.21.15.6:34324) 2023-11-28T13:03:22.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.477+0000 7f578a59c700 1 -- 172.21.15.6:0/1909405677 learned_addr learned my addr 172.21.15.6:0/1909405677 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:22.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f578b59e700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f578c15c680 0x7f578c153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f578ad9d700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f578c06aa70 0x7f578c1467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f578a59c700 1 -- 172.21.15.6:0/1909405677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f578c06aa70 msgr2=0x7f578c1467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.482 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f578a59c700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f578c06aa70 0x7f578c1467d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f578a59c700 1 -- 172.21.15.6:0/1909405677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f578c15c680 msgr2=0x7f578c153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f578a59c700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f578c15c680 0x7f578c153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f578a59c700 1 -- 172.21.15.6:0/1909405677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578400bd40 con 0x7f578c158a30 2023-11-28T13:03:22.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f578a59c700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c158a30 0x7f578c0ca550 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f578000d830 tx=0x7f578000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:22.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f57737fe700 1 -- 172.21.15.6:0/1909405677 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57800125f0 con 0x7f578c158a30 2023-11-28T13:03:22.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f57737fe700 1 -- 172.21.15.6:0/1909405677 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780012750 con 0x7f578c158a30 2023-11-28T13:03:22.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f5791125700 1 -- 172.21.15.6:0/1909405677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578c06f340 con 0x7f578c158a30 2023-11-28T13:03:22.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f57737fe700 1 -- 172.21.15.6:0/1909405677 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780012b00 con 0x7f578c158a30 2023-11-28T13:03:22.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.478+0000 7f5791125700 1 -- 172.21.15.6:0/1909405677 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578c0ceda0 con 0x7f578c158a30 2023-11-28T13:03:22.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.480+0000 7f57737fe700 1 -- 172.21.15.6:0/1909405677 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5780012c60 con 0x7f578c158a30 2023-11-28T13:03:22.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.480+0000 7f57737fe700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57780682f0 0x7f577806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.480+0000 7f57737fe700 1 -- 172.21.15.6:0/1909405677 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5780085ee0 con 0x7f578c158a30 2023-11-28T13:03:22.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.480+0000 7f578ad9d700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57780682f0 0x7f577806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.481+0000 7f578ad9d700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57780682f0 0x7f577806a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f577c007fd0 tx=0x7f577c007ec0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:22.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.482+0000 7f579d2aeb80 1 Processor -- start 2023-11-28T13:03:22.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.482+0000 7f579d2aeb80 1 -- start start 2023-11-28T13:03:22.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de1810 0x562d03db7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03de1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de2120 0x562d03dfd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03ca1bc0 con 0x562d03de1810 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03d10350 con 0x562d03de2120 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d03d09880 con 0x562d03ddf130 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f578ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de1810 0x562d03db7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f578ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de1810 0x562d03db7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34330/0 (socket says 172.21.15.6:34330) 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f578ad9d700 1 -- 172.21.15.6:0/2561105510 learned_addr learned my addr 172.21.15.6:0/2561105510 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f578a59c700 1 --2- 172.21.15.6:0/2561105510 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03de1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f578b59e700 1 --2- 172.21.15.6:0/2561105510 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de2120 0x562d03dfd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f578ad9d700 1 -- 172.21.15.6:0/2561105510 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 msgr2=0x562d03de1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f578ad9d700 1 --2- 172.21.15.6:0/2561105510 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03de1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f578ad9d700 1 -- 172.21.15.6:0/2561105510 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de2120 msgr2=0x562d03dfd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f578ad9d700 1 --2- 172.21.15.6:0/2561105510 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de2120 0x562d03dfd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.483+0000 7f578ad9d700 1 -- 172.21.15.6:0/2561105510 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d03b8b5e0 con 0x562d03de1810 2023-11-28T13:03:22.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f578ad9d700 1 --2- 172.21.15.6:0/2561105510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de1810 0x562d03db7a20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f577c003e30 tx=0x7f577c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17b2469313cf42e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:22.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f5770ff9700 1 -- 172.21.15.6:0/2561105510 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c003e60 con 0x562d03de1810 2023-11-28T13:03:22.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f5770ff9700 1 -- 172.21.15.6:0/2561105510 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c00fd60 con 0x562d03de1810 2023-11-28T13:03:22.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f5770ff9700 1 -- 172.21.15.6:0/2561105510 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c004270 con 0x562d03de1810 2023-11-28T13:03:22.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2561105510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de1810 msgr2=0x562d03db7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/2561105510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de1810 0x562d03db7a20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f577c003e30 tx=0x7f577c012000 comp rx=0 tx=0).stop 2023-11-28T13:03:22.492 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2561105510 shutdown_connections 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/2561105510 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de2120 0x562d03dfd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/2561105510 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de1810 0x562d03db7a20 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/2561105510 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03de1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2561105510 >> 172.21.15.6:0/2561105510 conn(0x562d03db4d60 msgr2=0x562d03db56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.484+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2561105510 shutdown_connections 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.485+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2561105510 wait complete. 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.485+0000 7f579d2aeb80 1 -- start start 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.485+0000 7f579d2aeb80 1 Processor -- start 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03de0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03de3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03de4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03b8d8d0 con 0x562d03de2120 2023-11-28T13:03:22.493 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03df5d90 con 0x562d03de1810 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d03dd07d0 con 0x562d03ddf130 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f579d2aeb80 10 client.0 _ll_register_callbacks cb 0x562d03baf240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578b59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03de4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578b59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03de4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34338/0 (socket says 172.21.15.6:34338) 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578b59e700 1 -- 172.21.15.6:0/2775141954 learned_addr learned my addr 172.21.15.6:0/2775141954 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578ad9d700 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03de0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578a59c700 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03de3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578b59e700 1 -- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 msgr2=0x562d03de0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578b59e700 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03de0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578b59e700 1 -- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 msgr2=0x562d03de3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578b59e700 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03de3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.494 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578b59e700 1 -- 172.21.15.6:0/2775141954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c003b10 con 0x562d03de2120 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578a59c700 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03de3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f578b59e700 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03de4180 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f578400d860 tx=0x7f5784004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41d242479ede5fe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.486+0000 7f5769ffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f5769ffb700 1 -- 172.21.15.6:0/2775141954 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784024070 con 0x562d03de2120 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f5769ffb700 1 -- 172.21.15.6:0/2775141954 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5784004240 con 0x562d03de2120 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2775141954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d03cd7140 con 0x562d03de2120 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f5769ffb700 1 -- 172.21.15.6:0/2775141954 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57840070e0 con 0x562d03de2120 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f579d2aeb80 10 client.6618 fetch_fsmap learned FSMap version 32 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2775141954 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x562d03b37d90 con 0x562d03de2120 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f5769ffb700 1 -- 172.21.15.6:0/2775141954 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562d03cd7140 con 0x562d03de2120 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f5769ffb700 1 -- 172.21.15.6:0/2775141954 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5784004400 con 0x562d03de2120 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f579d2aeb80 10 client.6618 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f579d2aeb80 10 client.6618 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T13:03:22.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f579d2aeb80 20 client.6618 populate_metadata read hostname 'smithi006' 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x562d03dd0110 0x562d03df3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f579d2aeb80 4 client.6618 mds_command: new command op to 6471 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2775141954 --> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d03b8df10 con 0x562d03dd0110 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.487+0000 7f578ad9d700 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x562d03dd0110 0x562d03df3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.488+0000 7f5769ffb700 10 client.6618 ms_handle_connect on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.488+0000 7f578ad9d700 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x562d03dd0110 0x562d03df3eb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca2fd47aebd46a server_cookie=64f80b091d5a7b6a in_seq=0 out_seq=0 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.488+0000 7f5769ffb700 1 -- 172.21.15.6:0/2775141954 <== mds.0 v2:172.21.15.61:6834/3076579711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x562d03b8df10 con 0x562d03dd0110 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.488+0000 7f5769ffb700 10 client.6618 handle_command_reply: tid=0 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.488+0000 7f579d2aeb80 1 client.6618 shutdown 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2775141954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 msgr2=0x562d03de4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03de4180 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f578400d860 tx=0x7f5784004b40 comp rx=0 tx=0).stop 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2775141954 shutdown_connections 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x562d03dd0110 0x562d03df3eb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.496 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03de4180 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03de0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f5769ffb700 0 client.6618 ms_handle_reset on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:22.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/2775141954 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03de3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2775141954 >> 172.21.15.6:0/2775141954 conn(0x562d03db4d60 msgr2=0x562d03db7010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:22.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2775141954 shutdown_connections 2023-11-28T13:03:22.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 1 -- 172.21.15.6:0/2775141954 wait complete. 2023-11-28T13:03:22.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.489+0000 7f579d2aeb80 20 client.6618 trim_cache size 0 max 16384 2023-11-28T13:03:22.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.506+0000 7f579d2aeb80 1 Processor -- start 2023-11-28T13:03:22.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.506+0000 7f579d2aeb80 1 -- start start 2023-11-28T13:03:22.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03db2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03db0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03db0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03e16050 con 0x562d03de2120 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03dd7e30 con 0x562d03de1810 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d03db5790 con 0x562d03ddf130 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578b59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03db0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578b59e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03db0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34346/0 (socket says 172.21.15.6:34346) 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578b59e700 1 -- 172.21.15.6:0/1574257688 learned_addr learned my addr 172.21.15.6:0/1574257688 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578b59e700 1 -- 172.21.15.6:0/1574257688 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 msgr2=0x562d03db2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578ad9d700 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03db2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578a59c700 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03db0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.514 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578b59e700 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03db2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578b59e700 1 -- 172.21.15.6:0/1574257688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 msgr2=0x562d03db0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578b59e700 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03db0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578b59e700 1 -- 172.21.15.6:0/1574257688 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d03cd0140 con 0x562d03de2120 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578ad9d700 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03db2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.507+0000 7f578a59c700 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03db0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f578b59e700 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03db0990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5784004920 tx=0x7f5784024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f1c1c24e0a4af3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f57717fa700 1 -- 172.21.15.6:0/1574257688 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784007160 con 0x562d03de2120 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f57717fa700 1 -- 172.21.15.6:0/1574257688 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5784004240 con 0x562d03de2120 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f57717fa700 1 -- 172.21.15.6:0/1574257688 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784009520 con 0x562d03de2120 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f579d2aeb80 1 -- 172.21.15.6:0/1574257688 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 msgr2=0x562d03db0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03db0990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5784004920 tx=0x7f5784024000 comp rx=0 tx=0).stop 2023-11-28T13:03:22.515 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f579d2aeb80 1 -- 172.21.15.6:0/1574257688 shutdown_connections 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03de2120 0x562d03db0990 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03ddf130 0x562d03db2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/1574257688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03db0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f579d2aeb80 1 -- 172.21.15.6:0/1574257688 >> 172.21.15.6:0/1574257688 conn(0x562d03db4d60 msgr2=0x562d03db8960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f579d2aeb80 1 -- 172.21.15.6:0/1574257688 shutdown_connections 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.508+0000 7f579d2aeb80 1 -- 172.21.15.6:0/1574257688 wait complete. 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.509+0000 7f579d2aeb80 1 -- start start 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.509+0000 7f579d2aeb80 1 Processor -- start 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03ddf130 0x562d03e61b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03e46e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f579d2aeb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03de2120 0x562d03ddf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03de0290 con 0x562d03ddf130 2023-11-28T13:03:22.516 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03df3e80 con 0x562d03de1810 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f579d2aeb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d03e61080 con 0x562d03de2120 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f579d2aeb80 10 client.0 _ll_register_callbacks cb 0x562d03baf240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03ddf130 0x562d03e61b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578ad9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03ddf130 0x562d03e61b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34348/0 (socket says 172.21.15.6:34348) 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578ad9d700 1 -- 172.21.15.6:0/6860936 learned_addr learned my addr 172.21.15.6:0/6860936 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578b59e700 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03de2120 0x562d03ddf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578a59c700 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03e46e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578ad9d700 1 -- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03de2120 msgr2=0x562d03ddf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578ad9d700 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03de2120 0x562d03ddf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578ad9d700 1 -- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 msgr2=0x562d03e46e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578ad9d700 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03e46e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578ad9d700 1 -- 172.21.15.6:0/6860936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d03b37d90 con 0x562d03ddf130 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f578ad9d700 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03ddf130 0x562d03e61b20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f577c003e00 tx=0x7f577c003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1313603471d226 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f5769ffb700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f5769ffb700 1 -- 172.21.15.6:0/6860936 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c031070 con 0x562d03ddf130 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f5769ffb700 1 -- 172.21.15.6:0/6860936 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f577c01c040 con 0x562d03ddf130 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f579d2aeb80 1 -- 172.21.15.6:0/6860936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d03de1fb0 con 0x562d03ddf130 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f5769ffb700 1 -- 172.21.15.6:0/6860936 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f577c014b80 con 0x562d03ddf130 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f579d2aeb80 10 client.6624 fetch_fsmap learned FSMap version 32 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.510+0000 7f579d2aeb80 1 -- 172.21.15.6:0/6860936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x562d03b5e200 con 0x562d03ddf130 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f5769ffb700 1 -- 172.21.15.6:0/6860936 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562d03de1fb0 con 0x562d03ddf130 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f5769ffb700 1 -- 172.21.15.6:0/6860936 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f577c014ce0 con 0x562d03ddf130 2023-11-28T13:03:22.518 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f579d2aeb80 10 client.6624 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:03:22.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f579d2aeb80 10 client.6624 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T13:03:22.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f579d2aeb80 20 client.6624 populate_metadata read hostname 'smithi006' 2023-11-28T13:03:22.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x562d03db0450 0x562d03db0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f579d2aeb80 4 client.6624 mds_command: new command op to 6471 tid=0[{"prefix": "status"}] 2023-11-28T13:03:22.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f579d2aeb80 1 -- 172.21.15.6:0/6860936 --> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d03cc7160 con 0x562d03db0450 2023-11-28T13:03:22.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f578a59c700 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x562d03db0450 0x562d03db0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f5769ffb700 10 client.6624 ms_handle_connect on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:22.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f578a59c700 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x562d03db0450 0x562d03db0820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4c00db506af07b server_cookie=3f3081ff168c1679 in_seq=0 out_seq=0 2023-11-28T13:03:22.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f5769ffb700 1 -- 172.21.15.6:0/6860936 <== mds.0 v2:172.21.15.61:6834/3076579711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x562d03cc7160 con 0x562d03db0450 2023-11-28T13:03:22.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f5769ffb700 10 client.6624 handle_command_reply: tid=0 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.511+0000 7f579d2aeb80 1 client.6624 shutdown 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 1 -- 172.21.15.6:0/6860936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03ddf130 msgr2=0x562d03e61b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03ddf130 0x562d03e61b20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f577c003e00 tx=0x7f577c003780 comp rx=0 tx=0).stop 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 1 -- 172.21.15.6:0/6860936 shutdown_connections 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x562d03db0450 0x562d03db0820 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x562d03de2120 0x562d03ddf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x562d03ddf130 0x562d03e61b20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5769ffb700 0 client.6624 ms_handle_reset on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 1 --2- 172.21.15.6:0/6860936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562d03de1810 0x562d03e46e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 1 -- 172.21.15.6:0/6860936 >> 172.21.15.6:0/6860936 conn(0x562d03db4d60 msgr2=0x562d03df4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 1 -- 172.21.15.6:0/6860936 shutdown_connections 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 1 -- 172.21.15.6:0/6860936 wait complete. 2023-11-28T13:03:22.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f579d2aeb80 20 client.6624 trim_cache size 0 max 16384 2023-11-28T13:03:22.521 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:03:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: "cluster_fsid": "dc643718-e974-4d8a-b829-ff6ced554687", 2023-11-28T13:03:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: "whoami": 0, 2023-11-28T13:03:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 6471, 2023-11-28T13:03:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: "want_state": "up:active", 2023-11-28T13:03:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: "state": "up:active", 2023-11-28T13:03:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs", 2023-11-28T13:03:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: "rank_uptime": 5.9796936010000001, 2023-11-28T13:03:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: "mdsmap_epoch": 32, 2023-11-28T13:03:22.521 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch": 66, 2023-11-28T13:03:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: "osdmap_epoch_barrier": 66, 2023-11-28T13:03:22.522 INFO:teuthology.orchestra.run.smithi006.stdout: "uptime": 11.298215376 2023-11-28T13:03:22.522 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:03:22.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 -- 172.21.15.6:0/1909405677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57780682f0 msgr2=0x7f577806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57780682f0 0x7f577806a7a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f577c007fd0 tx=0x7f577c007ec0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 -- 172.21.15.6:0/1909405677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c158a30 msgr2=0x7f578c0ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c158a30 0x7f578c0ca550 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f578000d830 tx=0x7f578000db40 comp rx=0 tx=0).stop 2023-11-28T13:03:22.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 -- 172.21.15.6:0/1909405677 shutdown_connections 2023-11-28T13:03:22.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f578c15c680 0x7f578c153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f57780682f0 0x7f577806a7a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f578c06aa70 0x7f578c1467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 --2- 172.21.15.6:0/1909405677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f578c158a30 0x7f578c0ca550 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 -- 172.21.15.6:0/1909405677 >> 172.21.15.6:0/1909405677 conn(0x7f578c0b0dd0 msgr2=0x7f578c151a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:22.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 -- 172.21.15.6:0/1909405677 shutdown_connections 2023-11-28T13:03:22.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.512+0000 7f5791125700 1 -- 172.21.15.6:0/1909405677 wait complete. 2023-11-28T13:03:22.526 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T13:03:22.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.650+0000 7f658265c700 1 Processor -- start 2023-11-28T13:03:22.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.650+0000 7f658265c700 1 -- start start 2023-11-28T13:03:22.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.651+0000 7f658265c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c15a230 0x7f657c15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.651+0000 7f658265c700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f657c0cd4a0 0x7f657c0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.653 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.651+0000 7f658265c700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c0d1a30 con 0x7f657c0b4e10 2023-11-28T13:03:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.651+0000 7f658265c700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c15a600 con 0x7f657c0cd4a0 2023-11-28T13:03:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.651+0000 7f658265c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0b5290 con 0x7f657c15a230 2023-11-28T13:03:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.651+0000 7f6580bf9700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f657c0b4e10 0x7f657c160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49912/0 (socket says 172.21.15.6:49912) 2023-11-28T13:03:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.651+0000 7f6580bf9700 1 -- 172.21.15.6:0/904650799 learned_addr learned my addr 172.21.15.6:0/904650799 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.651+0000 7f657b7fe700 1 --2- 172.21.15.6:0/904650799 >> v2:172.21.15.61:3300/0 conn(0x7f657c0cd4a0 0x7f657c0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.651+0000 7f657bfff700 1 --2- 172.21.15.6:0/904650799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c15a230 0x7f657c15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.651+0000 7f657affd700 1 -- 172.21.15.6:0/904650799 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48926075 0 0) 0x7f657c0d1a30 con 0x7f657c0b4e10 2023-11-28T13:03:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.652+0000 7f657affd700 1 -- 172.21.15.6:0/904650799 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65640035b0 con 0x7f657c0b4e10 2023-11-28T13:03:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.652+0000 7f657b7fe700 1 -- 172.21.15.6:0/904650799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c15a230 msgr2=0x7f657c15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.652+0000 7f657b7fe700 1 --2- 172.21.15.6:0/904650799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c15a230 0x7f657c15d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.652+0000 7f657b7fe700 1 -- 172.21.15.6:0/904650799 >> v1:172.21.15.6:6789/0 conn(0x7f657c0b4e10 legacy=0x7f657c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.652+0000 7f657b7fe700 1 -- 172.21.15.6:0/904650799 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c06abb0 con 0x7f657c0cd4a0 2023-11-28T13:03:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.652+0000 7f657b7fe700 1 --2- 172.21.15.6:0/904650799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c0cd4a0 0x7f657c0b48d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6570009840 tx=0x7f6570009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33645917b499c0f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.652+0000 7f657affd700 1 -- 172.21.15.6:0/904650799 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570011830 con 0x7f657c0cd4a0 2023-11-28T13:03:22.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.652+0000 7f657affd700 1 -- 172.21.15.6:0/904650799 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6570011990 con 0x7f657c0cd4a0 2023-11-28T13:03:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.652+0000 7f657affd700 1 -- 172.21.15.6:0/904650799 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570011d00 con 0x7f657c0cd4a0 2023-11-28T13:03:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.653+0000 7f658265c700 1 -- 172.21.15.6:0/904650799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c0cd4a0 msgr2=0x7f657c0b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.653+0000 7f658265c700 1 --2- 172.21.15.6:0/904650799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c0cd4a0 0x7f657c0b48d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6570009840 tx=0x7f6570009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.653+0000 7f658265c700 1 -- 172.21.15.6:0/904650799 shutdown_connections 2023-11-28T13:03:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.653+0000 7f658265c700 1 --2- 172.21.15.6:0/904650799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c15a230 0x7f657c15d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.653+0000 7f658265c700 1 --2- 172.21.15.6:0/904650799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c0cd4a0 0x7f657c0b48d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.653+0000 7f658265c700 1 -- 172.21.15.6:0/904650799 >> 172.21.15.6:0/904650799 conn(0x7f657c0cf8d0 msgr2=0x7f657c0cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:22.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.653+0000 7f658265c700 1 -- 172.21.15.6:0/904650799 shutdown_connections 2023-11-28T13:03:22.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.653+0000 7f658265c700 1 -- 172.21.15.6:0/904650799 wait complete. 2023-11-28T13:03:22.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.654+0000 7f658265c700 1 Processor -- start 2023-11-28T13:03:22.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.654+0000 7f658265c700 1 -- start start 2023-11-28T13:03:22.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f658265c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f657c0b4e10 0x7f657c155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f658265c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c0cd4a0 0x7f657c1f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f658265c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c15a230 0x7f657c153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f658265c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c1545a0 con 0x7f657c0b4e10 2023-11-28T13:03:22.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f658265c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c154880 con 0x7f657c15a230 2023-11-28T13:03:22.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f658265c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c154b60 con 0x7f657c0cd4a0 2023-11-28T13:03:22.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f657bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f657c0b4e10 0x7f657c155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f657bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f657c0b4e10 0x7f657c155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34362/0 (socket says 172.21.15.6:34362) 2023-11-28T13:03:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f657bfff700 1 -- 172.21.15.6:0/1496332089 learned_addr learned my addr 172.21.15.6:0/1496332089 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f657b7fe700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c0cd4a0 0x7f657c1f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f6580bf9700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c15a230 0x7f657c153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f657bfff700 1 -- 172.21.15.6:0/1496332089 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c0cd4a0 msgr2=0x7f657c1f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f657bfff700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c0cd4a0 0x7f657c1f2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f657bfff700 1 -- 172.21.15.6:0/1496332089 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c15a230 msgr2=0x7f657c153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f657bfff700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c15a230 0x7f657c153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.655+0000 7f657bfff700 1 -- 172.21.15.6:0/1496332089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65700094a0 con 0x7f657c0b4e10 2023-11-28T13:03:22.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.656+0000 7f657bfff700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f657c0b4e10 0x7f657c155640 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f656c00e870 tx=0x7f656c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:22.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.656+0000 7f6578ff9700 1 -- 172.21.15.6:0/1496332089 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c009590 con 0x7f657c0b4e10 2023-11-28T13:03:22.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.656+0000 7f6578ff9700 1 -- 172.21.15.6:0/1496332089 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c013680 con 0x7f657c0b4e10 2023-11-28T13:03:22.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.656+0000 7f658265c700 1 -- 172.21.15.6:0/1496332089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c06a560 con 0x7f657c0b4e10 2023-11-28T13:03:22.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.656+0000 7f6578ff9700 1 -- 172.21.15.6:0/1496332089 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c0139f0 con 0x7f657c0b4e10 2023-11-28T13:03:22.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.656+0000 7f658265c700 1 -- 172.21.15.6:0/1496332089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c06a280 con 0x7f657c0b4e10 2023-11-28T13:03:22.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.657+0000 7f658265c700 1 -- 172.21.15.6:0/1496332089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c06a840 con 0x7f657c0b4e10 2023-11-28T13:03:22.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.657+0000 7f6578ff9700 1 -- 172.21.15.6:0/1496332089 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f656c019030 con 0x7f657c0b4e10 2023-11-28T13:03:22.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.658+0000 7f6578ff9700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65680682f0 0x7f656806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:22.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.658+0000 7f6578ff9700 1 -- 172.21.15.6:0/1496332089 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f656c085a90 con 0x7f657c0b4e10 2023-11-28T13:03:22.663 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.658+0000 7f657b7fe700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65680682f0 0x7f656806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:22.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.658+0000 7f657b7fe700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65680682f0 0x7f656806a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6570000c00 tx=0x7f6570008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:22.664 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.662+0000 7f6578ff9700 1 -- 172.21.15.6:0/1496332089 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f656c0521b0 con 0x7f657c0b4e10 2023-11-28T13:03:22.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.899+0000 7f658265c700 1 -- 172.21.15.6:0/1496332089 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f657c155490 con 0x7f657c0b4e10 2023-11-28T13:03:22.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.900+0000 7f6578ff9700 1 -- 172.21.15.6:0/1496332089 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f656c0557d0 con 0x7f657c0b4e10 2023-11-28T13:03:22.900 INFO:teuthology.orchestra.run.smithi006.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T13:03:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 -- 172.21.15.6:0/1496332089 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65680682f0 msgr2=0x7f656806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65680682f0 0x7f656806a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6570000c00 tx=0x7f6570008030 comp rx=0 tx=0).stop 2023-11-28T13:03:22.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 -- 172.21.15.6:0/1496332089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f657c0b4e10 msgr2=0x7f657c155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f657c0b4e10 0x7f657c155640 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f656c00e870 tx=0x7f656c00eb80 comp rx=0 tx=0).stop 2023-11-28T13:03:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 -- 172.21.15.6:0/1496332089 shutdown_connections 2023-11-28T13:03:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f657c15a230 0x7f657c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f657c0b4e10 0x7f657c155640 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f65680682f0 0x7f656806a7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 --2- 172.21.15.6:0/1496332089 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f657c0cd4a0 0x7f657c1f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 -- 172.21.15.6:0/1496332089 >> 172.21.15.6:0/1496332089 conn(0x7f657c0cf8d0 msgr2=0x7f657c15d310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 -- 172.21.15.6:0/1496332089 shutdown_connections 2023-11-28T13:03:22.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:22.902+0000 7f658265c700 1 -- 172.21.15.6:0/1496332089 wait complete. 2023-11-28T13:03:22.915 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T13:03:22.915 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T13:03:22.915 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-28T13:03:22.916 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T13:03:22.916 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T13:03:22.916 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T13:03:22.916 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T13:03:22.916 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T13:03:22.916 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:03:22.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:22.916 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-28T13:03:22.935 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T13:03:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T13:03:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T13:03:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:22.935 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-28T13:03:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:22.936 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:03:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:03:22.936 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:03:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:03:22.936 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T13:03:22.936 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:03:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T13:03:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42160sec preferred_lft 42160sec 2023-11-28T13:03:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link 2023-11-28T13:03:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:22.937 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T13:03:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-28T13:03:22.937 INFO:teuthology.orchestra.run.smithi006.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T13:03:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 8e:02:21:fe:ce:1c brd ff:ff:ff:ff:ff:ff 2023-11-28T13:03:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T13:03:22.937 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::68d7:25ff:fe50:263/64 scope link 2023-11-28T13:03:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:22.938 INFO:teuthology.orchestra.run.smithi006.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:03:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether ba:07:72:c8:31:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:03:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::b807:72ff:fec8:3118/64 scope link 2023-11-28T13:03:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:22.938 INFO:teuthology.orchestra.run.smithi006.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T13:03:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 8e:02:21:fe:ce:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:03:22.938 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::8c02:21ff:fefe:ce1c/64 scope link 2023-11-28T13:03:22.939 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-28T13:03:22.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:22.939 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T13:03:23.001 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-28T13:03:23.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:23.003 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-28T13:03:23.062 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-28T13:03:23.063 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T13:03:23.063 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T13:03:23.063 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T13:03:23.063 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T13:03:23.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:23.064 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-28T13:03:23.064 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-28T13:03:23.064 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T13:03:23.064 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T13:03:23.064 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T13:03:23.064 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-28T13:03:23.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:23.170 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-28T13:03:23.187 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:03:23.188 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:03:23.188 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:03:23.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:23.188 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2023-11-28T13:03:23.257 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T13:03:23.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:03:23.257 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T13:03:23.315 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T13:03:23.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T13:03:23.315 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:03:23.372 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:03:23.440 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:03:23.440 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:03:23.506 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:03:23.508 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:03:23.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:23.509 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:03:23.566 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T13:03:23.566 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T13:03:23.608 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-28T13:03:23.636 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T13:03:23.637 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T13:03:23.698 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:2023-11-28T13:03:23.698+0000 7f5fe053a5c0 -1 init, newargv = 0x5651d97b9790 newargc=15 2023-11-28T13:03:23.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[129574]: starting ceph client 2023-11-28T13:03:23.703 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[129574]: starting fuse 2023-11-28T13:03:23.704 INFO:teuthology.orchestra.run.smithi006.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T13:03:23.705 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T13:03:23.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:23.706 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-28T13:03:23.765 INFO:teuthology.orchestra.run.smithi006.stdout:47 2023-11-28T13:03:23.766 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T13:03:23.766 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:03:23.766 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2023-11-28T13:03:23.766 DEBUG:teuthology.orchestra.run.smithi006:> import re 2023-11-28T13:03:23.766 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:03:23.766 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2023-11-28T13:03:23.766 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:03:23.766 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2023-11-28T13:03:23.766 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2023-11-28T13:03:23.767 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2023-11-28T13:03:23.768 DEBUG:teuthology.orchestra.run.smithi006:> return f 2023-11-28T13:03:23.768 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T13:03:23.768 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:03:23.768 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.0")) 2023-11-28T13:03:23.768 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:03:23.891 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.0.129574.asok 2023-11-28T13:03:24.073 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.129574.asok 2023-11-28T13:03:24.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:24.073 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.129574.asok status 2023-11-28T13:03:24.237 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:03:24.237 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2023-11-28T13:03:24.237 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T13:03:24.237 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T13:03:24.237 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "0", 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "129574", 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 6639, 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2023-11-28T13:03:24.238 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 6639 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3958060792 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-28T13:03:24.239 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3958060792 2023-11-28T13:03:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.6639 192.168.0.1:0/3958060792", 2023-11-28T13:03:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.1:0/3958060792", 2023-11-28T13:03:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2023-11-28T13:03:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 32, 2023-11-28T13:03:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 66, 2023-11-28T13:03:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2023-11-28T13:03:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2023-11-28T13:03:24.240 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2023-11-28T13:03:24.240 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:03:24.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:24.247 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2023-11-28T13:03:24.247 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T13:03:24.264 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2023-11-28T13:03:24.264 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T13:03:24.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:24.265 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T13:03:24.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:24.334 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-28T13:03:24.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:03:24.398 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-28T13:03:24.459 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T13:03:24.459 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-28T13:03:24.459 DEBUG:teuthology.orchestra.run.smithi006:> import stat 2023-11-28T13:03:24.459 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-28T13:03:24.459 DEBUG:teuthology.orchestra.run.smithi006:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-28T13:03:24.460 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-28T13:03:24.568 INFO:teuthology.orchestra.run.smithi006.stdout:1099511627777 2023-11-28T13:03:24.756 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:24.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.921+0000 7fbbffb2f700 1 Processor -- start 2023-11-28T13:03:24.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.922+0000 7fbbffb2f700 1 -- start start 2023-11-28T13:03:24.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.922+0000 7fbbffb2f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbbf8161090 0x7fbbf80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:24.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.922+0000 7fbbffb2f700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf80ce340 con 0x7fbbf815c990 2023-11-28T13:03:24.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.922+0000 7fbbffb2f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf80cebe0 con 0x7fbbf80caeb0 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.922+0000 7fbbffb2f700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80cf780 con 0x7fbbf8161090 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.922+0000 7fbbfd0ca700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbbf8161090 0x7fbbf80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.922+0000 7fbbfd8cb700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fbbf815c990 0x7fbbf80cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49936/0 (socket says 172.21.15.6:49936) 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.923+0000 7fbbfd8cb700 1 -- 172.21.15.6:0/3607090522 learned_addr learned my addr 172.21.15.6:0/3607090522 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.923+0000 7fbbfd0ca700 1 -- 172.21.15.6:0/3607090522 >> v1:172.21.15.61:6789/0 conn(0x7fbbf80caeb0 legacy=0x7fbbf80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.923+0000 7fbbfd0ca700 1 -- 172.21.15.6:0/3607090522 >> v1:172.21.15.6:6789/0 conn(0x7fbbf815c990 legacy=0x7fbbf80cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.923+0000 7fbbfc8c9700 1 -- 172.21.15.6:0/3607090522 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581135414 0 0) 0x7fbbf80ce340 con 0x7fbbf815c990 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.923+0000 7fbbfd0ca700 1 -- 172.21.15.6:0/3607090522 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf80cb3f0 con 0x7fbbf8161090 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.923+0000 7fbbfd0ca700 1 --2- 172.21.15.6:0/3607090522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf8161090 0x7fbbf80ca970 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009840 tx=0x7fbbec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11abf94f9c4a1342 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.923+0000 7fbbfc8c9700 1 -- 172.21.15.6:0/3607090522 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec004230 con 0x7fbbf8161090 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.923+0000 7fbbfc8c9700 1 -- 172.21.15.6:0/3607090522 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec004390 con 0x7fbbf8161090 2023-11-28T13:03:24.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.923+0000 7fbbfc8c9700 1 -- 172.21.15.6:0/3607090522 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec0036a0 con 0x7fbbf8161090 2023-11-28T13:03:24.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.924+0000 7fbbffb2f700 1 -- 172.21.15.6:0/3607090522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf8161090 msgr2=0x7fbbf80ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:24.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.924+0000 7fbbffb2f700 1 --2- 172.21.15.6:0/3607090522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf8161090 0x7fbbf80ca970 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009840 tx=0x7fbbec009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.924+0000 7fbbffb2f700 1 -- 172.21.15.6:0/3607090522 shutdown_connections 2023-11-28T13:03:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.924+0000 7fbbffb2f700 1 --2- 172.21.15.6:0/3607090522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf8161090 0x7fbbf80ca970 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.924+0000 7fbbffb2f700 1 -- 172.21.15.6:0/3607090522 >> 172.21.15.6:0/3607090522 conn(0x7fbbf80b1650 msgr2=0x7fbbf80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.924+0000 7fbbffb2f700 1 -- 172.21.15.6:0/3607090522 shutdown_connections 2023-11-28T13:03:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.924+0000 7fbbffb2f700 1 -- 172.21.15.6:0/3607090522 wait complete. 2023-11-28T13:03:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.925+0000 7fbbffb2f700 1 Processor -- start 2023-11-28T13:03:24.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.925+0000 7fbbffb2f700 1 -- start start 2023-11-28T13:03:24.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.926+0000 7fbbffb2f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf80caeb0 0x7fbbf80b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:24.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.926+0000 7fbbffb2f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbf815c990 0x7fbbf80adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:24.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.926+0000 7fbbffb2f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbf8161090 0x7fbbf80ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:24.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.926+0000 7fbbffb2f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80ac720 con 0x7fbbf80caeb0 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.926+0000 7fbbffb2f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80ae5d0 con 0x7fbbf8161090 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.926+0000 7fbbffb2f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80ab960 con 0x7fbbf815c990 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.926+0000 7fbbfd8cb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf80caeb0 0x7fbbf80b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbfd8cb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf80caeb0 0x7fbbf80b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34390/0 (socket says 172.21.15.6:34390) 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbfd8cb700 1 -- 172.21.15.6:0/1708633823 learned_addr learned my addr 172.21.15.6:0/1708633823 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbfe0cc700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbf8161090 0x7fbbf80ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbfd8cb700 1 -- 172.21.15.6:0/1708633823 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbf815c990 msgr2=0x7fbbf80adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbfd8cb700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbf815c990 0x7fbbf80adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbfd8cb700 1 -- 172.21.15.6:0/1708633823 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbf8161090 msgr2=0x7fbbf80ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbfd8cb700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbf8161090 0x7fbbf80ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:24.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbfd8cb700 1 -- 172.21.15.6:0/1708633823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec0094a0 con 0x7fbbf80caeb0 2023-11-28T13:03:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbfe0cc700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbf8161090 0x7fbbf80ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbfd8cb700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf80caeb0 0x7fbbf80b4ca0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40011e0 tx=0x7fbbf400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbea7fc700 1 -- 172.21.15.6:0/1708633823 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4004c80 con 0x7fbbf80caeb0 2023-11-28T13:03:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbea7fc700 1 -- 172.21.15.6:0/1708633823 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4004e00 con 0x7fbbf80caeb0 2023-11-28T13:03:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbffb2f700 1 -- 172.21.15.6:0/1708633823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf80ce870 con 0x7fbbf80caeb0 2023-11-28T13:03:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbea7fc700 1 -- 172.21.15.6:0/1708633823 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4007310 con 0x7fbbf80caeb0 2023-11-28T13:03:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.927+0000 7fbbffb2f700 1 -- 172.21.15.6:0/1708633823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf80ce590 con 0x7fbbf80caeb0 2023-11-28T13:03:24.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.928+0000 7fbbffb2f700 1 -- 172.21.15.6:0/1708633823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf80cca20 con 0x7fbbf80caeb0 2023-11-28T13:03:24.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.929+0000 7fbbea7fc700 1 -- 172.21.15.6:0/1708633823 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbbf401a020 con 0x7fbbf80caeb0 2023-11-28T13:03:24.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.929+0000 7fbbea7fc700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbbdc0682f0 0x7fbbdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:24.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.929+0000 7fbbea7fc700 1 -- 172.21.15.6:0/1708633823 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbbf4085a20 con 0x7fbbf80caeb0 2023-11-28T13:03:24.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.929+0000 7fbbfd0ca700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbbdc0682f0 0x7fbbdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:24.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.930+0000 7fbbfd0ca700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbbdc0682f0 0x7fbbdc06a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbbf81f5e30 tx=0x7fbbec010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:24.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:24.933+0000 7fbbea7fc700 1 -- 172.21.15.6:0/1708633823 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbbf4052140 con 0x7fbbf80caeb0 2023-11-28T13:03:25.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.174+0000 7fbbffb2f700 1 -- 172.21.15.6:0/1708633823 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbf80675a0 con 0x7fbbf80caeb0 2023-11-28T13:03:25.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.175+0000 7fbbea7fc700 1 -- 172.21.15.6:0/1708633823 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4641 (secure 0 0 0) 0x7fbbf4055760 con 0x7fbbf80caeb0 2023-11-28T13:03:25.176 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:25.176 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1198233336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1198233336},{"type":"v1","addr":"172.21.15.6:6836","nonce":1198233336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/235204249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":235204249},{"type":"v1","addr":"172.21.15.6:6837","nonce":235204249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2675481701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2675481701},{"type":"v1","addr":"172.21.15.61:6833","nonce":2675481701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:03:16.530114+0000","modified":"2023-11-28T13:03:19.551191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.61:6835/3076579711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":3076579711},{"type":"v1","addr":"172.21.15.61:6835","nonce":3076579711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:03:25.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 -- 172.21.15.6:0/1708633823 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbbdc0682f0 msgr2=0x7fbbdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbbdc0682f0 0x7fbbdc06a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbbf81f5e30 tx=0x7fbbec010000 comp rx=0 tx=0).stop 2023-11-28T13:03:25.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 -- 172.21.15.6:0/1708633823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf80caeb0 msgr2=0x7fbbf80b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf80caeb0 0x7fbbf80b4ca0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40011e0 tx=0x7fbbf400cae0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 -- 172.21.15.6:0/1708633823 shutdown_connections 2023-11-28T13:03:25.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbbf8161090 0x7fbbf80ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbf80caeb0 0x7fbbf80b4ca0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbbdc0682f0 0x7fbbdc06a7a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 --2- 172.21.15.6:0/1708633823 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbbf815c990 0x7fbbf80adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 -- 172.21.15.6:0/1708633823 >> 172.21.15.6:0/1708633823 conn(0x7fbbf80b1650 msgr2=0x7fbbf8160250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 -- 172.21.15.6:0/1708633823 shutdown_connections 2023-11-28T13:03:25.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.177+0000 7fbbffb2f700 1 -- 172.21.15.6:0/1708633823 wait complete. 2023-11-28T13:03:25.179 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 32 2023-11-28T13:03:25.190 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2023-11-28T13:03:25.354 INFO:teuthology.orchestra.run.smithi061.stdout:{"message":"","return_code":0} 2023-11-28T13:03:25.354 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T13:03:25.354 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T13:03:25.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.399+0000 7f38e8862080 1 Processor -- start 2023-11-28T13:03:25.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.399+0000 7f38e8862080 1 -- start start 2023-11-28T13:03:25.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38e8862080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa041870 0x55a6fa113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38e8862080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a6fa06dd70 0x55a6fa116cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38e8862080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a6fa1176e0 0x55a6fa088af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38e8862080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6f9fe7000 con 0x55a6fa041870 2023-11-28T13:03:25.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38e8862080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6f9f79fe0 con 0x55a6fa1176e0 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38e8862080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6fa107a40 con 0x55a6fa06dd70 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38de965700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa041870 0x55a6fa113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38de965700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa041870 0x55a6fa113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34404/0 (socket says 172.21.15.6:34404) 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38de965700 1 -- 172.21.15.6:0/486644331 learned_addr learned my addr 172.21.15.6:0/486644331 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38df166700 1 --2- 172.21.15.6:0/486644331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a6fa1176e0 0x55a6fa088af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38de164700 1 --2- 172.21.15.6:0/486644331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a6fa06dd70 0x55a6fa116cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38de965700 1 -- 172.21.15.6:0/486644331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a6fa06dd70 msgr2=0x55a6fa116cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38de965700 1 --2- 172.21.15.6:0/486644331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a6fa06dd70 0x55a6fa116cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38de965700 1 -- 172.21.15.6:0/486644331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a6fa1176e0 msgr2=0x55a6fa088af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38de965700 1 --2- 172.21.15.6:0/486644331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a6fa1176e0 0x55a6fa088af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38de965700 1 -- 172.21.15.6:0/486644331 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d4005d20 con 0x55a6fa041870 2023-11-28T13:03:25.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38df166700 1 --2- 172.21.15.6:0/486644331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a6fa1176e0 0x55a6fa088af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.400+0000 7f38de965700 1 --2- 172.21.15.6:0/486644331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa041870 0x55a6fa113fd0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x55a6fa0889f0 tx=0x7f38d400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d634a2beda82346c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38dd162700 1 -- 172.21.15.6:0/486644331 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40138f0 con 0x55a6fa041870 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38dd162700 1 -- 172.21.15.6:0/486644331 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d4013a50 con 0x55a6fa041870 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38dd162700 1 -- 172.21.15.6:0/486644331 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4013e00 con 0x55a6fa041870 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38e8862080 1 -- 172.21.15.6:0/486644331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa041870 msgr2=0x55a6fa113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38e8862080 1 --2- 172.21.15.6:0/486644331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa041870 0x55a6fa113fd0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x55a6fa0889f0 tx=0x7f38d400d000 comp rx=0 tx=0).stop 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38e8862080 1 -- 172.21.15.6:0/486644331 shutdown_connections 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38e8862080 1 --2- 172.21.15.6:0/486644331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a6fa1176e0 0x55a6fa088af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38e8862080 1 --2- 172.21.15.6:0/486644331 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa041870 0x55a6fa113fd0 secure :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0x55a6fa0889f0 tx=0x7f38d400d000 comp rx=0 tx=0).stop 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38e8862080 1 --2- 172.21.15.6:0/486644331 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a6fa06dd70 0x55a6fa116cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38e8862080 1 -- 172.21.15.6:0/486644331 >> 172.21.15.6:0/486644331 conn(0x55a6fa06c300 msgr2=0x55a6fa110640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38e8862080 1 -- 172.21.15.6:0/486644331 shutdown_connections 2023-11-28T13:03:25.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.401+0000 7f38e8862080 1 -- 172.21.15.6:0/486644331 wait complete. 2023-11-28T13:03:25.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.402+0000 7f38e8862080 1 Processor -- start 2023-11-28T13:03:25.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.402+0000 7f38e8862080 1 -- start start 2023-11-28T13:03:25.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.402+0000 7f38e8862080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a6fa06dd70 0x55a6fa100be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.402+0000 7f38e8862080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a6fa1176e0 0x55a6fa06c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.402+0000 7f38e8862080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa19de30 0x55a6fa06cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38e8862080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fa040bb0 con 0x55a6fa19de30 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38e8862080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6fa19d9a0 con 0x55a6fa06dd70 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38e8862080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6fa1ad220 con 0x55a6fa1176e0 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38df166700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa19de30 0x55a6fa06cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38df166700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa19de30 0x55a6fa06cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34412/0 (socket says 172.21.15.6:34412) 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38df166700 1 -- 172.21.15.6:0/1640754986 learned_addr learned my addr 172.21.15.6:0/1640754986 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38de965700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a6fa06dd70 0x55a6fa100be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38de164700 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a6fa1176e0 0x55a6fa06c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38df166700 1 -- 172.21.15.6:0/1640754986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a6fa1176e0 msgr2=0x55a6fa06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38df166700 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a6fa1176e0 0x55a6fa06c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38df166700 1 -- 172.21.15.6:0/1640754986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a6fa06dd70 msgr2=0x55a6fa100be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38df166700 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a6fa06dd70 0x55a6fa100be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38df166700 1 -- 172.21.15.6:0/1640754986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6f9f92120 con 0x55a6fa19de30 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38df166700 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa19de30 0x55a6fa06cc10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f38d800d4c0 tx=0x7f38d800d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38c7fff700 1 -- 172.21.15.6:0/1640754986 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8013480 con 0x55a6fa19de30 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38c7fff700 1 -- 172.21.15.6:0/1640754986 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d80135e0 con 0x55a6fa19de30 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38e8862080 1 -- 172.21.15.6:0/1640754986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a6f9f85830 con 0x55a6fa19de30 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38c7fff700 1 -- 172.21.15.6:0/1640754986 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8013990 con 0x55a6fa19de30 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.403+0000 7f38e8862080 1 -- 172.21.15.6:0/1640754986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6f9f92360 con 0x55a6fa19de30 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.405+0000 7f38c7fff700 1 -- 172.21.15.6:0/1640754986 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f38d801a030 con 0x55a6fa19de30 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.405+0000 7f38c7fff700 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f38cc066c40 0x7f38cc0690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.405+0000 7f38c7fff700 1 -- 172.21.15.6:0/1640754986 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f38d8083f90 con 0x55a6fa19de30 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.405+0000 7f38de965700 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f38cc066c40 0x7f38cc0690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.405+0000 7f38e8862080 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55a6fa1121c0 0x55a6fa06ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.405+0000 7f38e8862080 1 -- 172.21.15.6:0/1640754986 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x55a6fa112710 con 0x55a6fa1121c0 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38de164700 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55a6fa1121c0 0x55a6fa06ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38de965700 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f38cc066c40 0x7f38cc0690f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55a6fa042250 tx=0x7f38d4008000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38de164700 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55a6fa1121c0 0x55a6fa06ac70 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38de164700 1 -- 172.21.15.6:0/1640754986 <== osd.5 v2:172.21.15.61:6808/103780 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f38d0010530 con 0x55a6fa1121c0 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38e8862080 1 -- 172.21.15.6:0/1640754986 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55a6fa1121c0 msgr2=0x55a6fa06ac70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38e8862080 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55a6fa1121c0 0x55a6fa06ac70 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38e8862080 1 -- 172.21.15.6:0/1640754986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f38cc066c40 msgr2=0x7f38cc0690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38e8862080 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f38cc066c40 0x7f38cc0690f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55a6fa042250 tx=0x7f38d4008000 comp rx=0 tx=0).stop 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38e8862080 1 -- 172.21.15.6:0/1640754986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa19de30 msgr2=0x55a6fa06cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38e8862080 1 --2- 172.21.15.6:0/1640754986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a6fa19de30 0x55a6fa06cc10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f38d800d4c0 tx=0x7f38d800d7d0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38df166700 1 -- 172.21.15.6:0/1640754986 reap_dead start 2023-11-28T13:03:25.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38e8862080 1 -- 172.21.15.6:0/1640754986 shutdown_connections 2023-11-28T13:03:25.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.406+0000 7f38e8862080 1 -- 172.21.15.6:0/1640754986 >> 172.21.15.6:0/1640754986 conn(0x55a6fa06c300 msgr2=0x55a6fa06b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.407+0000 7f38e8862080 1 -- 172.21.15.6:0/1640754986 shutdown_connections 2023-11-28T13:03:25.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.407+0000 7f38e8862080 1 -- 172.21.15.6:0/1640754986 wait complete. 2023-11-28T13:03:25.413 INFO:teuthology.orchestra.run.smithi006.stdout:Xfiledir_test 2023-11-28T13:03:25.414 DEBUG:teuthology.orchestra.run.smithi006:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T13:03:25.470 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:03:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511627777, 2023-11-28T13:03:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "ancestors": [ 2023-11-28T13:03:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "dirino": 1099511627776, 2023-11-28T13:03:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "dname": "file", 2023-11-28T13:03:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 2 2023-11-28T13:03:25.470 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-28T13:03:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "dirino": 1, 2023-11-28T13:03:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "dname": "dir_test", 2023-11-28T13:03:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "version": 4 2023-11-28T13:03:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: ], 2023-11-28T13:03:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "pool": 11, 2023-11-28T13:03:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: "old_pools": [] 2023-11-28T13:03:25.471 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:03:25.471 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:25.472 DEBUG:teuthology.orchestra.run.smithi006:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-28T13:03:25.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.557+0000 7f61c889e080 1 Processor -- start 2023-11-28T13:03:25.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.557+0000 7f61c889e080 1 -- start start 2023-11-28T13:03:25.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61c889e080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac59980880 0x55ac599aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61c889e080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac599acd80 0x55ac599a9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61c889e080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ac59a566f0 0x55ac59a4d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61c889e080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac598b8fe0 con 0x55ac599acd80 2023-11-28T13:03:25.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61c889e080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac59a4c630 con 0x55ac59980880 2023-11-28T13:03:25.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61c889e080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac599aa160 con 0x55ac59a566f0 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61be1a0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac599acd80 0x55ac599a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61be1a0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac599acd80 0x55ac599a9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34428/0 (socket says 172.21.15.6:34428) 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61be1a0700 1 -- 172.21.15.6:0/3500867506 learned_addr learned my addr 172.21.15.6:0/3500867506 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61be9a1700 1 --2- 172.21.15.6:0/3500867506 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac59980880 0x55ac599aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61bf1a2700 1 --2- 172.21.15.6:0/3500867506 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ac59a566f0 0x55ac59a4d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61be1a0700 1 -- 172.21.15.6:0/3500867506 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ac59a566f0 msgr2=0x55ac59a4d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61be1a0700 1 --2- 172.21.15.6:0/3500867506 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ac59a566f0 0x55ac59a4d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61be1a0700 1 -- 172.21.15.6:0/3500867506 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac59980880 msgr2=0x55ac599aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61be1a0700 1 --2- 172.21.15.6:0/3500867506 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac59980880 0x55ac599aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61be1a0700 1 -- 172.21.15.6:0/3500867506 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b0005de0 con 0x55ac599acd80 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.558+0000 7f61bf1a2700 1 --2- 172.21.15.6:0/3500867506 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ac59a566f0 0x55ac59a4d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:25.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61be1a0700 1 --2- 172.21.15.6:0/3500867506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac599acd80 0x55ac599a9510 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f61b000aa00 tx=0x7f61b000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2dd3bf389aa2d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61bd19e700 1 -- 172.21.15.6:0/3500867506 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b001c630 con 0x55ac599acd80 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61bd19e700 1 -- 172.21.15.6:0/3500867506 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b001c790 con 0x55ac599acd80 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61bd19e700 1 -- 172.21.15.6:0/3500867506 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b001cb00 con 0x55ac599acd80 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61c889e080 1 -- 172.21.15.6:0/3500867506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac599acd80 msgr2=0x55ac599a9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61c889e080 1 --2- 172.21.15.6:0/3500867506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac599acd80 0x55ac599a9510 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f61b000aa00 tx=0x7f61b000f000 comp rx=0 tx=0).stop 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61c889e080 1 -- 172.21.15.6:0/3500867506 shutdown_connections 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61c889e080 1 --2- 172.21.15.6:0/3500867506 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ac59a566f0 0x55ac59a4d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61c889e080 1 --2- 172.21.15.6:0/3500867506 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac59980880 0x55ac599aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61c889e080 1 --2- 172.21.15.6:0/3500867506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac599acd80 0x55ac599a9510 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61c889e080 1 -- 172.21.15.6:0/3500867506 >> 172.21.15.6:0/3500867506 conn(0x55ac599ab310 msgr2=0x55ac5997f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61c889e080 1 -- 172.21.15.6:0/3500867506 shutdown_connections 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.559+0000 7f61c889e080 1 -- 172.21.15.6:0/3500867506 wait complete. 2023-11-28T13:03:25.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.560+0000 7f61c889e080 1 Processor -- start 2023-11-28T13:03:25.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.560+0000 7f61c889e080 1 -- start start 2023-11-28T13:03:25.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61c889e080 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac59980880 0x55ac599ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61c889e080 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ac599acd80 0x55ac59a41240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61c889e080 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac59a566f0 0x55ac599a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61c889e080 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac599ad850 con 0x55ac59a566f0 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61c889e080 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac599adb30 con 0x55ac59980880 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61c889e080 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac599ade10 con 0x55ac599acd80 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61bf1a2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac59a566f0 0x55ac599a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61be1a0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ac599acd80 0x55ac59a41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61be9a1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac59980880 0x55ac599ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61bf1a2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac59a566f0 0x55ac599a9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34436/0 (socket says 172.21.15.6:34436) 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61bf1a2700 1 -- 172.21.15.6:0/643430108 learned_addr learned my addr 172.21.15.6:0/643430108 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61bf1a2700 1 -- 172.21.15.6:0/643430108 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ac599acd80 msgr2=0x55ac59a41240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61bf1a2700 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55ac599acd80 0x55ac59a41240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61bf1a2700 1 -- 172.21.15.6:0/643430108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac59980880 msgr2=0x55ac599ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61bf1a2700 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac59980880 0x55ac599ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61bf1a2700 1 -- 172.21.15.6:0/643430108 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac598eb4f0 con 0x55ac59a566f0 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61bf1a2700 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac59a566f0 0x55ac599a9510 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f61b800d410 tx=0x7f61b800d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.561+0000 7f61a7fff700 1 -- 172.21.15.6:0/643430108 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b800de90 con 0x55ac59a566f0 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.562+0000 7f61a7fff700 1 -- 172.21.15.6:0/643430108 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b8003110 con 0x55ac59a566f0 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.562+0000 7f61c889e080 1 -- 172.21.15.6:0/643430108 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ac598d1120 con 0x55ac59a566f0 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.562+0000 7f61a7fff700 1 -- 172.21.15.6:0/643430108 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8006030 con 0x55ac59a566f0 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.562+0000 7f61c889e080 1 -- 172.21.15.6:0/643430108 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac598c4830 con 0x55ac59a566f0 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.563+0000 7f61a7fff700 1 -- 172.21.15.6:0/643430108 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f61b801a030 con 0x55ac59a566f0 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.563+0000 7f61a7fff700 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61ac066c10 0x7f61ac0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.563+0000 7f61a7fff700 1 -- 172.21.15.6:0/643430108 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f61b8084080 con 0x55ac59a566f0 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.563+0000 7f61be9a1700 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61ac066c10 0x7f61ac0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.564+0000 7f61c889e080 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55ac599c49b0 0x55ac59af84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.564+0000 7f61c889e080 1 -- 172.21.15.6:0/643430108 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x55ac59a442b0 con 0x55ac599c49b0 2023-11-28T13:03:25.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.564+0000 7f61be1a0700 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55ac599c49b0 0x55ac59af84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.564+0000 7f61be9a1700 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61ac066c10 0x7f61ac0690c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005400 tx=0x7f61b400c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.564+0000 7f61be1a0700 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55ac599c49b0 0x55ac59af84b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.565+0000 7f61be1a0700 1 -- 172.21.15.6:0/643430108 <== osd.5 v2:172.21.15.61:6808/103780 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v66'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f61b0022020 con 0x55ac599c49b0 2023-11-28T13:03:25.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61c889e080 1 -- 172.21.15.6:0/643430108 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55ac599c49b0 msgr2=0x55ac59af84b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61c889e080 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x55ac599c49b0 0x55ac59af84b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61c889e080 1 -- 172.21.15.6:0/643430108 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61ac066c10 msgr2=0x7f61ac0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61c889e080 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61ac066c10 0x7f61ac0690c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005400 tx=0x7f61b400c040 comp rx=0 tx=0).stop 2023-11-28T13:03:25.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61c889e080 1 -- 172.21.15.6:0/643430108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac59a566f0 msgr2=0x55ac599a9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61c889e080 1 --2- 172.21.15.6:0/643430108 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55ac59a566f0 0x55ac599a9510 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f61b800d410 tx=0x7f61b800d720 comp rx=0 tx=0).stop 2023-11-28T13:03:25.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61bf1a2700 1 -- 172.21.15.6:0/643430108 reap_dead start 2023-11-28T13:03:25.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61c889e080 1 -- 172.21.15.6:0/643430108 shutdown_connections 2023-11-28T13:03:25.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61c889e080 1 -- 172.21.15.6:0/643430108 >> 172.21.15.6:0/643430108 conn(0x55ac599ab310 msgr2=0x55ac599aa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61c889e080 1 -- 172.21.15.6:0/643430108 shutdown_connections 2023-11-28T13:03:25.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.566+0000 7f61c889e080 1 -- 172.21.15.6:0/643430108 wait complete. 2023-11-28T13:03:25.572 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-28T13:03:25.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.699+0000 7f07ba938700 1 Processor -- start 2023-11-28T13:03:25.700 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.699+0000 7f07ba938700 1 -- start start 2023-11-28T13:03:25.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07ba938700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f07b415c560 0x7f07b406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.702 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07ba938700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b40b2660 0x7f07b40ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07ba938700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b40b30b0 con 0x7f07b40cecf0 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07ba938700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b406c020 con 0x7f07b415c560 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07ba938700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b40cf110 con 0x7f07b40b2660 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07b3fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f07b415c560 0x7f07b406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07b8ed5700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f07b40cecf0 0x7f07b4160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:49998/0 (socket says 172.21.15.6:49998) 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07b8ed5700 1 -- 172.21.15.6:0/348832643 learned_addr learned my addr 172.21.15.6:0/348832643 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07b37fe700 1 --2- 172.21.15.6:0/348832643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b40b2660 0x7f07b40ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07b3fff700 1 -- 172.21.15.6:0/348832643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b40b2660 msgr2=0x7f07b40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07b3fff700 1 --2- 172.21.15.6:0/348832643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b40b2660 0x7f07b40ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07b3fff700 1 -- 172.21.15.6:0/348832643 >> v1:172.21.15.6:6789/0 conn(0x7f07b40cecf0 legacy=0x7f07b4160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07b3fff700 1 -- 172.21.15.6:0/348832643 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b40d09c0 con 0x7f07b415c560 2023-11-28T13:03:25.703 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.700+0000 7f07b3fff700 1 --2- 172.21.15.6:0/348832643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b415c560 0x7f07b406f8b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f079c009840 tx=0x7f079c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39df36607359d80e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07b2ffd700 1 -- 172.21.15.6:0/348832643 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c011830 con 0x7f07b415c560 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07b2ffd700 1 -- 172.21.15.6:0/348832643 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c011990 con 0x7f07b415c560 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07b2ffd700 1 -- 172.21.15.6:0/348832643 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c011d20 con 0x7f07b415c560 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07ba938700 1 -- 172.21.15.6:0/348832643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b415c560 msgr2=0x7f07b406f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07ba938700 1 --2- 172.21.15.6:0/348832643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b415c560 0x7f07b406f8b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f079c009840 tx=0x7f079c009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07ba938700 1 -- 172.21.15.6:0/348832643 shutdown_connections 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07ba938700 1 --2- 172.21.15.6:0/348832643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b415c560 0x7f07b406f8b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07ba938700 1 --2- 172.21.15.6:0/348832643 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b40b2660 0x7f07b40ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07ba938700 1 -- 172.21.15.6:0/348832643 >> 172.21.15.6:0/348832643 conn(0x7f07b40cd660 msgr2=0x7f07b40cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07ba938700 1 -- 172.21.15.6:0/348832643 shutdown_connections 2023-11-28T13:03:25.704 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.701+0000 7f07ba938700 1 -- 172.21.15.6:0/348832643 wait complete. 2023-11-28T13:03:25.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.702+0000 7f07ba938700 1 Processor -- start 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.703+0000 7f07ba938700 1 -- start start 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.703+0000 7f07ba938700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b40b2660 0x7f07b40c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.703+0000 7f07ba938700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b40cecf0 0x7f07b40c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.703+0000 7f07ba938700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07b415c560 0x7f07b4156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.703+0000 7f07ba938700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b40c5de0 con 0x7f07b40cecf0 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.703+0000 7f07ba938700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b40c6200 con 0x7f07b415c560 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.703+0000 7f07ba938700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b40c5160 con 0x7f07b40b2660 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.703+0000 7f07b37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b40cecf0 0x7f07b40c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.703+0000 7f07b37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b40cecf0 0x7f07b40c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34450/0 (socket says 172.21.15.6:34450) 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.703+0000 7f07b37fe700 1 -- 172.21.15.6:0/2206937793 learned_addr learned my addr 172.21.15.6:0/2206937793 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.707 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b8ed5700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07b415c560 0x7f07b4156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b3fff700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b40b2660 0x7f07b40c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b37fe700 1 -- 172.21.15.6:0/2206937793 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b40b2660 msgr2=0x7f07b40c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b37fe700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b40b2660 0x7f07b40c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b37fe700 1 -- 172.21.15.6:0/2206937793 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07b415c560 msgr2=0x7f07b4156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b37fe700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07b415c560 0x7f07b4156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b37fe700 1 -- 172.21.15.6:0/2206937793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c0094a0 con 0x7f07b40cecf0 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b37fe700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b40cecf0 0x7f07b40c4710 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f07a400e9b0 tx=0x7f07a400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b0ff9700 1 -- 172.21.15.6:0/2206937793 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4009710 con 0x7f07b40cecf0 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b0ff9700 1 -- 172.21.15.6:0/2206937793 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a400c7f0 con 0x7f07b40cecf0 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07ba938700 1 -- 172.21.15.6:0/2206937793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b40b2af0 con 0x7f07b40cecf0 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07b0ff9700 1 -- 172.21.15.6:0/2206937793 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400cb60 con 0x7f07b40cecf0 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.704+0000 7f07ba938700 1 -- 172.21.15.6:0/2206937793 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b406fdf0 con 0x7f07b40cecf0 2023-11-28T13:03:25.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.705+0000 7f07b0ff9700 1 -- 172.21.15.6:0/2206937793 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f07a4019030 con 0x7f07b40cecf0 2023-11-28T13:03:25.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.706+0000 7f07b0ff9700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07940682f0 0x7f079406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.706+0000 7f07b0ff9700 1 -- 172.21.15.6:0/2206937793 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f07a4085d60 con 0x7f07b40cecf0 2023-11-28T13:03:25.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.706+0000 7f07b3fff700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07940682f0 0x7f079406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.706+0000 7f07b3fff700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07940682f0 0x7f079406a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f079c000c00 tx=0x7f079c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.708+0000 7f07c6ac1b80 1 Processor -- start 2023-11-28T13:03:25.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.708+0000 7f07c6ac1b80 1 -- start start 2023-11-28T13:03:25.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2cf4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2d1ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d3a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2bdec70 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2c4d400 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2c46930 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2d1ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2d1ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34458/0 (socket says 172.21.15.6:34458) 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b37fe700 1 -- 172.21.15.6:0/523845339 learned_addr learned my addr 172.21.15.6:0/523845339 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b3fff700 1 --2- 172.21.15.6:0/523845339 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2cf4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b8ed5700 1 --2- 172.21.15.6:0/523845339 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d3a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b37fe700 1 -- 172.21.15.6:0/523845339 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 msgr2=0x55e4f2d3a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b37fe700 1 --2- 172.21.15.6:0/523845339 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d3a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b37fe700 1 -- 172.21.15.6:0/523845339 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 msgr2=0x55e4f2cf4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b37fe700 1 --2- 172.21.15.6:0/523845339 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2cf4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b37fe700 1 -- 172.21.15.6:0/523845339 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4f2ac8690 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07b37fe700 1 --2- 172.21.15.6:0/523845339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2d1ec90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f07a40842e0 tx=0x7f07a405e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b415c18332a219 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.709+0000 7f07a9ffb700 1 -- 172.21.15.6:0/523845339 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4055830 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07a9ffb700 1 -- 172.21.15.6:0/523845339 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a406d1c0 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07a9ffb700 1 -- 172.21.15.6:0/523845339 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a405dd40 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/523845339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 msgr2=0x55e4f2d1ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/523845339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2d1ec90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f07a40842e0 tx=0x7f07a405e350 comp rx=0 tx=0).stop 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/523845339 shutdown_connections 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/523845339 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d3a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/523845339 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2cf4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/523845339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2d1ec90 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/523845339 >> 172.21.15.6:0/523845339 conn(0x55e4f2cf1e10 msgr2=0x55e4f2cf27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/523845339 shutdown_connections 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.710+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/523845339 wait complete. 2023-11-28T13:03:25.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.711+0000 7f07c6ac1b80 1 -- start start 2023-11-28T13:03:25.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.711+0000 7f07c6ac1b80 1 Processor -- start 2023-11-28T13:03:25.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2d04b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2d20cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d22080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2aca980 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2d05510 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2d05080 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07c6ac1b80 10 client.0 _ll_register_callbacks cb 0x55e4f2aec2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:25.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2d20cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2d20cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34464/0 (socket says 172.21.15.6:34464) 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b37fe700 1 -- 172.21.15.6:0/762385706 learned_addr learned my addr 172.21.15.6:0/762385706 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b8ed5700 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d22080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b3fff700 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2d04b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b37fe700 1 -- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 msgr2=0x55e4f2d22080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b37fe700 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d22080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b37fe700 1 -- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1c1e0 msgr2=0x55e4f2d04b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b37fe700 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2d04b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b37fe700 1 -- 172.21.15.6:0/762385706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b406fdf0 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f07b37fe700 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2d20cc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f07a4056070 tx=0x7f07a400efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e743e90e9ca10d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f0792ffd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.712+0000 7f0792ffd700 1 -- 172.21.15.6:0/762385706 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4052480 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f0792ffd700 1 -- 172.21.15.6:0/762385706 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f07a4051f30 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/762385706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4f2c141f0 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f0792ffd700 1 -- 172.21.15.6:0/762385706 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07a40520f0 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f07c6ac1b80 10 client.6681 fetch_fsmap learned FSMap version 32 2023-11-28T13:03:25.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/762385706 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e4f2a74e40 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f0792ffd700 1 -- 172.21.15.6:0/762385706 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e4f2c141f0 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f0792ffd700 1 -- 172.21.15.6:0/762385706 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f07a4069400 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f07c6ac1b80 10 client.6681 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:03:25.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f07c6ac1b80 10 client.6681 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-11-28T13:03:25.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f07c6ac1b80 20 client.6681 populate_metadata read hostname 'smithi006' 2023-11-28T13:03:25.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55e4f2d21200 0x55e4f2d41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f07c6ac1b80 4 client.6681 mds_command: new command op to 6471 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/762385706 --> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4f2acafc0 con 0x55e4f2d21200 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.713+0000 7f07b3fff700 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55e4f2d21200 0x55e4f2d41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f0792ffd700 10 client.6681 ms_handle_connect on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07b3fff700 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55e4f2d21200 0x55e4f2d41350 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a6706de1a7207c server_cookie=5619fbd12f605dfa in_seq=0 out_seq=0 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f0792ffd700 1 -- 172.21.15.6:0/762385706 <== mds.0 v2:172.21.15.61:6834/3076579711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55e4f2acafc0 con 0x55e4f2d21200 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f0792ffd700 10 client.6681 handle_command_reply: tid=0 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 client.6681 shutdown 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/762385706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1e8c0 msgr2=0x55e4f2d20cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2d20cc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f07a4056070 tx=0x7f07a400efb0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/762385706 shutdown_connections 2023-11-28T13:03:25.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55e4f2d21200 0x55e4f2d41350 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d22080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2d04b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f0792ffd700 0 client.6681 ms_handle_reset on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:25.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/762385706 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2d20cc0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/762385706 >> 172.21.15.6:0/762385706 conn(0x55e4f2cf1e10 msgr2=0x55e4f2cf4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/762385706 shutdown_connections 2023-11-28T13:03:25.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.714+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/762385706 wait complete. 2023-11-28T13:03:25.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.715+0000 7f07c6ac1b80 20 client.6681 trim_cache size 0 max 16384 2023-11-28T13:03:25.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.731+0000 7f07c6ac1b80 1 Processor -- start 2023-11-28T13:03:25.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.731+0000 7f07c6ac1b80 1 -- start start 2023-11-28T13:03:25.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2cef070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2cef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d13690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2d14a60 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2d04d80 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2d3fe90 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07b3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2cef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07b3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2cef070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34470/0 (socket says 172.21.15.6:34470) 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07b3fff700 1 -- 172.21.15.6:0/469375458 learned_addr learned my addr 172.21.15.6:0/469375458 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07b3fff700 1 -- 172.21.15.6:0/469375458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 msgr2=0x55e4f2d13690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07b37fe700 1 --2- 172.21.15.6:0/469375458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2cef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07b8ed5700 1 --2- 172.21.15.6:0/469375458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d13690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07b3fff700 1 --2- 172.21.15.6:0/469375458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d13690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07b3fff700 1 -- 172.21.15.6:0/469375458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 msgr2=0x55e4f2cef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07b3fff700 1 --2- 172.21.15.6:0/469375458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2cef5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.732+0000 7f07b3fff700 1 -- 172.21.15.6:0/469375458 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4f2c0d1f0 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07b3fff700 1 --2- 172.21.15.6:0/469375458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2cef070 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f079c014040 tx=0x7f079c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60252bead0395c26 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07aa7fc700 1 -- 172.21.15.6:0/469375458 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c011700 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07aa7fc700 1 -- 172.21.15.6:0/469375458 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c02c040 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07aa7fc700 1 -- 172.21.15.6:0/469375458 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c032420 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/469375458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 msgr2=0x55e4f2cef070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/469375458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2cef070 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f079c014040 tx=0x7f079c010000 comp rx=0 tx=0).stop 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/469375458 shutdown_connections 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/469375458 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1f1d0 0x55e4f2d13690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/469375458 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1c1e0 0x55e4f2cef070 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/469375458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2cef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/469375458 >> 172.21.15.6:0/469375458 conn(0x55e4f2cf1e10 msgr2=0x55e4f2cf5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/469375458 shutdown_connections 2023-11-28T13:03:25.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.733+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/469375458 wait complete. 2023-11-28T13:03:25.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.734+0000 7f07c6ac1b80 1 -- start start 2023-11-28T13:03:25.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.734+0000 7f07c6ac1b80 1 Processor -- start 2023-11-28T13:03:25.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1c1e0 0x55e4f2d87db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2d2f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.743 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07c6ac1b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1f1d0 0x55e4f2d2f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2d51fd0 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2d52500 con 0x55e4f2d1e8c0 2023-11-28T13:03:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07c6ac1b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4f2d527e0 con 0x55e4f2d1c1e0 2023-11-28T13:03:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07c6ac1b80 10 client.0 _ll_register_callbacks cb 0x55e4f2aec2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b8ed5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1f1d0 0x55e4f2d2f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b3fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1c1e0 0x55e4f2d87db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b8ed5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1f1d0 0x55e4f2d2f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34480/0 (socket says 172.21.15.6:34480) 2023-11-28T13:03:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b8ed5700 1 -- 172.21.15.6:0/1896859500 learned_addr learned my addr 172.21.15.6:0/1896859500 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b37fe700 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2d2f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b8ed5700 1 -- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1c1e0 msgr2=0x55e4f2d87db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b8ed5700 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1c1e0 0x55e4f2d87db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b8ed5700 1 -- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 msgr2=0x55e4f2d2f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b8ed5700 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2d2f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b8ed5700 1 -- 172.21.15.6:0/1896859500 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b40d09c0 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f07b8ed5700 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1f1d0 0x55e4f2d2f5a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005c90 tx=0x7f07ac007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7946e5c45613bcf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.735+0000 7f0792ffd700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f0792ffd700 1 -- 172.21.15.6:0/1896859500 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac017460 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f0792ffd700 1 -- 172.21.15.6:0/1896859500 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f07ac0175c0 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/1896859500 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4f2c04210 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f0792ffd700 1 -- 172.21.15.6:0/1896859500 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07ac017780 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f07c6ac1b80 10 client.6687 fetch_fsmap learned FSMap version 32 2023-11-28T13:03:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/1896859500 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e4f2a9b2b0 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f0792ffd700 1 -- 172.21.15.6:0/1896859500 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e4f2c04210 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f0792ffd700 1 -- 172.21.15.6:0/1896859500 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f07ac0178e0 con 0x55e4f2d1f1d0 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f07c6ac1b80 10 client.6687 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f07c6ac1b80 10 client.6687 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f07c6ac1b80 20 client.6687 populate_metadata read hostname 'smithi006' 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55e4f2d2fc70 0x55e4f2d13690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f07c6ac1b80 4 client.6687 mds_command: new command op to 6471 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/1896859500 --> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55e4f2bee220 con 0x55e4f2d2fc70 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.736+0000 7f07b3fff700 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55e4f2d2fc70 0x55e4f2d13690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.737+0000 7f0792ffd700 10 client.6687 ms_handle_connect on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.737+0000 7f07b3fff700 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55e4f2d2fc70 0x55e4f2d13690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb2eda4a5579498 server_cookie=33ca405a7765a215 in_seq=0 out_seq=0 2023-11-28T13:03:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.737+0000 7f0792ffd700 1 -- 172.21.15.6:0/1896859500 <== mds.0 v2:172.21.15.61:6834/3076579711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e4f2bee220 con 0x55e4f2d2fc70 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.737+0000 7f0792ffd700 10 client.6687 handle_command_reply: tid=0 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.737+0000 7f07c6ac1b80 1 client.6687 shutdown 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/1896859500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1f1d0 msgr2=0x55e4f2d2f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1f1d0 0x55e4f2d2f5a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f07ac005c90 tx=0x7f07ac007520 comp rx=0 tx=0).stop 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/1896859500 shutdown_connections 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55e4f2d2fc70 0x55e4f2d13690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55e4f2d1f1d0 0x55e4f2d2f5a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f0792ffd700 0 client.6687 ms_handle_reset on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55e4f2d1c1e0 0x55e4f2d87db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 1 --2- 172.21.15.6:0/1896859500 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e4f2d1e8c0 0x55e4f2d2f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/1896859500 >> 172.21.15.6:0/1896859500 conn(0x55e4f2cf1e10 msgr2=0x55e4f2cef070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/1896859500 shutdown_connections 2023-11-28T13:03:25.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 1 -- 172.21.15.6:0/1896859500 wait complete. 2023-11-28T13:03:25.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07c6ac1b80 20 client.6687 trim_cache size 0 max 16384 2023-11-28T13:03:25.748 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:03:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "return_code": 0, 2023-11-28T13:03:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "scrub_tag": "62c162ed-305e-4815-9559-16db3a07f334", 2023-11-28T13:03:25.748 INFO:teuthology.orchestra.run.smithi006.stdout: "mode": "asynchronous" 2023-11-28T13:03:25.748 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:03:25.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 -- 172.21.15.6:0/2206937793 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07940682f0 msgr2=0x7f079406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07940682f0 0x7f079406a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f079c000c00 tx=0x7f079c01c040 comp rx=0 tx=0).stop 2023-11-28T13:03:25.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 -- 172.21.15.6:0/2206937793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b40cecf0 msgr2=0x7f07b40c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b40cecf0 0x7f07b40c4710 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f07a400e9b0 tx=0x7f07a400ed70 comp rx=0 tx=0).stop 2023-11-28T13:03:25.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 -- 172.21.15.6:0/2206937793 shutdown_connections 2023-11-28T13:03:25.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f07b415c560 0x7f07b4156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f07940682f0 0x7f079406a7a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f07b40b2660 0x7f07b40c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 --2- 172.21.15.6:0/2206937793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f07b40cecf0 0x7f07b40c4710 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 -- 172.21.15.6:0/2206937793 >> 172.21.15.6:0/2206937793 conn(0x7f07b40cd660 msgr2=0x7f07b40b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.738+0000 7f07ba938700 1 -- 172.21.15.6:0/2206937793 shutdown_connections 2023-11-28T13:03:25.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.739+0000 7f07ba938700 1 -- 172.21.15.6:0/2206937793 wait complete. 2023-11-28T13:03:25.751 INFO:tasks.ceph.mds.b.smithi061.stderr:2023-11-28T13:03:25.737+0000 7fb594ed8700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6639=0-4194304@1} caps={6639=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6639 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5569f24e5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T13:03:25.752 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T13:03:25.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.877+0000 7fe8db14e700 1 Processor -- start 2023-11-28T13:03:25.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.877+0000 7fe8db14e700 1 -- start start 2023-11-28T13:03:25.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.878+0000 7fe8db14e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe8d415a640 0x7fe8d40c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.878+0000 7fe8db14e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 0x7fe8d406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.878+0000 7fe8db14e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d40ae730 con 0x7fe8d406d140 2023-11-28T13:03:25.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.878+0000 7fe8db14e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d40cb210 con 0x7fe8d415a640 2023-11-28T13:03:25.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.878+0000 7fe8db14e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d406e9e0 con 0x7fe8d40cd140 2023-11-28T13:03:25.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.878+0000 7fe8d8eea700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe8d415a640 0x7fe8d40c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.878+0000 7fe8d8eea700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe8d415a640 0x7fe8d40c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34484/0 (socket says 172.21.15.6:34484) 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.878+0000 7fe8d8eea700 1 -- 172.21.15.6:0/3147715035 learned_addr learned my addr 172.21.15.6:0/3147715035 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/3147715035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 0x7fe8d406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8d8eea700 1 -- 172.21.15.6:0/3147715035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 msgr2=0x7fe8d406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/3147715035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 0x7fe8d406b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8d8eea700 1 -- 172.21.15.6:0/3147715035 >> v1:172.21.15.61:6789/0 conn(0x7fe8d406d140 legacy=0x7fe8d40cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8d8eea700 1 -- 172.21.15.6:0/3147715035 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40ae980 con 0x7fe8d415a640 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/3147715035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 0x7fe8d406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/3147715035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d415a640 0x7fe8d40c3700 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80098f0 tx=0x7fe8c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925a99d9aaed2709 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8d37fe700 1 -- 172.21.15.6:0/3147715035 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8004230 con 0x7fe8d415a640 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8d37fe700 1 -- 172.21.15.6:0/3147715035 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c8004390 con 0x7fe8d415a640 2023-11-28T13:03:25.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8d37fe700 1 -- 172.21.15.6:0/3147715035 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80036a0 con 0x7fe8d415a640 2023-11-28T13:03:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8db14e700 1 -- 172.21.15.6:0/3147715035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d415a640 msgr2=0x7fe8d40c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8db14e700 1 --2- 172.21.15.6:0/3147715035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d415a640 0x7fe8d40c3700 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80098f0 tx=0x7fe8c8009c00 comp rx=0 tx=0).stop 2023-11-28T13:03:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.879+0000 7fe8db14e700 1 -- 172.21.15.6:0/3147715035 shutdown_connections 2023-11-28T13:03:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.880+0000 7fe8db14e700 1 --2- 172.21.15.6:0/3147715035 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d415a640 0x7fe8d40c3700 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.880+0000 7fe8db14e700 1 --2- 172.21.15.6:0/3147715035 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 0x7fe8d406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.880+0000 7fe8db14e700 1 -- 172.21.15.6:0/3147715035 >> 172.21.15.6:0/3147715035 conn(0x7fe8d40cc630 msgr2=0x7fe8d40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.880+0000 7fe8db14e700 1 -- 172.21.15.6:0/3147715035 shutdown_connections 2023-11-28T13:03:25.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.880+0000 7fe8db14e700 1 -- 172.21.15.6:0/3147715035 wait complete. 2023-11-28T13:03:25.886 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.881+0000 7fe8db14e700 1 Processor -- start 2023-11-28T13:03:25.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.881+0000 7fe8db14e700 1 -- start start 2023-11-28T13:03:25.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8db14e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d406d140 0x7fe8d41577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8db14e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 0x7fe8d4154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8db14e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8d415a640 0x7fe8d41e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8db14e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4157510 con 0x7fe8d406d140 2023-11-28T13:03:25.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8db14e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4154dc0 con 0x7fe8d415a640 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8db14e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d41550a0 con 0x7fe8d40cd140 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d8eea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d406d140 0x7fe8d41577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d8eea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d406d140 0x7fe8d41577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34490/0 (socket says 172.21.15.6:34490) 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d8eea700 1 -- 172.21.15.6:0/379898466 learned_addr learned my addr 172.21.15.6:0/379898466 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 0x7fe8d4154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d96eb700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8d415a640 0x7fe8d41e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d8eea700 1 -- 172.21.15.6:0/379898466 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 msgr2=0x7fe8d4154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 0x7fe8d4154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d8eea700 1 -- 172.21.15.6:0/379898466 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8d415a640 msgr2=0x7fe8d41e39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8d415a640 0x7fe8d41e39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d8eea700 1 -- 172.21.15.6:0/379898466 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c8009550 con 0x7fe8d406d140 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 0x7fe8d4154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:25.888 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d96eb700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8d415a640 0x7fe8d41e39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d406d140 0x7fe8d41577f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009130 tx=0x7fe8c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.882+0000 7fe8d17fa700 1 -- 172.21.15.6:0/379898466 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8003e70 con 0x7fe8d406d140 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.883+0000 7fe8d17fa700 1 -- 172.21.15.6:0/379898466 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c801f070 con 0x7fe8d406d140 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.883+0000 7fe8db14e700 1 -- 172.21.15.6:0/379898466 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d40aef40 con 0x7fe8d406d140 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.883+0000 7fe8d17fa700 1 -- 172.21.15.6:0/379898466 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800b8a0 con 0x7fe8d406d140 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.883+0000 7fe8db14e700 1 -- 172.21.15.6:0/379898466 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d40aec60 con 0x7fe8d406d140 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.884+0000 7fe8d17fa700 1 -- 172.21.15.6:0/379898466 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe8c802a020 con 0x7fe8d406d140 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.884+0000 7fe8d17fa700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8b80682f0 0x7fe8b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.885+0000 7fe8d17fa700 1 -- 172.21.15.6:0/379898466 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe8c8087690 con 0x7fe8d406d140 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.885+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8b80682f0 0x7fe8b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.885+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8b80682f0 0x7fe8b806a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0008910 tx=0x7fe8c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.886+0000 7fe8e72d7b80 1 Processor -- start 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.887+0000 7fe8e72d7b80 1 -- start start 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.887+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfe9ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.887+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bfec5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.887+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec6170 0x55a8bfee17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.887+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfd85c10 con 0x55a8bfec5860 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.887+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfdf43a0 con 0x55a8bfec3180 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.887+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfded8d0 con 0x55a8bfec6170 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.887+0000 7fe8d8eea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfe9ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.887+0000 7fe8d8eea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfe9ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34498/0 (socket says 172.21.15.6:34498) 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.887+0000 7fe8d8eea700 1 -- 172.21.15.6:0/4279850794 learned_addr learned my addr 172.21.15.6:0/4279850794 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.888+0000 7fe8d8eea700 1 -- 172.21.15.6:0/4279850794 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec6170 msgr2=0x55a8bfee17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.888+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/4279850794 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec6170 0x55a8bfee17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.888+0000 7fe8d8eea700 1 -- 172.21.15.6:0/4279850794 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 msgr2=0x55a8bfec5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.888+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/4279850794 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bfec5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.888+0000 7fe8d8eea700 1 -- 172.21.15.6:0/4279850794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8bfc6f630 con 0x55a8bfec5860 2023-11-28T13:03:25.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.888+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/4279850794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfe9ba70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800b0e0 tx=0x7fe8c8010380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de27c53b116fea11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.888+0000 7fe8be7fc700 1 -- 172.21.15.6:0/4279850794 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8055c70 con 0x55a8bfec5860 2023-11-28T13:03:25.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.888+0000 7fe8be7fc700 1 -- 172.21.15.6:0/4279850794 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c8054180 con 0x55a8bfec5860 2023-11-28T13:03:25.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.888+0000 7fe8be7fc700 1 -- 172.21.15.6:0/4279850794 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8057160 con 0x55a8bfec5860 2023-11-28T13:03:25.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.889+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/4279850794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 msgr2=0x55a8bfe9ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.889+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/4279850794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfe9ba70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800b0e0 tx=0x7fe8c8010380 comp rx=0 tx=0).stop 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.889+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/4279850794 shutdown_connections 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.889+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/4279850794 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec6170 0x55a8bfee17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.889+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/4279850794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfe9ba70 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.889+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/4279850794 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bfec5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.889+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/4279850794 >> 172.21.15.6:0/4279850794 conn(0x55a8bfe98db0 msgr2=0x55a8bfe99740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.889+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/4279850794 shutdown_connections 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.889+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/4279850794 wait complete. 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.889+0000 7fe8e72d7b80 1 -- start start 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.890+0000 7fe8e72d7b80 1 Processor -- start 2023-11-28T13:03:25.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.890+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bfeb5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.890+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 0x55a8bfed9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.890+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfe95b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.890+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfc71920 con 0x55a8bfec6170 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.890+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfec85d0 con 0x55a8bfec3180 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.890+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfeefe80 con 0x55a8bfec5860 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.890+0000 7fe8e72d7b80 10 client.0 _ll_register_callbacks cb 0x55a8bfc93290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.890+0000 7fe8d96eb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfe95b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8d96eb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfe95b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34508/0 (socket says 172.21.15.6:34508) 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8d96eb700 1 -- 172.21.15.6:0/171315337 learned_addr learned my addr 172.21.15.6:0/171315337 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 0x55a8bfed9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bfeb5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8d96eb700 1 -- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 msgr2=0x55a8bfed9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8d96eb700 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 0x55a8bfed9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8d96eb700 1 -- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 msgr2=0x55a8bfeb5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8d96eb700 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bfeb5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8d96eb700 1 -- 172.21.15.6:0/171315337 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d40aec60 con 0x55a8bfec6170 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8d96eb700 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfe95b00 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c940 tx=0x7fe8cc00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7f4eaba21089ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8b37fe700 1 -- 172.21.15.6:0/171315337 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc003c00 con 0x55a8bfec6170 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8b37fe700 1 -- 172.21.15.6:0/171315337 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe8cc008040 con 0x55a8bfec6170 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/171315337 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8bfdbb190 con 0x55a8bfec6170 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8b37fe700 1 -- 172.21.15.6:0/171315337 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8cc00ce20 con 0x55a8bfec6170 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8e72d7b80 10 client.6699 fetch_fsmap learned FSMap version 32 2023-11-28T13:03:25.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/171315337 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55a8bfc1bde0 con 0x55a8bfec6170 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.891+0000 7fe8b37fe700 1 -- 172.21.15.6:0/171315337 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8bfdbb190 con 0x55a8bfec6170 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.892+0000 7fe8b37fe700 1 -- 172.21.15.6:0/171315337 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe8cc016030 con 0x55a8bfec6170 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.892+0000 7fe8e72d7b80 10 client.6699 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.892+0000 7fe8e72d7b80 10 client.6699 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.892+0000 7fe8e72d7b80 20 client.6699 populate_metadata read hostname 'smithi006' 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.892+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55a8bfec3ac0 0x55a8bfec3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.892+0000 7fe8e72d7b80 4 client.6699 mds_command: new command op to 6471 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.892+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/171315337 --> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8bfc71f60 con 0x55a8bfec3ac0 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.892+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55a8bfec3ac0 0x55a8bfec3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.892+0000 7fe8b37fe700 10 client.6699 ms_handle_connect on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.892+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55a8bfec3ac0 0x55a8bfec3ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e135a6710db1a8cf server_cookie=823888e792685a89 in_seq=0 out_seq=0 2023-11-28T13:03:25.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8b37fe700 1 -- 172.21.15.6:0/171315337 <== mds.0 v2:172.21.15.61:6834/3076579711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55a8bfc71f60 con 0x55a8bfec3ac0 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8b37fe700 10 client.6699 handle_command_reply: tid=0 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 client.6699 shutdown 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/171315337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 msgr2=0x55a8bfe95b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfe95b00 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c940 tx=0x7fe8cc00cd00 comp rx=0 tx=0).stop 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/171315337 shutdown_connections 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55a8bfec3ac0 0x55a8bfec3ec0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfe95b00 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bfeb5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8b37fe700 0 client.6699 ms_handle_reset on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/171315337 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 0x55a8bfed9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/171315337 >> 172.21.15.6:0/171315337 conn(0x55a8bfe98db0 msgr2=0x55a8bfe9b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/171315337 shutdown_connections 2023-11-28T13:03:25.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/171315337 wait complete. 2023-11-28T13:03:25.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.893+0000 7fe8e72d7b80 20 client.6699 trim_cache size 0 max 16384 2023-11-28T13:03:25.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.910+0000 7fe8e72d7b80 1 Processor -- start 2023-11-28T13:03:25.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.910+0000 7fe8e72d7b80 1 -- start start 2023-11-28T13:03:25.913 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.910+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec3180 0x55a8bfed9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.910+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfed96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.910+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec6170 0x55a8bfeeec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.910+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfe9ab80 con 0x55a8bfec5860 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.910+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfebd060 con 0x55a8bfec6170 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.910+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfe9c720 con 0x55a8bfec3180 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.910+0000 7fe8d3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfed96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8d3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfed96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34514/0 (socket says 172.21.15.6:34514) 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8d3fff700 1 -- 172.21.15.6:0/3934965695 learned_addr learned my addr 172.21.15.6:0/3934965695 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8d3fff700 1 -- 172.21.15.6:0/3934965695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec3180 msgr2=0x55a8bfed9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/3934965695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec3180 0x55a8bfed9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8d3fff700 1 -- 172.21.15.6:0/3934965695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec6170 msgr2=0x55a8bfeeec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/3934965695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec6170 0x55a8bfeeec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8d3fff700 1 -- 172.21.15.6:0/3934965695 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8bfdb4190 con 0x55a8bfec5860 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/3934965695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfed96a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000d850 tx=0x7fe8c0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da368efb7ef37785 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8beffd700 1 -- 172.21.15.6:0/3934965695 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c000b050 con 0x55a8bfec5860 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8beffd700 1 -- 172.21.15.6:0/3934965695 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c000c040 con 0x55a8bfec5860 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.911+0000 7fe8beffd700 1 -- 172.21.15.6:0/3934965695 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0012870 con 0x55a8bfec5860 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3934965695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 msgr2=0x55a8bfed96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/3934965695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfed96a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000d850 tx=0x7fe8c0010000 comp rx=0 tx=0).stop 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3934965695 shutdown_connections 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/3934965695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec6170 0x55a8bfeeec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/3934965695 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec3180 0x55a8bfed9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/3934965695 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec5860 0x55a8bfed96a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3934965695 >> 172.21.15.6:0/3934965695 conn(0x55a8bfe98db0 msgr2=0x55a8bfe9c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3934965695 shutdown_connections 2023-11-28T13:03:25.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3934965695 wait complete. 2023-11-28T13:03:25.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 -- start start 2023-11-28T13:03:25.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.912+0000 7fe8e72d7b80 1 Processor -- start 2023-11-28T13:03:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.913+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bff459a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.913+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 0x55a8bfebd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.913+0000 7fe8e72d7b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfec8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.913+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfeb5920 con 0x55a8bfec6170 2023-11-28T13:03:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.913+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfec3e90 con 0x55a8bfec3180 2023-11-28T13:03:25.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.913+0000 7fe8e72d7b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8bfec2900 con 0x55a8bfec5860 2023-11-28T13:03:25.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.913+0000 7fe8e72d7b80 10 client.0 _ll_register_callbacks cb 0x55a8bfc93290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:25.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.913+0000 7fe8d96eb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfec8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.913+0000 7fe8d96eb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfec8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34522/0 (socket says 172.21.15.6:34522) 2023-11-28T13:03:25.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.913+0000 7fe8d96eb700 1 -- 172.21.15.6:0/3714317745 learned_addr learned my addr 172.21.15.6:0/3714317745 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:25.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 0x55a8bfebd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bff459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8d96eb700 1 -- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 msgr2=0x55a8bfebd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8d96eb700 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 0x55a8bfebd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8d96eb700 1 -- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 msgr2=0x55a8bff459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:25.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8d96eb700 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bff459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8d96eb700 1 -- 172.21.15.6:0/3714317745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c001b040 con 0x55a8bfec6170 2023-11-28T13:03:25.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bff459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:25.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8d3fff700 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 0x55a8bfebd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:25.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8d96eb700 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfec8dd0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00cd40 tx=0x7fe8cc004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cf9547c767c67f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:25.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:03:25.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8b37fe700 1 -- 172.21.15.6:0/3714317745 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00abb0 con 0x55a8bfec6170 2023-11-28T13:03:25.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8b37fe700 1 -- 172.21.15.6:0/3714317745 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe8cc00a500 con 0x55a8bfec6170 2023-11-28T13:03:25.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3714317745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8bfdab1b0 con 0x55a8bfec6170 2023-11-28T13:03:25.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8b37fe700 1 -- 172.21.15.6:0/3714317745 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8cc00ad10 con 0x55a8bfec6170 2023-11-28T13:03:25.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8e72d7b80 10 client.6705 fetch_fsmap learned FSMap version 32 2023-11-28T13:03:25.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3714317745 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55a8bfc42250 con 0x55a8bfec6170 2023-11-28T13:03:25.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8b37fe700 1 -- 172.21.15.6:0/3714317745 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8bfdab1b0 con 0x55a8bfec6170 2023-11-28T13:03:25.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8b37fe700 1 -- 172.21.15.6:0/3714317745 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 32) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fe8cc016070 con 0x55a8bfec6170 2023-11-28T13:03:25.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8e72d7b80 10 client.6705 fetch_fsmap finished waiting for FSMap version 32 2023-11-28T13:03:25.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8e72d7b80 10 client.6705 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-11-28T13:03:25.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8e72d7b80 20 client.6705 populate_metadata read hostname 'smithi006' 2023-11-28T13:03:25.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55a8bfebca80 0x55a8bfeeec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:25.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8e72d7b80 4 client.6705 mds_command: new command op to 6471 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:03:25.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.914+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3714317745 --> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a8bfd951c0 con 0x55a8bfebca80 2023-11-28T13:03:25.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55a8bfebca80 0x55a8bfeeec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:25.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8b37fe700 10 client.6705 ms_handle_connect on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:25.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8d8eea700 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55a8bfebca80 0x55a8bfeeec50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b32a01c2491efd3 server_cookie=804c368c15f3b79d in_seq=0 out_seq=0 2023-11-28T13:03:25.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8b37fe700 1 -- 172.21.15.6:0/3714317745 <== mds.0 v2:172.21.15.61:6834/3076579711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x55a8bfd951c0 con 0x55a8bfebca80 2023-11-28T13:03:25.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8b37fe700 10 client.6705 handle_command_reply: tid=0 2023-11-28T13:03:25.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8e72d7b80 1 client.6705 shutdown 2023-11-28T13:03:25.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3714317745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 msgr2=0x55a8bfec8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfec8dd0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00cd40 tx=0x7fe8cc004a90 comp rx=0 tx=0).stop 2023-11-28T13:03:25.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3714317745 shutdown_connections 2023-11-28T13:03:25.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55a8bfebca80 0x55a8bfeeec50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55a8bfec6170 0x55a8bfec8dd0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8bfec3180 0x55a8bff459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8b37fe700 0 client.6705 ms_handle_reset on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:25.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8e72d7b80 1 --2- 172.21.15.6:0/3714317745 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55a8bfec5860 0x55a8bfebd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.915+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3714317745 >> 172.21.15.6:0/3714317745 conn(0x55a8bfe98db0 msgr2=0x55a8bfe9c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3714317745 shutdown_connections 2023-11-28T13:03:25.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8e72d7b80 1 -- 172.21.15.6:0/3714317745 wait complete. 2023-11-28T13:03:25.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8e72d7b80 20 client.6705 trim_cache size 0 max 16384 2023-11-28T13:03:25.929 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:03:25.929 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T13:03:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": { 2023-11-28T13:03:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "62c162ed-305e-4815-9559-16db3a07f334": { 2023-11-28T13:03:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/", 2023-11-28T13:03:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "tag": "62c162ed-305e-4815-9559-16db3a07f334", 2023-11-28T13:03:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: "options": "recursive" 2023-11-28T13:03:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:25.930 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:25.930 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:03:25.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 -- 172.21.15.6:0/379898466 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8b80682f0 msgr2=0x7fe8b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8b80682f0 0x7fe8b806a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0008910 tx=0x7fe8c0008040 comp rx=0 tx=0).stop 2023-11-28T13:03:25.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 -- 172.21.15.6:0/379898466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d406d140 msgr2=0x7fe8d41577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:25.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d406d140 0x7fe8d41577f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009130 tx=0x7fe8c8004750 comp rx=0 tx=0).stop 2023-11-28T13:03:25.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 -- 172.21.15.6:0/379898466 shutdown_connections 2023-11-28T13:03:25.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8d415a640 0x7fe8d41e39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8d406d140 0x7fe8d41577f0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8b80682f0 0x7fe8b806a7a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 --2- 172.21.15.6:0/379898466 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8d40cd140 0x7fe8d4154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:25.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 -- 172.21.15.6:0/379898466 >> 172.21.15.6:0/379898466 conn(0x7fe8d40cc630 msgr2=0x7fe8d40b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:25.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 -- 172.21.15.6:0/379898466 shutdown_connections 2023-11-28T13:03:25.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:25.916+0000 7fe8db14e700 1 -- 172.21.15.6:0/379898466 wait complete. 2023-11-28T13:03:25.934 INFO:tasks.cephfs.filesystem:scrub status for tag:62c162ed-305e-4815-9559-16db3a07f334 - {'path': '/', 'tag': '62c162ed-305e-4815-9559-16db3a07f334', 'options': 'recursive'} 2023-11-28T13:03:26.904 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:03:26.908 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:03:29.045 INFO:tasks.ceph.mon.a.smithi006.stderr:2023-11-28T13:03:29.043+0000 7fa6275a1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-28T13:03:55.936 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T13:03:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.077+0000 7fcf834e3700 1 Processor -- start 2023-11-28T13:03:56.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.077+0000 7fcf834e3700 1 -- start start 2023-11-28T13:03:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf834e3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 0x7fcf7c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf834e3700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fcf7c15dbf0 0x7fcf7c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf834e3700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcf7c0b1630 0x7fcf7c0b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf834e3700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c0cede0 con 0x7fcf7c0b1630 2023-11-28T13:03:56.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf834e3700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c0b1ff0 con 0x7fcf7c15dbf0 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf834e3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c15e040 con 0x7fcf7c159500 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf81a80700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcf7c0b1630 0x7fcf7c0b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf81a80700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcf7c0b1630 0x7fcf7c0b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34526/0 (socket says 172.21.15.6:34526) 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf81a80700 1 -- 172.21.15.6:0/1762576479 learned_addr learned my addr 172.21.15.6:0/1762576479 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/1762576479 >> v2:172.21.15.61:3300/0 conn(0x7fcf7c15dbf0 0x7fcf7c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf8127f700 1 --2- 172.21.15.6:0/1762576479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 0x7fcf7c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf81a80700 1 -- 172.21.15.6:0/1762576479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 msgr2=0x7fcf7c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf81a80700 1 --2- 172.21.15.6:0/1762576479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 0x7fcf7c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf81a80700 1 -- 172.21.15.6:0/1762576479 >> v2:172.21.15.61:3300/0 conn(0x7fcf7c15dbf0 msgr2=0x7fcf7c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf81a80700 1 --2- 172.21.15.6:0/1762576479 >> v2:172.21.15.61:3300/0 conn(0x7fcf7c15dbf0 0x7fcf7c0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf81a80700 1 -- 172.21.15.6:0/1762576479 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c0ceff0 con 0x7fcf7c0b1630 2023-11-28T13:03:56.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/1762576479 >> v2:172.21.15.61:3300/0 conn(0x7fcf7c15dbf0 0x7fcf7c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf8127f700 1 --2- 172.21.15.6:0/1762576479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 0x7fcf7c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf81a80700 1 --2- 172.21.15.6:0/1762576479 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 0x7fcf7c0b1ab0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0714f0 tx=0x7fcf7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753011c3f8fe2f1b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf6ffff700 1 -- 172.21.15.6:0/1762576479 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400e070 con 0x7fcf7c0b1630 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf6ffff700 1 -- 172.21.15.6:0/1762576479 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7400b9c0 con 0x7fcf7c0b1630 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.078+0000 7fcf6ffff700 1 -- 172.21.15.6:0/1762576479 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400bd70 con 0x7fcf7c0b1630 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.079+0000 7fcf834e3700 1 -- 172.21.15.6:0/1762576479 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 msgr2=0x7fcf7c0b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.079+0000 7fcf834e3700 1 --2- 172.21.15.6:0/1762576479 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 0x7fcf7c0b1ab0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0714f0 tx=0x7fcf7400b330 comp rx=0 tx=0).stop 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.079+0000 7fcf834e3700 1 -- 172.21.15.6:0/1762576479 shutdown_connections 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.079+0000 7fcf834e3700 1 --2- 172.21.15.6:0/1762576479 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 0x7fcf7c0b1ab0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.079+0000 7fcf834e3700 1 --2- 172.21.15.6:0/1762576479 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 0x7fcf7c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.082 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.079+0000 7fcf834e3700 1 --2- 172.21.15.6:0/1762576479 >> v2:172.21.15.61:3300/0 conn(0x7fcf7c15dbf0 0x7fcf7c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.079+0000 7fcf834e3700 1 -- 172.21.15.6:0/1762576479 >> 172.21.15.6:0/1762576479 conn(0x7fcf7c0cc630 msgr2=0x7fcf7c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.079+0000 7fcf834e3700 1 -- 172.21.15.6:0/1762576479 shutdown_connections 2023-11-28T13:03:56.083 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.079+0000 7fcf834e3700 1 -- 172.21.15.6:0/1762576479 wait complete. 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.080+0000 7fcf834e3700 1 Processor -- start 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.080+0000 7fcf834e3700 1 -- start start 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf834e3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 0x7fcf7c1ec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf834e3700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 0x7fcf7c0c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf834e3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf7c15dbf0 0x7fcf7c0c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf834e3700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c15ce50 con 0x7fcf7c0b1630 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf834e3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1f3520 con 0x7fcf7c15dbf0 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf834e3700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1f2910 con 0x7fcf7c159500 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf8127f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 0x7fcf7c1ec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf8127f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 0x7fcf7c1ec3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34532/0 (socket says 172.21.15.6:34532) 2023-11-28T13:03:56.087 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf8127f700 1 -- 172.21.15.6:0/1663334074 learned_addr learned my addr 172.21.15.6:0/1663334074 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf81a80700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf7c15dbf0 0x7fcf7c0c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.081+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 0x7fcf7c0c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf8127f700 1 -- 172.21.15.6:0/1663334074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 msgr2=0x7fcf7c0c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf8127f700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 0x7fcf7c0c5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf8127f700 1 -- 172.21.15.6:0/1663334074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf7c15dbf0 msgr2=0x7fcf7c0c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf8127f700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf7c15dbf0 0x7fcf7c0c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf8127f700 1 -- 172.21.15.6:0/1663334074 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf74009b30 con 0x7fcf7c0b1630 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf8127f700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 0x7fcf7c1ec3c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800c7f0 tx=0x7fcf7800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf6dffb700 1 -- 172.21.15.6:0/1663334074 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78011800 con 0x7fcf7c0b1630 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf6dffb700 1 -- 172.21.15.6:0/1663334074 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf78011960 con 0x7fcf7c0b1630 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf834e3700 1 -- 172.21.15.6:0/1663334074 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c15d110 con 0x7fcf7c0b1630 2023-11-28T13:03:56.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf6dffb700 1 -- 172.21.15.6:0/1663334074 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78011d10 con 0x7fcf7c0b1630 2023-11-28T13:03:56.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.082+0000 7fcf834e3700 1 -- 172.21.15.6:0/1663334074 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c0cf2d0 con 0x7fcf7c0b1630 2023-11-28T13:03:56.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.084+0000 7fcf6dffb700 1 -- 172.21.15.6:0/1663334074 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcf78021070 con 0x7fcf7c0b1630 2023-11-28T13:03:56.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.084+0000 7fcf6dffb700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcf600682f0 0x7fcf6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.084+0000 7fcf6dffb700 1 -- 172.21.15.6:0/1663334074 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcf78085be0 con 0x7fcf7c0b1630 2023-11-28T13:03:56.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.084+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcf600682f0 0x7fcf6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.085+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcf600682f0 0x7fcf6006a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0abfa0 tx=0x7fcf70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.086+0000 7fcf8f66cb80 1 Processor -- start 2023-11-28T13:03:56.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.086+0000 7fcf8f66cb80 1 -- start start 2023-11-28T13:03:56.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180d860 0x5556317e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180b180 0x55563180dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180e170 0x5556318297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556316cdc10 con 0x55563180b180 2023-11-28T13:03:56.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563173c3a0 con 0x55563180d860 2023-11-28T13:03:56.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556317358d0 con 0x55563180e170 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf80a7e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180b180 0x55563180dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf80a7e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180b180 0x55563180dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34544/0 (socket says 172.21.15.6:34544) 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf80a7e700 1 -- 172.21.15.6:0/1728378238 learned_addr learned my addr 172.21.15.6:0/1728378238 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf8127f700 1 --2- 172.21.15.6:0/1728378238 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180d860 0x5556317e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf81a80700 1 --2- 172.21.15.6:0/1728378238 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180e170 0x5556318297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf80a7e700 1 -- 172.21.15.6:0/1728378238 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180e170 msgr2=0x5556318297c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/1728378238 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180e170 0x5556318297c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf80a7e700 1 -- 172.21.15.6:0/1728378238 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180d860 msgr2=0x5556317e3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/1728378238 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180d860 0x5556317e3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf80a7e700 1 -- 172.21.15.6:0/1728378238 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556315b7630 con 0x55563180b180 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.087+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/1728378238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180b180 0x55563180dc30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000eaa0 tx=0x7fcf70011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cda3a2aa2d94117 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf66ffd700 1 -- 172.21.15.6:0/1728378238 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70013660 con 0x55563180b180 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf66ffd700 1 -- 172.21.15.6:0/1728378238 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf70018070 con 0x55563180b180 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf66ffd700 1 -- 172.21.15.6:0/1728378238 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf700139c0 con 0x55563180b180 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1728378238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180b180 msgr2=0x55563180dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/1728378238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180b180 0x55563180dc30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000eaa0 tx=0x7fcf70011000 comp rx=0 tx=0).stop 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1728378238 shutdown_connections 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/1728378238 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180e170 0x5556318297c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/1728378238 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180d860 0x5556317e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/1728378238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180b180 0x55563180dc30 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1728378238 >> 172.21.15.6:0/1728378238 conn(0x5556317e0db0 msgr2=0x5556317e1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1728378238 shutdown_connections 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.088+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1728378238 wait complete. 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.089+0000 7fcf8f66cb80 1 -- start start 2023-11-28T13:03:56.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.089+0000 7fcf8f66cb80 1 Processor -- start 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180b180 0x55563180fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180d860 0x555631810000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180e170 0x555631810cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556315b9920 con 0x55563180e170 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555631805ab0 con 0x55563180d860 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55563182d520 con 0x55563180b180 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf8f66cb80 10 client.0 _ll_register_callbacks cb 0x5556315db290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf81a80700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180e170 0x555631810cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf81a80700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180e170 0x555631810cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34550/0 (socket says 172.21.15.6:34550) 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf81a80700 1 -- 172.21.15.6:0/1809943465 learned_addr learned my addr 172.21.15.6:0/1809943465 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180d860 0x555631810000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf8127f700 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180b180 0x55563180fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf81a80700 1 -- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180b180 msgr2=0x55563180fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf81a80700 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180b180 0x55563180fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf81a80700 1 -- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180d860 msgr2=0x555631810000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf81a80700 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180d860 0x555631810000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.090+0000 7fcf81a80700 1 -- 172.21.15.6:0/1809943465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7000e750 con 0x55563180e170 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf81a80700 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180e170 0x555631810cc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c066a20 tx=0x7fcf74004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5198809f1fe7e541 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf5bfff700 1 -- 172.21.15.6:0/1809943465 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74004450 con 0x55563180e170 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf5bfff700 1 -- 172.21.15.6:0/1809943465 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcf740045b0 con 0x55563180e170 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1809943465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555631703190 con 0x55563180e170 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf5bfff700 1 -- 172.21.15.6:0/1809943465 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf74004770 con 0x55563180e170 2023-11-28T13:03:56.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf8f66cb80 10 client.6717 fetch_fsmap learned FSMap version 33 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1809943465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x555631563de0 con 0x55563180e170 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf5bfff700 1 -- 172.21.15.6:0/1809943465 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555631703190 con 0x55563180e170 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf5bfff700 1 -- 172.21.15.6:0/1809943465 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fcf7400b050 con 0x55563180e170 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf8f66cb80 10 client.6717 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf8f66cb80 10 client.6717 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf8f66cb80 20 client.6717 populate_metadata read hostname 'smithi006' 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x555631811200 0x555631804e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf8f66cb80 4 client.6717 mds_command: new command op to 6471 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1809943465 --> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556315b9f60 con 0x555631811200 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.091+0000 7fcf8127f700 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x555631811200 0x555631804e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.092+0000 7fcf5bfff700 10 client.6717 ms_handle_connect on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:56.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.092+0000 7fcf8127f700 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x555631811200 0x555631804e50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91316fc1c47cf48e server_cookie=e6c86cbec598a96d in_seq=0 out_seq=0 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.092+0000 7fcf5bfff700 1 -- 172.21.15.6:0/1809943465 <== mds.0 v2:172.21.15.61:6834/3076579711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x5556315b9f60 con 0x555631811200 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.092+0000 7fcf5bfff700 10 client.6717 handle_command_reply: tid=0 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.092+0000 7fcf8f66cb80 1 client.6717 shutdown 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1809943465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180e170 msgr2=0x555631810cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180e170 0x555631810cc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c066a20 tx=0x7fcf74004b40 comp rx=0 tx=0).stop 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1809943465 shutdown_connections 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x555631811200 0x555631804e50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180e170 0x555631810cc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180b180 0x55563180fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf5bfff700 0 client.6717 ms_handle_reset on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/1809943465 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180d860 0x555631810000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.101 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1809943465 >> 172.21.15.6:0/1809943465 conn(0x5556317e0db0 msgr2=0x55563180ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1809943465 shutdown_connections 2023-11-28T13:03:56.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/1809943465 wait complete. 2023-11-28T13:03:56.102 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.093+0000 7fcf8f66cb80 20 client.6717 trim_cache size 0 max 16384 2023-11-28T13:03:56.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf8f66cb80 1 Processor -- start 2023-11-28T13:03:56.111 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf8f66cb80 1 -- start start 2023-11-28T13:03:56.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180b180 0x5556317de640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180e170 0x55563182bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x555631830b00 con 0x55563180d860 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55563180ae20 con 0x55563180e170 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x555631830490 con 0x55563180b180 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf80a7e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf80a7e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34554/0 (socket says 172.21.15.6:34554) 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.110+0000 7fcf80a7e700 1 -- 172.21.15.6:0/3156944045 learned_addr learned my addr 172.21.15.6:0/3156944045 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf81a80700 1 --2- 172.21.15.6:0/3156944045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180e170 0x55563182bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf8127f700 1 --2- 172.21.15.6:0/3156944045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180b180 0x5556317de640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf80a7e700 1 -- 172.21.15.6:0/3156944045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180b180 msgr2=0x5556317de640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/3156944045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180b180 0x5556317de640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf80a7e700 1 -- 172.21.15.6:0/3156944045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180e170 msgr2=0x55563182bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/3156944045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180e170 0x55563182bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf80a7e700 1 -- 172.21.15.6:0/3156944045 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556316fc190 con 0x55563180d860 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/3156944045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dde70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcf70020000 tx=0x7fcf70013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48dda097793f43a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf677fe700 1 -- 172.21.15.6:0/3156944045 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70005ae0 con 0x55563180d860 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf677fe700 1 -- 172.21.15.6:0/3156944045 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf70018070 con 0x55563180d860 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.111+0000 7fcf677fe700 1 -- 172.21.15.6:0/3156944045 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70005e40 con 0x55563180d860 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/3156944045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 msgr2=0x5556317dde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/3156944045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dde70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcf70020000 tx=0x7fcf70013000 comp rx=0 tx=0).stop 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/3156944045 shutdown_connections 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/3156944045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180e170 0x55563182bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/3156944045 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180b180 0x5556317de640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/3156944045 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dde70 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/3156944045 >> 172.21.15.6:0/3156944045 conn(0x5556317e0db0 msgr2=0x555631803330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/3156944045 shutdown_connections 2023-11-28T13:03:56.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/3156944045 wait complete. 2023-11-28T13:03:56.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 -- start start 2023-11-28T13:03:56.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.112+0000 7fcf8f66cb80 1 Processor -- start 2023-11-28T13:03:56.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180b180 0x55563182b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf8f66cb80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180e170 0x55563181e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556317dd660 con 0x55563180d860 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556318481a0 con 0x55563180b180 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf8f66cb80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556317dcc90 con 0x55563180e170 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf8f66cb80 10 client.0 _ll_register_callbacks cb 0x5556315db290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf80a7e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf80a7e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34562/0 (socket says 172.21.15.6:34562) 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf80a7e700 1 -- 172.21.15.6:0/443783107 learned_addr learned my addr 172.21.15.6:0/443783107 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf8127f700 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180b180 0x55563182b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.113+0000 7fcf81a80700 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180e170 0x55563181e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf80a7e700 1 -- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180e170 msgr2=0x55563181e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180e170 0x55563181e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf80a7e700 1 -- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180b180 msgr2=0x55563182b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.120 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180b180 0x55563182b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf80a7e700 1 -- 172.21.15.6:0/443783107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556315b7630 con 0x55563180d860 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf80a7e700 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dd120 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000a7d0 tx=0x7fcf700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457590fa3d2d241c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf5bfff700 1 -- 172.21.15.6:0/443783107 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70013e20 con 0x55563180d860 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/443783107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556316f31b0 con 0x55563180d860 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf5bfff700 1 -- 172.21.15.6:0/443783107 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcf70005e30 con 0x55563180d860 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf5bfff700 1 -- 172.21.15.6:0/443783107 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf70006000 con 0x55563180d860 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf8f66cb80 10 client.6723 fetch_fsmap learned FSMap version 33 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/443783107 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55563158a250 con 0x55563180d860 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf5bfff700 1 -- 172.21.15.6:0/443783107 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5556316f31b0 con 0x55563180d860 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf5bfff700 1 -- 172.21.15.6:0/443783107 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fcf70006160 con 0x55563180d860 2023-11-28T13:03:56.121 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf8f66cb80 10 client.6723 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf8f66cb80 10 client.6723 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf8f66cb80 20 client.6723 populate_metadata read hostname 'smithi006' 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55563180f830 0x55563181e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf8f66cb80 4 client.6723 mds_command: new command op to 6471 tid=0[{"prefix": "scrub status"}] 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.114+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/443783107 --> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5556316dd1c0 con 0x55563180f830 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8127f700 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55563180f830 0x55563181e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf5bfff700 10 client.6723 ms_handle_connect on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8127f700 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55563180f830 0x55563181e540 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3375c55c240b62a server_cookie=e81fe7d1ecdd38f1 in_seq=0 out_seq=0 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf5bfff700 1 -- 172.21.15.6:0/443783107 <== mds.0 v2:172.21.15.61:6834/3076579711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5556316dd1c0 con 0x55563180f830 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf5bfff700 10 client.6723 handle_command_reply: tid=0 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8f66cb80 1 client.6723 shutdown 2023-11-28T13:03:56.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/443783107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 msgr2=0x5556317dd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dd120 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000a7d0 tx=0x7fcf700047c0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/443783107 shutdown_connections 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55563180f830 0x55563181e540 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55563180e170 0x55563181e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf5bfff700 0 client.6723 ms_handle_reset on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55563180b180 0x55563182b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8f66cb80 1 --2- 172.21.15.6:0/443783107 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55563180d860 0x5556317dd120 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.115+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/443783107 >> 172.21.15.6:0/443783107 conn(0x5556317e0db0 msgr2=0x555631804e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/443783107 shutdown_connections 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf8f66cb80 1 -- 172.21.15.6:0/443783107 wait complete. 2023-11-28T13:03:56.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf8f66cb80 20 client.6723 trim_cache size 0 max 16384 2023-11-28T13:03:56.124 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:03:56.124 INFO:teuthology.orchestra.run.smithi006.stdout: "status": "no active scrubs running", 2023-11-28T13:03:56.124 INFO:teuthology.orchestra.run.smithi006.stdout: "scrubs": {} 2023-11-28T13:03:56.124 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 -- 172.21.15.6:0/1663334074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcf600682f0 msgr2=0x7fcf6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcf600682f0 0x7fcf6006a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0abfa0 tx=0x7fcf70008040 comp rx=0 tx=0).stop 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 -- 172.21.15.6:0/1663334074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 msgr2=0x7fcf7c1ec3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 0x7fcf7c1ec3c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800c7f0 tx=0x7fcf7800cbb0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 -- 172.21.15.6:0/1663334074 shutdown_connections 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf7c15dbf0 0x7fcf7c0c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf7c0b1630 0x7fcf7c1ec3c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcf600682f0 0x7fcf6006a7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 --2- 172.21.15.6:0/1663334074 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcf7c159500 0x7fcf7c0c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 -- 172.21.15.6:0/1663334074 >> 172.21.15.6:0/1663334074 conn(0x7fcf7c0cc630 msgr2=0x7fcf7c1581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 -- 172.21.15.6:0/1663334074 shutdown_connections 2023-11-28T13:03:56.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.116+0000 7fcf834e3700 1 -- 172.21.15.6:0/1663334074 wait complete. 2023-11-28T13:03:56.130 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T13:03:56.130 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:03:56.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.253+0000 7f88513b2700 1 Processor -- start 2023-11-28T13:03:56.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.253+0000 7f88513b2700 1 -- start start 2023-11-28T13:03:56.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.254+0000 7f88513b2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f884c0cc310 0x7f884c0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.254+0000 7f88513b2700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c0ce340 con 0x7f884c15c990 2023-11-28T13:03:56.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.254+0000 7f88513b2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c0cebe0 con 0x7f884c161090 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.254+0000 7f88513b2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c0cf780 con 0x7f884c0cc310 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.254+0000 7f884affd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f884c15c990 0x7f884c0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50120/0 (socket says 172.21.15.6:50120) 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.254+0000 7f884affd700 1 -- 172.21.15.6:0/2177627224 learned_addr learned my addr 172.21.15.6:0/2177627224 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884b7fe700 1 --2- 172.21.15.6:0/2177627224 >> v2:172.21.15.61:3300/0 conn(0x7f884c0cc310 0x7f884c0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765591823 0 0) 0x7f884c0ce340 con 0x7f884c15c990 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88340035b0 con 0x7f884c15c990 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660161466 0 0) 0x7f884c0cebe0 con 0x7f884c161090 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f884c0ce340 con 0x7f884c161090 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3854817301 0 0) 0x7f88340035b0 con 0x7f884c15c990 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 >> v2:172.21.15.61:3300/0 conn(0x7f884c0cc310 msgr2=0x7f884c0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 --2- 172.21.15.6:0/2177627224 >> v2:172.21.15.61:3300/0 conn(0x7f884c0cc310 0x7f884c0cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 >> v1:172.21.15.61:6789/0 conn(0x7f884c161090 legacy=0x7f884c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c0cccb0 con 0x7f884c15c990 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f8840001670 con 0x7f884c15c990 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 >> v1:172.21.15.6:6789/0 conn(0x7f884c15c990 legacy=0x7f884c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8834003be0 0x7f8834003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8834004850 0x7f8834004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f883400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0cf780 con 0x7f88340052c0 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883400c200 con 0x7f8834004850 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883400c3c0 con 0x7f8834003be0 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884b7fe700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f883400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884affd700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8834003be0 0x7f8834003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884a7fc700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8834004850 0x7f8834004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884b7fe700 1 -- 172.21.15.6:0/2177627224 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8834003be0 msgr2=0x7f8834003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884b7fe700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8834003be0 0x7f8834003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884b7fe700 1 -- 172.21.15.6:0/2177627224 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8834004850 msgr2=0x7f8834004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884b7fe700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8834004850 0x7f8834004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884b7fe700 1 -- 172.21.15.6:0/2177627224 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88340035b0 con 0x7f88340052c0 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884affd700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8834003be0 0x7f8834003fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.255+0000 7f884a7fc700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8834004850 0x7f8834004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f884b7fe700 1 -- 172.21.15.6:0/2177627224 reap_dead start 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f884b7fe700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f883400ba10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8844009fc0 tx=0x7f8834005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df1c9b8621f1fcb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844003a20 con 0x7f88340052c0 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884400f040 con 0x7f88340052c0 2023-11-28T13:03:56.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f8849ffb700 1 -- 172.21.15.6:0/2177627224 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844003de0 con 0x7f88340052c0 2023-11-28T13:03:56.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f88513b2700 1 -- 172.21.15.6:0/2177627224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 msgr2=0x7f883400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f88513b2700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f883400ba10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8844009fc0 tx=0x7f8834005880 comp rx=0 tx=0).stop 2023-11-28T13:03:56.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f88513b2700 1 -- 172.21.15.6:0/2177627224 shutdown_connections 2023-11-28T13:03:56.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f88513b2700 1 --2- 172.21.15.6:0/2177627224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f883400ba10 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f88513b2700 1 -- 172.21.15.6:0/2177627224 >> 172.21.15.6:0/2177627224 conn(0x7f884c0b1650 msgr2=0x7f884c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f88513b2700 1 -- 172.21.15.6:0/2177627224 shutdown_connections 2023-11-28T13:03:56.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.256+0000 7f88513b2700 1 -- 172.21.15.6:0/2177627224 wait complete. 2023-11-28T13:03:56.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.258+0000 7f88513b2700 1 Processor -- start 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.258+0000 7f88513b2700 1 -- start start 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.258+0000 7f88513b2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f884c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.258+0000 7f88513b2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c0c5d50 0x7f884c159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.258+0000 7f88513b2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f884c159580 0x7f884c06b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.258+0000 7f88513b2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0b3a80 con 0x7f88340052c0 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.258+0000 7f88513b2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0c6210 con 0x7f884c0c5d50 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.258+0000 7f88513b2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0c3550 con 0x7f884c159580 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f884c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f884c159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34580/0 (socket says 172.21.15.6:34580) 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884affd700 1 -- 172.21.15.6:0/1455057319 learned_addr learned my addr 172.21.15.6:0/1455057319 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884b7fe700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f884c159580 0x7f884c06b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884affd700 1 -- 172.21.15.6:0/1455057319 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f884c159580 msgr2=0x7f884c06b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884a7fc700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c0c5d50 0x7f884c159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884affd700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f884c159580 0x7f884c06b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884affd700 1 -- 172.21.15.6:0/1455057319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c0c5d50 msgr2=0x7f884c159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884affd700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c0c5d50 0x7f884c159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884affd700 1 -- 172.21.15.6:0/1455057319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c0ce590 con 0x7f88340052c0 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884b7fe700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f884c159580 0x7f884c06b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884a7fc700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c0c5d50 0x7f884c159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f884affd700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f884c159980 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f884000e780 tx=0x7f884000eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f88337fe700 1 -- 172.21.15.6:0/1455057319 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88400105d0 con 0x7f88340052c0 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f88337fe700 1 -- 172.21.15.6:0/1455057319 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8840010730 con 0x7f88340052c0 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f88513b2700 1 -- 172.21.15.6:0/1455057319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c0cb3f0 con 0x7f88340052c0 2023-11-28T13:03:56.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f88337fe700 1 -- 172.21.15.6:0/1455057319 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840010aa0 con 0x7f88340052c0 2023-11-28T13:03:56.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.259+0000 7f88513b2700 1 -- 172.21.15.6:0/1455057319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c0ce870 con 0x7f88340052c0 2023-11-28T13:03:56.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.260+0000 7f88513b2700 1 -- 172.21.15.6:0/1455057319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c0c3390 con 0x7f88340052c0 2023-11-28T13:03:56.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.261+0000 7f88337fe700 1 -- 172.21.15.6:0/1455057319 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f884001b030 con 0x7f88340052c0 2023-11-28T13:03:56.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.261+0000 7f88337fe700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88380682f0 0x7f883806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.261+0000 7f88337fe700 1 -- 172.21.15.6:0/1455057319 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8840085450 con 0x7f88340052c0 2023-11-28T13:03:56.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.261+0000 7f884a7fc700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88380682f0 0x7f883806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.262+0000 7f884a7fc700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88380682f0 0x7f883806a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f883c007fd0 tx=0x7f883c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.266+0000 7f88337fe700 1 -- 172.21.15.6:0/1455057319 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f884004d5e0 con 0x7f88340052c0 2023-11-28T13:03:56.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.504+0000 7f88513b2700 1 -- 172.21.15.6:0/1455057319 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f884c0675a0 con 0x7f88340052c0 2023-11-28T13:03:56.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.504+0000 7f88337fe700 1 -- 172.21.15.6:0/1455057319 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4641 (secure 0 0 0) 0x7f8840019090 con 0x7f88340052c0 2023-11-28T13:03:56.506 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:56.506 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1198233336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1198233336},{"type":"v1","addr":"172.21.15.6:6836","nonce":1198233336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/235204249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":235204249},{"type":"v1","addr":"172.21.15.6:6837","nonce":235204249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2675481701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2675481701},{"type":"v1","addr":"172.21.15.61:6833","nonce":2675481701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:03:16.530114+0000","modified":"2023-11-28T13:03:29.044885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.61:6835/3076579711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":3076579711},{"type":"v1","addr":"172.21.15.61:6835","nonce":3076579711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:03:56.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 -- 172.21.15.6:0/1455057319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88380682f0 msgr2=0x7f883806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88380682f0 0x7f883806a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f883c007fd0 tx=0x7f883c009040 comp rx=0 tx=0).stop 2023-11-28T13:03:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 -- 172.21.15.6:0/1455057319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 msgr2=0x7f884c159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f884c159980 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f884000e780 tx=0x7f884000eb40 comp rx=0 tx=0).stop 2023-11-28T13:03:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 -- 172.21.15.6:0/1455057319 shutdown_connections 2023-11-28T13:03:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f88380682f0 0x7f883806a7a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f884c159580 0x7f884c06b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88340052c0 0x7f884c159980 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 --2- 172.21.15.6:0/1455057319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c0c5d50 0x7f884c159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 -- 172.21.15.6:0/1455057319 >> 172.21.15.6:0/1455057319 conn(0x7f884c0b1650 msgr2=0x7f884c06d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 -- 172.21.15.6:0/1455057319 shutdown_connections 2023-11-28T13:03:56.508 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.506+0000 7f88513b2700 1 -- 172.21.15.6:0/1455057319 wait complete. 2023-11-28T13:03:56.509 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 33 2023-11-28T13:03:56.519 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2023-11-28T13:03:56.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.646+0000 7f4ed888c700 1 Processor -- start 2023-11-28T13:03:56.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.646+0000 7f4ed888c700 1 -- start start 2023-11-28T13:03:56.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.647+0000 7f4ed888c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40b1630 0x7f4ed40cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.647+0000 7f4ed888c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4ed40b1a00 0x7f4ed40caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.647+0000 7f4ed888c700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed40ce550 con 0x7f4ed40cb3e0 2023-11-28T13:03:56.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.647+0000 7f4ed888c700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed40ce830 con 0x7f4ed40b1a00 2023-11-28T13:03:56.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.647+0000 7f4ed888c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40ceb10 con 0x7f4ed40b1630 2023-11-28T13:03:56.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.647+0000 7f4ed1d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4ed40b1a00 0x7f4ed40caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.647+0000 7f4ed1d9b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4ed40b1a00 0x7f4ed40caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34590/0 (socket says 172.21.15.6:34590) 2023-11-28T13:03:56.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.647+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/3626385298 learned_addr learned my addr 172.21.15.6:0/3626385298 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.647+0000 7f4ed259c700 1 --2- 172.21.15.6:0/3626385298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40b1630 0x7f4ed40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/3626385298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40b1630 msgr2=0x7f4ed40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/3626385298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40b1630 0x7f4ed40cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/3626385298 >> v1:172.21.15.61:6789/0 conn(0x7f4ed40cb3e0 legacy=0x7f4ed4160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/3626385298 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40cedf0 con 0x7f4ed40b1a00 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/3626385298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 0x7f4ed40caea0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80098f0 tx=0x7f4ec8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fe067e16c2b27c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed159a700 1 -- 172.21.15.6:0/3626385298 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec800b8a0 con 0x7f4ed40b1a00 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed159a700 1 -- 172.21.15.6:0/3626385298 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec800ba00 con 0x7f4ed40b1a00 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed159a700 1 -- 172.21.15.6:0/3626385298 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec800bd90 con 0x7f4ed40b1a00 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed888c700 1 -- 172.21.15.6:0/3626385298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 msgr2=0x7f4ed40caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed888c700 1 --2- 172.21.15.6:0/3626385298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 0x7f4ed40caea0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80098f0 tx=0x7f4ec8009c00 comp rx=0 tx=0).stop 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed888c700 1 -- 172.21.15.6:0/3626385298 shutdown_connections 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed888c700 1 --2- 172.21.15.6:0/3626385298 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40b1630 0x7f4ed40cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.648+0000 7f4ed888c700 1 --2- 172.21.15.6:0/3626385298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 0x7f4ed40caea0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.649+0000 7f4ed888c700 1 -- 172.21.15.6:0/3626385298 >> 172.21.15.6:0/3626385298 conn(0x7f4ed40cc630 msgr2=0x7f4ed40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.649+0000 7f4ed888c700 1 -- 172.21.15.6:0/3626385298 shutdown_connections 2023-11-28T13:03:56.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.649+0000 7f4ed888c700 1 -- 172.21.15.6:0/3626385298 wait complete. 2023-11-28T13:03:56.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.650+0000 7f4ed888c700 1 Processor -- start 2023-11-28T13:03:56.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.650+0000 7f4ed888c700 1 -- start start 2023-11-28T13:03:56.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed888c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ed40b1630 0x7f4ed40b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed888c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 0x7f4ed40c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed888c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40cb3e0 0x7f4ed40c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed888c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41e6070 con 0x7f4ed40b1a00 2023-11-28T13:03:56.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed888c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40b47e0 con 0x7f4ed40b1630 2023-11-28T13:03:56.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed888c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed40c3810 con 0x7f4ed40cb3e0 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 0x7f4ed40c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 0x7f4ed40c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34596/0 (socket says 172.21.15.6:34596) 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/2500834368 learned_addr learned my addr 172.21.15.6:0/2500834368 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ed40b1630 0x7f4ed40b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40cb3e0 0x7f4ed40c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/2500834368 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40cb3e0 msgr2=0x7f4ed40c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40cb3e0 0x7f4ed40c9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/2500834368 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ed40b1630 msgr2=0x7f4ed40b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ed40b1630 0x7f4ed40b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/2500834368 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec8009550 con 0x7f4ed40b1a00 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40cb3e0 0x7f4ed40c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:56.655 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ed40b1630 0x7f4ed40b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:03:56.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 0x7f4ed40c8e50 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8000c00 tx=0x7f4ec8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ec2ffd700 1 -- 172.21.15.6:0/2500834368 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8004480 con 0x7f4ed40b1a00 2023-11-28T13:03:56.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ec2ffd700 1 -- 172.21.15.6:0/2500834368 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec801f070 con 0x7f4ed40b1a00 2023-11-28T13:03:56.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed888c700 1 -- 172.21.15.6:0/2500834368 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed40b1e70 con 0x7f4ed40b1a00 2023-11-28T13:03:56.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ec2ffd700 1 -- 172.21.15.6:0/2500834368 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80038a0 con 0x7f4ed40b1a00 2023-11-28T13:03:56.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.651+0000 7f4ed888c700 1 -- 172.21.15.6:0/2500834368 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed40cf0d0 con 0x7f4ed40b1a00 2023-11-28T13:03:56.656 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.653+0000 7f4ec2ffd700 1 -- 172.21.15.6:0/2500834368 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4ec802a020 con 0x7f4ed40b1a00 2023-11-28T13:03:56.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.653+0000 7f4ec2ffd700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4eb40682f0 0x7f4eb406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.653+0000 7f4ec2ffd700 1 -- 172.21.15.6:0/2500834368 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4ec80876b0 con 0x7f4ed40b1a00 2023-11-28T13:03:56.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.653+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4eb40682f0 0x7f4eb406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.654+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4eb40682f0 0x7f4eb406a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009a00 tx=0x7f4ec4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.655+0000 7f4ee4a15b80 1 Processor -- start 2023-11-28T13:03:56.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.656+0000 7f4ee4a15b80 1 -- start start 2023-11-28T13:03:56.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.656+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce51ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.656+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 0x55d46ce7bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.656+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 0x55d46ce97810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.656+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46cd3bc60 con 0x55d46ce7b8b0 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.656+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46cdaa3f0 con 0x55d46ce791d0 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.656+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d46cda3920 con 0x55d46ce7c1c0 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.656+0000 7f4ed259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce51ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.656+0000 7f4ed259c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce51ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34602/0 (socket says 172.21.15.6:34602) 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.656+0000 7f4ed259c700 1 -- 172.21.15.6:0/2450022073 learned_addr learned my addr 172.21.15.6:0/2450022073 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/2450022073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 0x55d46ce7bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/2450022073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 0x55d46ce97810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4ed259c700 1 -- 172.21.15.6:0/2450022073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 msgr2=0x55d46ce97810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2450022073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 0x55d46ce97810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.660 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4ed259c700 1 -- 172.21.15.6:0/2450022073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 msgr2=0x55d46ce7bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2450022073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 0x55d46ce7bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4ed259c700 1 -- 172.21.15.6:0/2450022073 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d46cc25680 con 0x55d46ce7b8b0 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2450022073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce51ac0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400f920 tx=0x7f4ec4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02b657f217e159b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4eb3fff700 1 -- 172.21.15.6:0/2450022073 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec40147e0 con 0x55d46ce7b8b0 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4eb3fff700 1 -- 172.21.15.6:0/2450022073 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec4019070 con 0x55d46ce7b8b0 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.657+0000 7f4eb3fff700 1 -- 172.21.15.6:0/2450022073 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4014b40 con 0x55d46ce7b8b0 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2450022073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 msgr2=0x55d46ce51ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2450022073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce51ac0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400f920 tx=0x7f4ec4012000 comp rx=0 tx=0).stop 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2450022073 shutdown_connections 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2450022073 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 0x55d46ce97810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2450022073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce51ac0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.661 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2450022073 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 0x55d46ce7bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2450022073 >> 172.21.15.6:0/2450022073 conn(0x55d46ce4ee00 msgr2=0x55d46ce4f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2450022073 shutdown_connections 2023-11-28T13:03:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2450022073 wait complete. 2023-11-28T13:03:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 -- start start 2023-11-28T13:03:56.662 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.658+0000 7f4ee4a15b80 1 Processor -- start 2023-11-28T13:03:56.665 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.659+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 0x55d46ce6c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.659+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce7b8b0 0x55d46ce7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.659+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.659+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46cc27970 con 0x55d46ce7c1c0 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.659+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46ce700c0 con 0x55d46ce7b8b0 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.659+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d46ce7e470 con 0x55d46ce791d0 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.659+0000 7f4ee4a15b80 10 client.0 _ll_register_callbacks cb 0x55d46cc492e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.659+0000 7f4ed2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.659+0000 7f4ed2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34612/0 (socket says 172.21.15.6:34612) 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.659+0000 7f4ed2d9d700 1 -- 172.21.15.6:0/2883850588 learned_addr learned my addr 172.21.15.6:0/2883850588 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 0x55d46ce6c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.666 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce7b8b0 0x55d46ce7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ed2d9d700 1 -- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 msgr2=0x55d46ce6c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 0x55d46ce6c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ed2d9d700 1 -- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce7b8b0 msgr2=0x55d46ce7aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce7b8b0 0x55d46ce7aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ed2d9d700 1 -- 172.21.15.6:0/2883850588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec400e040 con 0x55d46ce7c1c0 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce715d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00e810 tx=0x7f4ecc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6e7268b07ae959 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4eb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4eb17fa700 1 -- 172.21.15.6:0/2883850588 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc013870 con 0x55d46ce7c1c0 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4eb17fa700 1 -- 172.21.15.6:0/2883850588 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4ecc0139d0 con 0x55d46ce7c1c0 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2883850588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d46cd711e0 con 0x55d46ce7c1c0 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4eb17fa700 1 -- 172.21.15.6:0/2883850588 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ecc013b90 con 0x55d46ce7c1c0 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ee4a15b80 10 client.6741 fetch_fsmap learned FSMap version 33 2023-11-28T13:03:56.667 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2883850588 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55d46cbd1e30 con 0x55d46ce7c1c0 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4eb17fa700 1 -- 172.21.15.6:0/2883850588 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d46cd711e0 con 0x55d46ce7c1c0 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4eb17fa700 1 -- 172.21.15.6:0/2883850588 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4ecc013cf0 con 0x55d46ce7c1c0 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ee4a15b80 10 client.6741 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.660+0000 7f4ee4a15b80 10 client.6741 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.661+0000 7f4ee4a15b80 20 client.6741 populate_metadata read hostname 'smithi006' 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.661+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55d46ce6e140 0x55d46ceb3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.661+0000 7f4ee4a15b80 4 client.6741 mds_command: new command op to 6471 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.661+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2883850588 --> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d46cc27fb0 con 0x55d46ce6e140 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.661+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55d46ce6e140 0x55d46ceb3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.661+0000 7f4eb17fa700 10 client.6741 ms_handle_connect on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.661+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55d46ce6e140 0x55d46ceb3760 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72bf681b1d9ba6c0 server_cookie=ad41d49b44d584b6 in_seq=0 out_seq=0 2023-11-28T13:03:56.668 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4eb17fa700 1 -- 172.21.15.6:0/2883850588 <== mds.0 v2:172.21.15.61:6834/3076579711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55d46cc27fb0 con 0x55d46ce6e140 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4eb17fa700 10 client.6741 handle_command_reply: tid=0 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 client.6741 shutdown 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2883850588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 msgr2=0x55d46ce715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce715d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00e810 tx=0x7f4ecc00ebd0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2883850588 shutdown_connections 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55d46ce6e140 0x55d46ceb3760 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce715d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 0x55d46ce6c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2883850588 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce7b8b0 0x55d46ce7aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4eb17fa700 0 client.6741 ms_handle_reset on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2883850588 >> 172.21.15.6:0/2883850588 conn(0x55d46ce4ee00 msgr2=0x55d46ce502d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2883850588 shutdown_connections 2023-11-28T13:03:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.662+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2883850588 wait complete. 2023-11-28T13:03:56.670 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.663+0000 7f4ee4a15b80 20 client.6741 trim_cache size 0 max 16384 2023-11-28T13:03:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.675+0000 7f4ee4a15b80 1 Processor -- start 2023-11-28T13:03:56.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.675+0000 7f4ee4a15b80 1 -- start start 2023-11-28T13:03:56.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 0x55d46ce70300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.679 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce52aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 0x55d46ceb7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46ce7a830 con 0x55d46ce7b8b0 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46ce7d560 con 0x55d46ce791d0 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d46ce6c000 con 0x55d46ce7c1c0 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ed1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce52aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ed1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce52aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34618/0 (socket says 172.21.15.6:34618) 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/2473950137 learned_addr learned my addr 172.21.15.6:0/2473950137 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 0x55d46ceb7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.676+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 0x55d46ce70300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/2473950137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 msgr2=0x55d46ceb7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 0x55d46ceb7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/2473950137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 msgr2=0x55d46ce70300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 0x55d46ce70300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ed1d9b700 1 -- 172.21.15.6:0/2473950137 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d46cd6a1e0 con 0x55d46ce7b8b0 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ed259c700 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 0x55d46ce70300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 0x55d46ceb7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce52aa0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8030030 tx=0x7f4ec8057710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8eb98f163afc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ec0ff9700 1 -- 172.21.15.6:0/2473950137 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec807fea0 con 0x55d46ce7b8b0 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ec0ff9700 1 -- 172.21.15.6:0/2473950137 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec802c390 con 0x55d46ce7b8b0 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ec0ff9700 1 -- 172.21.15.6:0/2473950137 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80570d0 con 0x55d46ce7b8b0 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2473950137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 msgr2=0x55d46ce52aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce52aa0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8030030 tx=0x7f4ec8057710 comp rx=0 tx=0).stop 2023-11-28T13:03:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2473950137 shutdown_connections 2023-11-28T13:03:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce7c1c0 0x55d46ceb7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce791d0 0x55d46ce70300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/2473950137 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7b8b0 0x55d46ce52aa0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.677+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2473950137 >> 172.21.15.6:0/2473950137 conn(0x55d46ce4ee00 msgr2=0x55d46ce502d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.678+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2473950137 shutdown_connections 2023-11-28T13:03:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.678+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/2473950137 wait complete. 2023-11-28T13:03:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.678+0000 7f4ee4a15b80 1 -- start start 2023-11-28T13:03:56.682 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.678+0000 7f4ee4a15b80 1 Processor -- start 2023-11-28T13:03:56.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 0x55d46ce4bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce7b8b0 0x55d46cefb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ee4a15b80 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce4a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46ce4c400 con 0x55d46ce7c1c0 2023-11-28T13:03:56.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46ce72000 con 0x55d46ce7b8b0 2023-11-28T13:03:56.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ee4a15b80 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d46ce69750 con 0x55d46ce791d0 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ee4a15b80 10 client.0 _ll_register_callbacks cb 0x55d46cc492e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ed2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce4a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ed2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce4a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34626/0 (socket says 172.21.15.6:34626) 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ed2d9d700 1 -- 172.21.15.6:0/108819554 learned_addr learned my addr 172.21.15.6:0/108819554 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ed259c700 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 0x55d46ce4bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ed1d9b700 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce7b8b0 0x55d46cefb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ed2d9d700 1 -- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 msgr2=0x55d46ce4bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 0x55d46ce4bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ed2d9d700 1 -- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce7b8b0 msgr2=0x55d46cefb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce7b8b0 0x55d46cefb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.679+0000 7f4ed2d9d700 1 -- 172.21.15.6:0/108819554 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed40cf0d0 con 0x55d46ce7c1c0 2023-11-28T13:03:56.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ed259c700 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 0x55d46ce4bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ed2d9d700 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce4a4f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc001bd0 tx=0x7f4ecc001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1712600b12b0770 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4eb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.6:3300/0 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4eb17fa700 1 -- 172.21.15.6:0/108819554 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc003c00 con 0x55d46ce7c1c0 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4eb17fa700 1 -- 172.21.15.6:0/108819554 <== mon.0 v2:172.21.15.6:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4ecc003d60 con 0x55d46ce7c1c0 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/108819554 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d46ce7c050 con 0x55d46ce7c1c0 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4eb17fa700 1 -- 172.21.15.6:0/108819554 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ecc0050d0 con 0x55d46ce7c1c0 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ee4a15b80 10 client.6747 fetch_fsmap learned FSMap version 33 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/108819554 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55d46cbf82a0 con 0x55d46ce7c1c0 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4eb17fa700 1 -- 172.21.15.6:0/108819554 <== mon.0 v2:172.21.15.6:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d46ce7c050 con 0x55d46ce7c1c0 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4eb17fa700 1 -- 172.21.15.6:0/108819554 <== mon.0 v2:172.21.15.6:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f4ecc02fd20 con 0x55d46ce7c1c0 2023-11-28T13:03:56.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ee4a15b80 10 client.6747 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ee4a15b80 10 client.6747 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ee4a15b80 20 client.6747 populate_metadata read hostname 'smithi006' 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55d46ce98f30 0x55d46ce99300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ee4a15b80 4 client.6747 mds_command: new command op to 6471 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/108819554 --> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55d46cd61200 con 0x55d46ce98f30 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.680+0000 7f4ed259c700 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55d46ce98f30 0x55d46ce99300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4eb17fa700 10 client.6747 ms_handle_connect on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ed259c700 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55d46ce98f30 0x55d46ce99300 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71fdc9af5192d867 server_cookie=383b0c51f86042ae in_seq=0 out_seq=0 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4eb17fa700 1 -- 172.21.15.6:0/108819554 <== mds.0 v2:172.21.15.61:6834/3076579711 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x55d46cd61200 con 0x55d46ce98f30 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4eb17fa700 10 client.6747 handle_command_reply: tid=0 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 client.6747 shutdown 2023-11-28T13:03:56.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/108819554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 msgr2=0x55d46ce4a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce4a4f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc001bd0 tx=0x7f4ecc001400 comp rx=0 tx=0).stop 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/108819554 shutdown_connections 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:6834/3076579711,v1:172.21.15.61:6835/3076579711] conn(0x55d46ce98f30 0x55d46ce99300 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55d46ce7c1c0 0x55d46ce4a4f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x55d46ce791d0 0x55d46ce4bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4eb17fa700 0 client.6747 ms_handle_reset on v2:172.21.15.61:6834/3076579711 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 --2- 172.21.15.6:0/108819554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d46ce7b8b0 0x55d46cefb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/108819554 >> 172.21.15.6:0/108819554 conn(0x55d46ce4ee00 msgr2=0x55d46ceb7810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/108819554 shutdown_connections 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 1 -- 172.21.15.6:0/108819554 wait complete. 2023-11-28T13:03:56.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.681+0000 7f4ee4a15b80 20 client.6747 trim_cache size 0 max 16384 2023-11-28T13:03:56.691 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:56.691 INFO:teuthology.orchestra.run.smithi006.stdout:[ 2023-11-28T13:03:56.691 INFO:teuthology.orchestra.run.smithi006.stdout: { 2023-11-28T13:03:56.691 INFO:teuthology.orchestra.run.smithi006.stdout: "damage_type": "backtrace", 2023-11-28T13:03:56.691 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 2052026661, 2023-11-28T13:03:56.691 INFO:teuthology.orchestra.run.smithi006.stdout: "ino": 1099511627777, 2023-11-28T13:03:56.691 INFO:teuthology.orchestra.run.smithi006.stdout: "path": "/dir_test/file" 2023-11-28T13:03:56.691 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-28T13:03:56.691 INFO:teuthology.orchestra.run.smithi006.stdout:] 2023-11-28T13:03:56.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 -- 172.21.15.6:0/2500834368 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4eb40682f0 msgr2=0x7f4eb406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4eb40682f0 0x7f4eb406a7a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009a00 tx=0x7f4ec4008040 comp rx=0 tx=0).stop 2023-11-28T13:03:56.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 -- 172.21.15.6:0/2500834368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 msgr2=0x7f4ed40c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 0x7f4ed40c8e50 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8000c00 tx=0x7f4ec8004750 comp rx=0 tx=0).stop 2023-11-28T13:03:56.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 -- 172.21.15.6:0/2500834368 shutdown_connections 2023-11-28T13:03:56.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4eb40682f0 0x7f4eb406a7a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4ed40cb3e0 0x7f4ed40c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ed40b1630 0x7f4ed40b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 --2- 172.21.15.6:0/2500834368 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ed40b1a00 0x7f4ed40c8e50 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 -- 172.21.15.6:0/2500834368 >> 172.21.15.6:0/2500834368 conn(0x7f4ed40cc630 msgr2=0x7f4ed406b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 -- 172.21.15.6:0/2500834368 shutdown_connections 2023-11-28T13:03:56.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.682+0000 7f4ed888c700 1 -- 172.21.15.6:0/2500834368 wait complete. 2023-11-28T13:03:56.695 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:03:56.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.817+0000 7fac31f6f700 1 Processor -- start 2023-11-28T13:03:56.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.817+0000 7fac31f6f700 1 -- start start 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac31f6f700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fac2c0cb220 0x7fac2c0b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac31f6f700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c15dbd0 con 0x7fac2c0cb6c0 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac31f6f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c15d610 con 0x7fac2c070520 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac31f6f700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c15d8f0 con 0x7fac2c0cb220 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac2bfff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fac2c0cb6c0 0x7fac2c15c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50186/0 (socket says 172.21.15.6:50186) 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac2bfff700 1 -- 172.21.15.6:0/3986690230 learned_addr learned my addr 172.21.15.6:0/3986690230 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac2affd700 1 --2- 172.21.15.6:0/3986690230 >> v2:172.21.15.61:3300/0 conn(0x7fac2c0cb220 0x7fac2c0b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac2a7fc700 1 -- 172.21.15.6:0/3986690230 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748751268 0 0) 0x7fac2c15dbd0 con 0x7fac2c0cb6c0 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac2a7fc700 1 -- 172.21.15.6:0/3986690230 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac140035b0 con 0x7fac2c0cb6c0 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac2a7fc700 1 -- 172.21.15.6:0/3986690230 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 572507247 0 0) 0x7fac140035b0 con 0x7fac2c0cb6c0 2023-11-28T13:03:56.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac2affd700 1 -- 172.21.15.6:0/3986690230 >> v1:172.21.15.61:6789/0 conn(0x7fac2c070520 legacy=0x7fac2c0708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac2affd700 1 -- 172.21.15.6:0/3986690230 >> v1:172.21.15.6:6789/0 conn(0x7fac2c0cb6c0 legacy=0x7fac2c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.818+0000 7fac2affd700 1 -- 172.21.15.6:0/3986690230 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c0b0290 con 0x7fac2c0cb220 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac2affd700 1 --2- 172.21.15.6:0/3986690230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac2c0cb220 0x7fac2c0b0a90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fac20009840 tx=0x7fac20009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b197459f3e5d81f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac2a7fc700 1 -- 172.21.15.6:0/3986690230 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac240019f0 con 0x7fac2c0cb220 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac2a7fc700 1 -- 172.21.15.6:0/3986690230 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac20004030 con 0x7fac2c0cb220 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac2a7fc700 1 -- 172.21.15.6:0/3986690230 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac200043c0 con 0x7fac2c0cb220 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac31f6f700 1 -- 172.21.15.6:0/3986690230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac2c0cb220 msgr2=0x7fac2c0b0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac31f6f700 1 --2- 172.21.15.6:0/3986690230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac2c0cb220 0x7fac2c0b0a90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fac20009840 tx=0x7fac20009b50 comp rx=0 tx=0).stop 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac31f6f700 1 -- 172.21.15.6:0/3986690230 shutdown_connections 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac31f6f700 1 --2- 172.21.15.6:0/3986690230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac2c0cb220 0x7fac2c0b0a90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac31f6f700 1 -- 172.21.15.6:0/3986690230 >> 172.21.15.6:0/3986690230 conn(0x7fac2c0d0bd0 msgr2=0x7fac2c0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:56.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac31f6f700 1 -- 172.21.15.6:0/3986690230 shutdown_connections 2023-11-28T13:03:56.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.819+0000 7fac31f6f700 1 -- 172.21.15.6:0/3986690230 wait complete. 2023-11-28T13:03:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.820+0000 7fac31f6f700 1 Processor -- start 2023-11-28T13:03:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.821+0000 7fac31f6f700 1 -- start start 2023-11-28T13:03:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.821+0000 7fac31f6f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac2c070520 0x7fac2c0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.821+0000 7fac31f6f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac2c0cb220 0x7fac2c0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.821+0000 7fac31f6f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac2c0cb6c0 0x7fac2c0c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.821+0000 7fac31f6f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c0b69e0 con 0x7fac2c0cb220 2023-11-28T13:03:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.821+0000 7fac31f6f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c0b6700 con 0x7fac2c070520 2023-11-28T13:03:56.826 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.821+0000 7fac31f6f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c15a010 con 0x7fac2c0cb6c0 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac2c0cb220 0x7fac2c0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac2c0cb220 0x7fac2c0b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34638/0 (socket says 172.21.15.6:34638) 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2affd700 1 -- 172.21.15.6:0/900402480 learned_addr learned my addr 172.21.15.6:0/900402480 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2bfff700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac2c0cb6c0 0x7fac2c0c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2b7fe700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac2c070520 0x7fac2c0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2affd700 1 -- 172.21.15.6:0/900402480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac2c0cb6c0 msgr2=0x7fac2c0c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2affd700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac2c0cb6c0 0x7fac2c0c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2affd700 1 -- 172.21.15.6:0/900402480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac2c070520 msgr2=0x7fac2c0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2affd700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac2c070520 0x7fac2c0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2affd700 1 -- 172.21.15.6:0/900402480 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200094a0 con 0x7fac2c0cb220 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac2affd700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac2c0cb220 0x7fac2c0b6000 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fac2000a010 tx=0x7fac20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac13fff700 1 -- 172.21.15.6:0/900402480 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20011630 con 0x7fac2c0cb220 2023-11-28T13:03:56.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac13fff700 1 -- 172.21.15.6:0/900402480 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2001f070 con 0x7fac2c0cb220 2023-11-28T13:03:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac31f6f700 1 -- 172.21.15.6:0/900402480 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2c0b2360 con 0x7fac2c0cb220 2023-11-28T13:03:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac13fff700 1 -- 172.21.15.6:0/900402480 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac200119f0 con 0x7fac2c0cb220 2023-11-28T13:03:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.822+0000 7fac31f6f700 1 -- 172.21.15.6:0/900402480 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c0b0570 con 0x7fac2c0cb220 2023-11-28T13:03:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.823+0000 7fac31f6f700 1 -- 172.21.15.6:0/900402480 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2c0b24f0 con 0x7fac2c0cb220 2023-11-28T13:03:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.824+0000 7fac13fff700 1 -- 172.21.15.6:0/900402480 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fac20017020 con 0x7fac2c0cb220 2023-11-28T13:03:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.824+0000 7fac13fff700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac180682f0 0x7fac1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:03:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.824+0000 7fac13fff700 1 -- 172.21.15.6:0/900402480 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fac20021070 con 0x7fac2c0cb220 2023-11-28T13:03:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.824+0000 7fac2b7fe700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac180682f0 0x7fac1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:03:56.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.825+0000 7fac2b7fe700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac180682f0 0x7fac1806a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac2c15bb60 tx=0x7fac1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:03:56.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:56.828+0000 7fac13fff700 1 -- 172.21.15.6:0/900402480 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fac20053f10 con 0x7fac2c0cb220 2023-11-28T13:03:56.974 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:03:56.977 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:03:57.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.066+0000 7fac31f6f700 1 -- 172.21.15.6:0/900402480 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fac2c066c60 con 0x7fac2c0cb220 2023-11-28T13:03:57.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.066+0000 7fac13fff700 1 -- 172.21.15.6:0/900402480 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fac20057530 con 0x7fac2c0cb220 2023-11-28T13:03:57.068 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:03:57.068 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:03:57.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 -- 172.21.15.6:0/900402480 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac180682f0 msgr2=0x7fac1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:57.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac180682f0 0x7fac1806a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac2c15bb60 tx=0x7fac1c009040 comp rx=0 tx=0).stop 2023-11-28T13:03:57.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 -- 172.21.15.6:0/900402480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac2c0cb220 msgr2=0x7fac2c0b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:03:57.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac2c0cb220 0x7fac2c0b6000 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fac2000a010 tx=0x7fac20004750 comp rx=0 tx=0).stop 2023-11-28T13:03:57.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 -- 172.21.15.6:0/900402480 shutdown_connections 2023-11-28T13:03:57.070 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac180682f0 0x7fac1806a7a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:57.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac2c0cb6c0 0x7fac2c0c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:57.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac2c070520 0x7fac2c0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:57.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 --2- 172.21.15.6:0/900402480 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac2c0cb220 0x7fac2c0b6000 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:03:57.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 -- 172.21.15.6:0/900402480 >> 172.21.15.6:0/900402480 conn(0x7fac2c0d0bd0 msgr2=0x7fac2c157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:03:57.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.068+0000 7fac31f6f700 1 -- 172.21.15.6:0/900402480 shutdown_connections 2023-11-28T13:03:57.071 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:03:57.069+0000 7fac31f6f700 1 -- 172.21.15.6:0/900402480 wait complete. 2023-11-28T13:03:57.081 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:02.082 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:05.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.182+0000 7f8399c15700 1 Processor -- start 2023-11-28T13:04:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.182+0000 7f8399c15700 1 -- start start 2023-11-28T13:04:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.183+0000 7f8399c15700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f839415c8a0 0x7f83940b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.183+0000 7f8399c15700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f83940b2290 0x7f83940ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.183+0000 7f8399c15700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83940d0280 con 0x7f839406f120 2023-11-28T13:04:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.183+0000 7f8399c15700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83940d0560 con 0x7f83940b2290 2023-11-28T13:04:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.183+0000 7f8399c15700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83940d0840 con 0x7f839415c8a0 2023-11-28T13:04:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.183+0000 7f8392ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f83940b2290 0x7f83940ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:05.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.183+0000 7f8392ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f83940b2290 0x7f83940ac660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34644/0 (socket says 172.21.15.6:34644) 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.183+0000 7f8392ffd700 1 -- 172.21.15.6:0/159659437 learned_addr learned my addr 172.21.15.6:0/159659437 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f8392ffd700 1 -- 172.21.15.6:0/159659437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f839415c8a0 msgr2=0x7f83940b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f8392ffd700 1 --2- 172.21.15.6:0/159659437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f839415c8a0 0x7f83940b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f8392ffd700 1 -- 172.21.15.6:0/159659437 >> v1:172.21.15.61:6789/0 conn(0x7f839406f120 legacy=0x7f83940cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f8392ffd700 1 -- 172.21.15.6:0/159659437 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83940b1550 con 0x7f83940b2290 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f8392ffd700 1 --2- 172.21.15.6:0/159659437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83940b2290 0x7f83940ac660 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f83880098f0 tx=0x7f8388009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f213f5ebb0a06305 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f83927fc700 1 -- 172.21.15.6:0/159659437 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388004230 con 0x7f83940b2290 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f83927fc700 1 -- 172.21.15.6:0/159659437 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388004390 con 0x7f83940b2290 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f83927fc700 1 -- 172.21.15.6:0/159659437 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83880036a0 con 0x7f83940b2290 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f8399c15700 1 -- 172.21.15.6:0/159659437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83940b2290 msgr2=0x7f83940ac660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f8399c15700 1 --2- 172.21.15.6:0/159659437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83940b2290 0x7f83940ac660 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f83880098f0 tx=0x7f8388009c00 comp rx=0 tx=0).stop 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f8399c15700 1 -- 172.21.15.6:0/159659437 shutdown_connections 2023-11-28T13:04:05.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f8399c15700 1 --2- 172.21.15.6:0/159659437 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f839415c8a0 0x7f83940b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.184+0000 7f8399c15700 1 --2- 172.21.15.6:0/159659437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f83940b2290 0x7f83940ac660 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.185+0000 7f8399c15700 1 -- 172.21.15.6:0/159659437 >> 172.21.15.6:0/159659437 conn(0x7f83940b3610 msgr2=0x7f83940cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.185+0000 7f8399c15700 1 -- 172.21.15.6:0/159659437 shutdown_connections 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.185+0000 7f8399c15700 1 -- 172.21.15.6:0/159659437 wait complete. 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.186+0000 7f8399c15700 1 Processor -- start 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.186+0000 7f8399c15700 1 -- start start 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8399c15700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f839406f120 0x7f83941f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8399c15700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83940b2290 0x7f8394152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8399c15700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f839415c8a0 0x7f8394153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8399c15700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8394153680 con 0x7f839415c8a0 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8399c15700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83940c4d60 con 0x7f839406f120 2023-11-28T13:04:05.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8399c15700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83940c4150 con 0x7f83940b2290 2023-11-28T13:04:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8393fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f839415c8a0 0x7f8394153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8393fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f839415c8a0 0x7f8394153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34652/0 (socket says 172.21.15.6:34652) 2023-11-28T13:04:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8393fff700 1 -- 172.21.15.6:0/1722777261 learned_addr learned my addr 172.21.15.6:0/1722777261 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f83937fe700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f839406f120 0x7f83941f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8392ffd700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83940b2290 0x7f8394152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8393fff700 1 -- 172.21.15.6:0/1722777261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83940b2290 msgr2=0x7f8394152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8393fff700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83940b2290 0x7f8394152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8393fff700 1 -- 172.21.15.6:0/1722777261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f839406f120 msgr2=0x7f83941f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8393fff700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f839406f120 0x7f83941f9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8393fff700 1 -- 172.21.15.6:0/1722777261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8388009550 con 0x7f839415c8a0 2023-11-28T13:04:05.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.187+0000 7f8393fff700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f839415c8a0 0x7f8394153140 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c7a0 tx=0x7f838c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:05.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.188+0000 7f837bfff700 1 -- 172.21.15.6:0/1722777261 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c003e10 con 0x7f839415c8a0 2023-11-28T13:04:05.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.188+0000 7f837bfff700 1 -- 172.21.15.6:0/1722777261 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c005070 con 0x7f839415c8a0 2023-11-28T13:04:05.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.188+0000 7f8399c15700 1 -- 172.21.15.6:0/1722777261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839415cc70 con 0x7f839415c8a0 2023-11-28T13:04:05.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.188+0000 7f837bfff700 1 -- 172.21.15.6:0/1722777261 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c005420 con 0x7f839415c8a0 2023-11-28T13:04:05.194 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.188+0000 7f8399c15700 1 -- 172.21.15.6:0/1722777261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83940b1830 con 0x7f839415c8a0 2023-11-28T13:04:05.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.189+0000 7f8399c15700 1 -- 172.21.15.6:0/1722777261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839406a6c0 con 0x7f839415c8a0 2023-11-28T13:04:05.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.189+0000 7f837bfff700 1 -- 172.21.15.6:0/1722777261 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f838c019030 con 0x7f839415c8a0 2023-11-28T13:04:05.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.189+0000 7f837bfff700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f83800682f0 0x7f838006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:05.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.190+0000 7f837bfff700 1 -- 172.21.15.6:0/1722777261 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f838c085220 con 0x7f839415c8a0 2023-11-28T13:04:05.195 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.190+0000 7f83937fe700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f83800682f0 0x7f838006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.190+0000 7f83937fe700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f83800682f0 0x7f838006a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8384009970 tx=0x7f8384009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:05.196 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.194+0000 7f837bfff700 1 -- 172.21.15.6:0/1722777261 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f838c051940 con 0x7f839415c8a0 2023-11-28T13:04:05.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.432+0000 7f8399c15700 1 -- 172.21.15.6:0/1722777261 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8394066c60 con 0x7f839415c8a0 2023-11-28T13:04:05.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.432+0000 7f837bfff700 1 -- 172.21.15.6:0/1722777261 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f838c0170c0 con 0x7f839415c8a0 2023-11-28T13:04:05.433 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:05.434 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:05.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 -- 172.21.15.6:0/1722777261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f83800682f0 msgr2=0x7f838006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f83800682f0 0x7f838006a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8384009970 tx=0x7f8384009110 comp rx=0 tx=0).stop 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 -- 172.21.15.6:0/1722777261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f839415c8a0 msgr2=0x7f8394153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f839415c8a0 0x7f8394153140 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c7a0 tx=0x7f838c00cab0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 -- 172.21.15.6:0/1722777261 shutdown_connections 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f839415c8a0 0x7f8394153140 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f839406f120 0x7f83941f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f83800682f0 0x7f838006a7a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 --2- 172.21.15.6:0/1722777261 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f83940b2290 0x7f8394152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 -- 172.21.15.6:0/1722777261 >> 172.21.15.6:0/1722777261 conn(0x7f83940b3610 msgr2=0x7f83940cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 -- 172.21.15.6:0/1722777261 shutdown_connections 2023-11-28T13:04:05.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:05.434+0000 7f8399c15700 1 -- 172.21.15.6:0/1722777261 wait complete. 2023-11-28T13:04:05.447 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:10.448 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:10.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.584+0000 7fed8e240700 1 Processor -- start 2023-11-28T13:04:10.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.584+0000 7fed8e240700 1 -- start start 2023-11-28T13:04:10.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.585+0000 7fed8e240700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fed880b6510 0x7fed880cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:10.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.585+0000 7fed8e240700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fed880ccfe0 0x7fed88070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:10.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.585+0000 7fed8e240700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed880ce9b0 con 0x7fed88070a60 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.585+0000 7fed8e240700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed88070ee0 con 0x7fed880b6510 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.585+0000 7fed8e240700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880b0d30 con 0x7fed880ccfe0 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.585+0000 7fed877fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fed880b6510 0x7fed880cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.585+0000 7fed877fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fed880b6510 0x7fed880cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34656/0 (socket says 172.21.15.6:34656) 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.585+0000 7fed877fe700 1 -- 172.21.15.6:0/583593063 learned_addr learned my addr 172.21.15.6:0/583593063 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.586+0000 7fed86ffd700 1 --2- 172.21.15.6:0/583593063 >> v2:172.21.15.61:3300/0 conn(0x7fed880ccfe0 0x7fed88070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.586+0000 7fed877fe700 1 -- 172.21.15.6:0/583593063 >> v2:172.21.15.61:3300/0 conn(0x7fed880ccfe0 msgr2=0x7fed88070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.586+0000 7fed877fe700 1 --2- 172.21.15.6:0/583593063 >> v2:172.21.15.61:3300/0 conn(0x7fed880ccfe0 0x7fed88070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.586+0000 7fed877fe700 1 -- 172.21.15.6:0/583593063 >> v1:172.21.15.61:6789/0 conn(0x7fed88070a60 legacy=0x7fed8815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.586+0000 7fed877fe700 1 -- 172.21.15.6:0/583593063 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed880b0fd0 con 0x7fed880b6510 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.586+0000 7fed877fe700 1 --2- 172.21.15.6:0/583593063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed880b6510 0x7fed880cb220 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fed880c7be0 tx=0x7fed78009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376e0ade555fbbf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:10.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.586+0000 7fed867fc700 1 -- 172.21.15.6:0/583593063 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7800b870 con 0x7fed880b6510 2023-11-28T13:04:10.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.586+0000 7fed867fc700 1 -- 172.21.15.6:0/583593063 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7800b9d0 con 0x7fed880b6510 2023-11-28T13:04:10.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.586+0000 7fed867fc700 1 -- 172.21.15.6:0/583593063 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7800bd60 con 0x7fed880b6510 2023-11-28T13:04:10.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.587+0000 7fed8e240700 1 -- 172.21.15.6:0/583593063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed880b6510 msgr2=0x7fed880cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:10.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.587+0000 7fed8e240700 1 --2- 172.21.15.6:0/583593063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed880b6510 0x7fed880cb220 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fed880c7be0 tx=0x7fed78009bd0 comp rx=0 tx=0).stop 2023-11-28T13:04:10.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.587+0000 7fed8e240700 1 -- 172.21.15.6:0/583593063 shutdown_connections 2023-11-28T13:04:10.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.587+0000 7fed8e240700 1 --2- 172.21.15.6:0/583593063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed880b6510 0x7fed880cb220 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:10.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.587+0000 7fed8e240700 1 --2- 172.21.15.6:0/583593063 >> v2:172.21.15.61:3300/0 conn(0x7fed880ccfe0 0x7fed88070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:10.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.587+0000 7fed8e240700 1 -- 172.21.15.6:0/583593063 >> 172.21.15.6:0/583593063 conn(0x7fed880d0bd0 msgr2=0x7fed880d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:10.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.587+0000 7fed8e240700 1 -- 172.21.15.6:0/583593063 shutdown_connections 2023-11-28T13:04:10.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.587+0000 7fed8e240700 1 -- 172.21.15.6:0/583593063 wait complete. 2023-11-28T13:04:10.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.588+0000 7fed8e240700 1 Processor -- start 2023-11-28T13:04:10.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.588+0000 7fed8e240700 1 -- start start 2023-11-28T13:04:10.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed8e240700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed88070a60 0x7fed880af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:10.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed8e240700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed880b6510 0x7fed8806cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:10.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed8e240700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed880ccfe0 0x7fed88156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:10.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed8e240700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8815a9b0 con 0x7fed88070a60 2023-11-28T13:04:10.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed8e240700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8815ac90 con 0x7fed880b6510 2023-11-28T13:04:10.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed8e240700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8815af70 con 0x7fed880ccfe0 2023-11-28T13:04:10.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed877fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed88070a60 0x7fed880af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:10.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed877fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed88070a60 0x7fed880af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34660/0 (socket says 172.21.15.6:34660) 2023-11-28T13:04:10.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed877fe700 1 -- 172.21.15.6:0/1810002532 learned_addr learned my addr 172.21.15.6:0/1810002532 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:10.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed86ffd700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed880b6510 0x7fed8806cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:10.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed87fff700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed880ccfe0 0x7fed88156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:10.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed877fe700 1 -- 172.21.15.6:0/1810002532 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed880ccfe0 msgr2=0x7fed88156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:10.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed877fe700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed880ccfe0 0x7fed88156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed877fe700 1 -- 172.21.15.6:0/1810002532 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed880b6510 msgr2=0x7fed8806cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed877fe700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed880b6510 0x7fed8806cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed877fe700 1 -- 172.21.15.6:0/1810002532 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed78009550 con 0x7fed88070a60 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed86ffd700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed880b6510 0x7fed8806cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.589+0000 7fed87fff700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed880ccfe0 0x7fed88156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.590+0000 7fed877fe700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed88070a60 0x7fed880af1b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fed78000c00 tx=0x7fed78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.590+0000 7fed7ffff700 1 -- 172.21.15.6:0/1810002532 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78004540 con 0x7fed88070a60 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.590+0000 7fed7ffff700 1 -- 172.21.15.6:0/1810002532 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed780036a0 con 0x7fed88070a60 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.590+0000 7fed7ffff700 1 -- 172.21.15.6:0/1810002532 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed78003a30 con 0x7fed88070a60 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.590+0000 7fed8e240700 1 -- 172.21.15.6:0/1810002532 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed880cb760 con 0x7fed88070a60 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.590+0000 7fed8e240700 1 -- 172.21.15.6:0/1810002532 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed880b12b0 con 0x7fed88070a60 2023-11-28T13:04:10.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.591+0000 7fed8e240700 1 -- 172.21.15.6:0/1810002532 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed880cfe50 con 0x7fed88070a60 2023-11-28T13:04:10.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.591+0000 7fed7ffff700 1 -- 172.21.15.6:0/1810002532 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fed7800f050 con 0x7fed88070a60 2023-11-28T13:04:10.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.592+0000 7fed7ffff700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed680682f0 0x7fed6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:10.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.592+0000 7fed86ffd700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed680682f0 0x7fed6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:10.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.592+0000 7fed7ffff700 1 -- 172.21.15.6:0/1810002532 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fed78087790 con 0x7fed88070a60 2023-11-28T13:04:10.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.592+0000 7fed86ffd700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed680682f0 0x7fed6806a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fed880c7420 tx=0x7fed70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:10.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.596+0000 7fed7ffff700 1 -- 172.21.15.6:0/1810002532 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fed78053e70 con 0x7fed88070a60 2023-11-28T13:04:10.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.833+0000 7fed8e240700 1 -- 172.21.15.6:0/1810002532 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fed880675a0 con 0x7fed88070a60 2023-11-28T13:04:10.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.833+0000 7fed7ffff700 1 -- 172.21.15.6:0/1810002532 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fed78057490 con 0x7fed88070a60 2023-11-28T13:04:10.835 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:10.835 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:10.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 -- 172.21.15.6:0/1810002532 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed680682f0 msgr2=0x7fed6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed680682f0 0x7fed6806a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fed880c7420 tx=0x7fed70008040 comp rx=0 tx=0).stop 2023-11-28T13:04:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 -- 172.21.15.6:0/1810002532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed88070a60 msgr2=0x7fed880af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed88070a60 0x7fed880af1b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fed78000c00 tx=0x7fed78004750 comp rx=0 tx=0).stop 2023-11-28T13:04:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 -- 172.21.15.6:0/1810002532 shutdown_connections 2023-11-28T13:04:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fed680682f0 0x7fed6806a7a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fed880ccfe0 0x7fed88156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:10.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fed88070a60 0x7fed880af1b0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 --2- 172.21.15.6:0/1810002532 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fed880b6510 0x7fed8806cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 -- 172.21.15.6:0/1810002532 >> 172.21.15.6:0/1810002532 conn(0x7fed880d0bd0 msgr2=0x7fed8806e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 -- 172.21.15.6:0/1810002532 shutdown_connections 2023-11-28T13:04:10.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:10.835+0000 7fed8e240700 1 -- 172.21.15.6:0/1810002532 wait complete. 2023-11-28T13:04:10.848 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:15.849 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:15.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.986+0000 7f9c9ad0e700 1 Processor -- start 2023-11-28T13:04:15.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.986+0000 7f9c9ad0e700 1 -- start start 2023-11-28T13:04:15.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.986+0000 7f9c9ad0e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 0x7f9c940cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:15.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.986+0000 7f9c9ad0e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9c940cdb50 0x7f9c940b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:15.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.986+0000 7f9c9ad0e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9c940aefe0 0x7f9c940cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:15.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.986+0000 7f9c9ad0e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c940b5880 con 0x7f9c940aefe0 2023-11-28T13:04:15.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.986+0000 7f9c9ad0e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c940d01e0 con 0x7f9c940cdb50 2023-11-28T13:04:15.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.986+0000 7f9c9ad0e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c940d04c0 con 0x7f9c940b4a70 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c992ab700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9c940aefe0 0x7f9c940cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c992ab700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9c940aefe0 0x7f9c940cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34672/0 (socket says 172.21.15.6:34672) 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c992ab700 1 -- 172.21.15.6:0/3669736763 learned_addr learned my addr 172.21.15.6:0/3669736763 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c98aaa700 1 --2- 172.21.15.6:0/3669736763 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 0x7f9c940cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c93fff700 1 --2- 172.21.15.6:0/3669736763 >> v2:172.21.15.61:3300/0 conn(0x7f9c940cdb50 0x7f9c940b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c992ab700 1 -- 172.21.15.6:0/3669736763 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 msgr2=0x7f9c940cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c992ab700 1 --2- 172.21.15.6:0/3669736763 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 0x7f9c940cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c992ab700 1 -- 172.21.15.6:0/3669736763 >> v2:172.21.15.61:3300/0 conn(0x7f9c940cdb50 msgr2=0x7f9c940b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c992ab700 1 --2- 172.21.15.6:0/3669736763 >> v2:172.21.15.61:3300/0 conn(0x7f9c940cdb50 0x7f9c940b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c992ab700 1 -- 172.21.15.6:0/3669736763 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c940d07a0 con 0x7f9c940aefe0 2023-11-28T13:04:15.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c93fff700 1 --2- 172.21.15.6:0/3669736763 >> v2:172.21.15.61:3300/0 conn(0x7f9c940cdb50 0x7f9c940b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:15.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c98aaa700 1 --2- 172.21.15.6:0/3669736763 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 0x7f9c940cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:15.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c992ab700 1 --2- 172.21.15.6:0/3669736763 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 0x7f9c940cb3c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9c94070920 tx=0x7f9c8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59905b8ec2999df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:15.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c937fe700 1 -- 172.21.15.6:0/3669736763 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00e070 con 0x7f9c940aefe0 2023-11-28T13:04:15.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c937fe700 1 -- 172.21.15.6:0/3669736763 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c00b9c0 con 0x7f9c940aefe0 2023-11-28T13:04:15.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.987+0000 7f9c937fe700 1 -- 172.21.15.6:0/3669736763 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00bd70 con 0x7f9c940aefe0 2023-11-28T13:04:15.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.988+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3669736763 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 msgr2=0x7f9c940cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:15.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.988+0000 7f9c9ad0e700 1 --2- 172.21.15.6:0/3669736763 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 0x7f9c940cb3c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9c94070920 tx=0x7f9c8c00b330 comp rx=0 tx=0).stop 2023-11-28T13:04:15.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.988+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3669736763 shutdown_connections 2023-11-28T13:04:15.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.988+0000 7f9c9ad0e700 1 --2- 172.21.15.6:0/3669736763 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 0x7f9c940cb3c0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.988+0000 7f9c9ad0e700 1 --2- 172.21.15.6:0/3669736763 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 0x7f9c940cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.988+0000 7f9c9ad0e700 1 --2- 172.21.15.6:0/3669736763 >> v2:172.21.15.61:3300/0 conn(0x7f9c940cdb50 0x7f9c940b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.988+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3669736763 >> 172.21.15.6:0/3669736763 conn(0x7f9c940b3610 msgr2=0x7f9c940cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.988+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3669736763 shutdown_connections 2023-11-28T13:04:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.988+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3669736763 wait complete. 2023-11-28T13:04:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.989+0000 7f9c9ad0e700 1 Processor -- start 2023-11-28T13:04:15.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.989+0000 7f9c9ad0e700 1 -- start start 2023-11-28T13:04:15.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c9ad0e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 0x7f9c941df050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:15.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c9ad0e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 0x7f9c941d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c9ad0e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c940cdb50 0x7f9c940c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c9ad0e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c940c74a0 con 0x7f9c940aefe0 2023-11-28T13:04:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c9ad0e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c940c83f0 con 0x7f9c940cdb50 2023-11-28T13:04:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c9ad0e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c940c9e30 con 0x7f9c940b4a70 2023-11-28T13:04:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c98aaa700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 0x7f9c941df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c98aaa700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 0x7f9c941df050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34674/0 (socket says 172.21.15.6:34674) 2023-11-28T13:04:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c98aaa700 1 -- 172.21.15.6:0/3578486537 learned_addr learned my addr 172.21.15.6:0/3578486537 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c93fff700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 0x7f9c941d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c992ab700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c940cdb50 0x7f9c940c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:15.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c98aaa700 1 -- 172.21.15.6:0/3578486537 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 msgr2=0x7f9c941d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c98aaa700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 0x7f9c941d0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c98aaa700 1 -- 172.21.15.6:0/3578486537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c940cdb50 msgr2=0x7f9c940c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c98aaa700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c940cdb50 0x7f9c940c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c98aaa700 1 -- 172.21.15.6:0/3578486537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c009b30 con 0x7f9c940aefe0 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.990+0000 7f9c992ab700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c940cdb50 0x7f9c940c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.991+0000 7f9c98aaa700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 0x7f9c941df050 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800c7f0 tx=0x7f9c8800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.991+0000 7f9c917fa700 1 -- 172.21.15.6:0/3578486537 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88011800 con 0x7f9c940aefe0 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.991+0000 7f9c917fa700 1 -- 172.21.15.6:0/3578486537 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c88011960 con 0x7f9c940aefe0 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.991+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3578486537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c940aec60 con 0x7f9c940aefe0 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.991+0000 7f9c917fa700 1 -- 172.21.15.6:0/3578486537 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88011d10 con 0x7f9c940aefe0 2023-11-28T13:04:15.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.991+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3578486537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c940aead0 con 0x7f9c940aefe0 2023-11-28T13:04:15.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.992+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3578486537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c940c8660 con 0x7f9c940aefe0 2023-11-28T13:04:15.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.992+0000 7f9c917fa700 1 -- 172.21.15.6:0/3578486537 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c88021070 con 0x7f9c940aefe0 2023-11-28T13:04:15.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.993+0000 7f9c917fa700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9c780682f0 0x7f9c7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:15.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.993+0000 7f9c93fff700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9c780682f0 0x7f9c7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:15.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.993+0000 7f9c917fa700 1 -- 172.21.15.6:0/3578486537 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9c88085cb0 con 0x7f9c940aefe0 2023-11-28T13:04:15.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.993+0000 7f9c93fff700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9c780682f0 0x7f9c7806a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9c940c3670 tx=0x7f9c80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:15.998 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:15.997+0000 7f9c917fa700 1 -- 172.21.15.6:0/3578486537 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9c88052220 con 0x7f9c940aefe0 2023-11-28T13:04:16.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.238+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3578486537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9c940675a0 con 0x7f9c940aefe0 2023-11-28T13:04:16.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.238+0000 7f9c917fa700 1 -- 172.21.15.6:0/3578486537 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9c88055840 con 0x7f9c940aefe0 2023-11-28T13:04:16.240 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:16.240 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:16.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3578486537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9c780682f0 msgr2=0x7f9c7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:16.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9c780682f0 0x7f9c7806a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9c940c3670 tx=0x7f9c80008040 comp rx=0 tx=0).stop 2023-11-28T13:04:16.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3578486537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 msgr2=0x7f9c941df050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:16.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 0x7f9c941df050 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800c7f0 tx=0x7f9c8800cb00 comp rx=0 tx=0).stop 2023-11-28T13:04:16.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3578486537 shutdown_connections 2023-11-28T13:04:16.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9c940cdb50 0x7f9c940c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9c940aefe0 0x7f9c941df050 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9c780682f0 0x7f9c7806a7a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 --2- 172.21.15.6:0/3578486537 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9c940b4a70 0x7f9c941d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:16.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3578486537 >> 172.21.15.6:0/3578486537 conn(0x7f9c940b3610 msgr2=0x7f9c940c58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:16.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3578486537 shutdown_connections 2023-11-28T13:04:16.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:16.240+0000 7f9c9ad0e700 1 -- 172.21.15.6:0/3578486537 wait complete. 2023-11-28T13:04:16.253 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:21.254 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:21.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.391+0000 7f90132e6700 1 Processor -- start 2023-11-28T13:04:21.392 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.391+0000 7f90132e6700 1 -- start start 2023-11-28T13:04:21.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f90132e6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f900c0b6510 0x7f900c0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f90132e6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f900c0ccfe0 0x7f900c070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f90132e6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c070a60 0x7f900c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f90132e6700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c070f00 con 0x7f900c0b6510 2023-11-28T13:04:21.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f90132e6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c0ce9b0 con 0x7f900c0ccfe0 2023-11-28T13:04:21.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f90132e6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0b0d30 con 0x7f900c070a60 2023-11-28T13:04:21.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f9011082700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f900c0b6510 0x7f900c0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f9011082700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f900c0b6510 0x7f900c0cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34682/0 (socket says 172.21.15.6:34682) 2023-11-28T13:04:21.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f9011082700 1 -- 172.21.15.6:0/1382713806 learned_addr learned my addr 172.21.15.6:0/1382713806 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f9011883700 1 --2- 172.21.15.6:0/1382713806 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c070a60 0x7f900c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f9010881700 1 --2- 172.21.15.6:0/1382713806 >> v2:172.21.15.61:3300/0 conn(0x7f900c0ccfe0 0x7f900c070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f9011082700 1 -- 172.21.15.6:0/1382713806 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c070a60 msgr2=0x7f900c15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f9011082700 1 --2- 172.21.15.6:0/1382713806 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c070a60 0x7f900c15df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f9011082700 1 -- 172.21.15.6:0/1382713806 >> v2:172.21.15.61:3300/0 conn(0x7f900c0ccfe0 msgr2=0x7f900c070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.392+0000 7f9011082700 1 --2- 172.21.15.6:0/1382713806 >> v2:172.21.15.61:3300/0 conn(0x7f900c0ccfe0 0x7f900c070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f9011082700 1 -- 172.21.15.6:0/1382713806 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0b0fd0 con 0x7f900c0b6510 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f9011883700 1 --2- 172.21.15.6:0/1382713806 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c070a60 0x7f900c15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f9011082700 1 --2- 172.21.15.6:0/1382713806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c0b6510 0x7f900c0cb220 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f900c0c7be0 tx=0x7f8ffc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2172f79a7ff047c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f9003fff700 1 -- 172.21.15.6:0/1382713806 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc011830 con 0x7f900c0b6510 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f9003fff700 1 -- 172.21.15.6:0/1382713806 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc011990 con 0x7f900c0b6510 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f9003fff700 1 -- 172.21.15.6:0/1382713806 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc011d20 con 0x7f900c0b6510 2023-11-28T13:04:21.396 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f90132e6700 1 -- 172.21.15.6:0/1382713806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c0b6510 msgr2=0x7f900c0cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:21.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f90132e6700 1 --2- 172.21.15.6:0/1382713806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c0b6510 0x7f900c0cb220 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f900c0c7be0 tx=0x7f8ffc009bd0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f90132e6700 1 -- 172.21.15.6:0/1382713806 shutdown_connections 2023-11-28T13:04:21.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f90132e6700 1 --2- 172.21.15.6:0/1382713806 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c070a60 0x7f900c15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f90132e6700 1 --2- 172.21.15.6:0/1382713806 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c0b6510 0x7f900c0cb220 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f90132e6700 1 --2- 172.21.15.6:0/1382713806 >> v2:172.21.15.61:3300/0 conn(0x7f900c0ccfe0 0x7f900c070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.393+0000 7f90132e6700 1 -- 172.21.15.6:0/1382713806 >> 172.21.15.6:0/1382713806 conn(0x7f900c0d0bd0 msgr2=0x7f900c0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:21.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.394+0000 7f90132e6700 1 -- 172.21.15.6:0/1382713806 shutdown_connections 2023-11-28T13:04:21.397 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.394+0000 7f90132e6700 1 -- 172.21.15.6:0/1382713806 wait complete. 2023-11-28T13:04:21.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.395+0000 7f90132e6700 1 Processor -- start 2023-11-28T13:04:21.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.395+0000 7f90132e6700 1 -- start start 2023-11-28T13:04:21.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f90132e6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c070a60 0x7f900c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f90132e6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f900c0b6510 0x7f900c0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f90132e6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c0ccfe0 0x7f900c0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f90132e6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0cac40 con 0x7f900c070a60 2023-11-28T13:04:21.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f90132e6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c5aa0 con 0x7f900c0b6510 2023-11-28T13:04:21.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f90132e6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c51b0 con 0x7f900c0ccfe0 2023-11-28T13:04:21.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9011082700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c070a60 0x7f900c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9011082700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c070a60 0x7f900c0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34686/0 (socket says 172.21.15.6:34686) 2023-11-28T13:04:21.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9011082700 1 -- 172.21.15.6:0/708878882 learned_addr learned my addr 172.21.15.6:0/708878882 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:21.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9010881700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f900c0b6510 0x7f900c0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9011883700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c0ccfe0 0x7f900c0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9011082700 1 -- 172.21.15.6:0/708878882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c0ccfe0 msgr2=0x7f900c0c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:21.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9011082700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c0ccfe0 0x7f900c0c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9011082700 1 -- 172.21.15.6:0/708878882 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f900c0b6510 msgr2=0x7f900c0c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:21.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9011082700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f900c0b6510 0x7f900c0c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9011082700 1 -- 172.21.15.6:0/708878882 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc009550 con 0x7f900c070a60 2023-11-28T13:04:21.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9010881700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f900c0b6510 0x7f900c0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:04:21.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9011082700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c070a60 0x7f900c0ca700 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000c00 tx=0x7f8ffc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:21.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9001ffb700 1 -- 172.21.15.6:0/708878882 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc004380 con 0x7f900c070a60 2023-11-28T13:04:21.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9001ffb700 1 -- 172.21.15.6:0/708878882 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc01f070 con 0x7f900c070a60 2023-11-28T13:04:21.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f90132e6700 1 -- 172.21.15.6:0/708878882 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c0cb760 con 0x7f900c070a60 2023-11-28T13:04:21.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.396+0000 7f9001ffb700 1 -- 172.21.15.6:0/708878882 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc0036a0 con 0x7f900c070a60 2023-11-28T13:04:21.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.397+0000 7f90132e6700 1 -- 172.21.15.6:0/708878882 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c0b12b0 con 0x7f900c070a60 2023-11-28T13:04:21.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.398+0000 7f90132e6700 1 -- 172.21.15.6:0/708878882 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c15e4b0 con 0x7f900c070a60 2023-11-28T13:04:21.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.398+0000 7f9001ffb700 1 -- 172.21.15.6:0/708878882 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ffc02a020 con 0x7f900c070a60 2023-11-28T13:04:21.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.398+0000 7f9001ffb700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8ff00682f0 0x7f8ff006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:21.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.399+0000 7f9001ffb700 1 -- 172.21.15.6:0/708878882 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f8ffc087400 con 0x7f900c070a60 2023-11-28T13:04:21.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.399+0000 7f9010881700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8ff00682f0 0x7f8ff006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:21.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.399+0000 7f9010881700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8ff00682f0 0x7f8ff006a7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f900c15a080 tx=0x7f9008008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:21.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.403+0000 7f9001ffb700 1 -- 172.21.15.6:0/708878882 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ffc053ae0 con 0x7f900c070a60 2023-11-28T13:04:21.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.642+0000 7f90132e6700 1 -- 172.21.15.6:0/708878882 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f900c0675a0 con 0x7f900c070a60 2023-11-28T13:04:21.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.643+0000 7f9001ffb700 1 -- 172.21.15.6:0/708878882 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8ffc057100 con 0x7f900c070a60 2023-11-28T13:04:21.645 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:21.645 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:21.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 -- 172.21.15.6:0/708878882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8ff00682f0 msgr2=0x7f8ff006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:21.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8ff00682f0 0x7f8ff006a7a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f900c15a080 tx=0x7f9008008040 comp rx=0 tx=0).stop 2023-11-28T13:04:21.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 -- 172.21.15.6:0/708878882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c070a60 msgr2=0x7f900c0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:21.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c070a60 0x7f900c0ca700 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000c00 tx=0x7f8ffc004750 comp rx=0 tx=0).stop 2023-11-28T13:04:21.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 -- 172.21.15.6:0/708878882 shutdown_connections 2023-11-28T13:04:21.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8ff00682f0 0x7f8ff006a7a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f900c0ccfe0 0x7f900c0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f900c070a60 0x7f900c0ca700 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 --2- 172.21.15.6:0/708878882 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f900c0b6510 0x7f900c0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:21.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 -- 172.21.15.6:0/708878882 >> 172.21.15.6:0/708878882 conn(0x7f900c0d0bd0 msgr2=0x7f900c15c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:21.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 -- 172.21.15.6:0/708878882 shutdown_connections 2023-11-28T13:04:21.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:21.645+0000 7f90132e6700 1 -- 172.21.15.6:0/708878882 wait complete. 2023-11-28T13:04:21.658 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:26.659 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:26.810 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.809+0000 7fac9c312700 1 Processor -- start 2023-11-28T13:04:26.811 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.809+0000 7fac9c312700 1 -- start start 2023-11-28T13:04:26.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac9c312700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac9415c6a0 0x7fac940b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:26.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac9c312700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac940d20e0 con 0x7fac9406abf0 2023-11-28T13:04:26.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac9c312700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac940b4fb0 con 0x7fac940cd5a0 2023-11-28T13:04:26.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac9c312700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac940b5570 con 0x7fac9415c6a0 2023-11-28T13:04:26.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac998ad700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fac9406abf0 0x7fac940d19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50248/0 (socket says 172.21.15.6:50248) 2023-11-28T13:04:26.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac998ad700 1 -- 172.21.15.6:0/2658810713 learned_addr learned my addr 172.21.15.6:0/2658810713 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:04:26.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac9a0ae700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac9415c6a0 0x7fac940b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573699174 0 0) 0x7fac940d20e0 con 0x7fac9406abf0 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac800035b0 con 0x7fac9406abf0 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 354488657 0 0) 0x7fac800035b0 con 0x7fac9406abf0 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac9415c6a0 msgr2=0x7fac940b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac9415c6a0 0x7fac940b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 >> v1:172.21.15.61:6789/0 conn(0x7fac940cd5a0 legacy=0x7fac940b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac940b5290 con 0x7fac9406abf0 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fac90001670 con 0x7fac9406abf0 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 >> v1:172.21.15.6:6789/0 conn(0x7fac9406abf0 legacy=0x7fac940d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac80003a30 0x7fac80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac9415c6a0 0x7fac80004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:26.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac80004d90 0x7fac80005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8000ba10 con 0x7fac80003a30 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8000bb50 con 0x7fac80004d90 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.811+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8000bce0 con 0x7fac9415c6a0 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac9a0ae700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac80003a30 0x7fac80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac998ad700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac9415c6a0 0x7fac80004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac9a8af700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac80004d90 0x7fac80005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac9a0ae700 1 -- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac9415c6a0 msgr2=0x7fac80004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac9a0ae700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac9415c6a0 0x7fac80004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac9a0ae700 1 -- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac80004d90 msgr2=0x7fac80005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac9a0ae700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac80004d90 0x7fac80005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac9a0ae700 1 -- 172.21.15.6:0/2658810713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac800035b0 con 0x7fac80003a30 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac998ad700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac9415c6a0 0x7fac80004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:04:26.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac9a0ae700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac80003a30 0x7fac80003e00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fac84009b60 tx=0x7fac80005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e829fe39993ea966 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:26.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400e6c0 con 0x7fac80003a30 2023-11-28T13:04:26.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8400e820 con 0x7fac80003a30 2023-11-28T13:04:26.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.812+0000 7fac990ac700 1 -- 172.21.15.6:0/2658810713 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400eb90 con 0x7fac80003a30 2023-11-28T13:04:26.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.813+0000 7fac9c312700 1 -- 172.21.15.6:0/2658810713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac80003a30 msgr2=0x7fac80003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:26.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.813+0000 7fac9c312700 1 --2- 172.21.15.6:0/2658810713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac80003a30 0x7fac80003e00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fac84009b60 tx=0x7fac80005880 comp rx=0 tx=0).stop 2023-11-28T13:04:26.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.813+0000 7fac9a8af700 1 -- 172.21.15.6:0/2658810713 reap_dead start 2023-11-28T13:04:26.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.813+0000 7fac9c312700 1 -- 172.21.15.6:0/2658810713 shutdown_connections 2023-11-28T13:04:26.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.813+0000 7fac9c312700 1 -- 172.21.15.6:0/2658810713 >> 172.21.15.6:0/2658810713 conn(0x7fac940d2f50 msgr2=0x7fac940d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:26.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.813+0000 7fac9c312700 1 -- 172.21.15.6:0/2658810713 shutdown_connections 2023-11-28T13:04:26.818 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.813+0000 7fac9c312700 1 -- 172.21.15.6:0/2658810713 wait complete. 2023-11-28T13:04:26.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.814+0000 7fac9c312700 1 Processor -- start 2023-11-28T13:04:26.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.814+0000 7fac9c312700 1 -- start start 2023-11-28T13:04:26.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9c312700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac80003a30 0x7fac941ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:26.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9c312700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac94156740 0x7fac94156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:26.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9c312700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac94154520 0x7fac941549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:26.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9c312700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac94154ee0 con 0x7fac94154520 2023-11-28T13:04:26.821 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9c312700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac94157850 con 0x7fac94156740 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9c312700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac94157290 con 0x7fac80003a30 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9a8af700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac94154520 0x7fac941549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9a8af700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac94154520 0x7fac941549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34710/0 (socket says 172.21.15.6:34710) 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9a8af700 1 -- 172.21.15.6:0/4119601829 learned_addr learned my addr 172.21.15.6:0/4119601829 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac998ad700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac94156740 0x7fac94156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9a0ae700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac80003a30 0x7fac941ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9a8af700 1 -- 172.21.15.6:0/4119601829 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac80003a30 msgr2=0x7fac941ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9a8af700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac80003a30 0x7fac941ef750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9a8af700 1 -- 172.21.15.6:0/4119601829 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac94156740 msgr2=0x7fac94156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9a8af700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac94156740 0x7fac94156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:26.822 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9a8af700 1 -- 172.21.15.6:0/4119601829 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac940d14b0 con 0x7fac94154520 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.815+0000 7fac9a8af700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac94154520 0x7fac941549a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fac940b5a20 tx=0x7fac8c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.816+0000 7fac8affd700 1 -- 172.21.15.6:0/4119601829 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c003c10 con 0x7fac94154520 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.816+0000 7fac8affd700 1 -- 172.21.15.6:0/4119601829 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c003d70 con 0x7fac94154520 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.816+0000 7fac9c312700 1 -- 172.21.15.6:0/4119601829 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac940d11d0 con 0x7fac94154520 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.816+0000 7fac8affd700 1 -- 172.21.15.6:0/4119601829 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c005310 con 0x7fac94154520 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.816+0000 7fac9c312700 1 -- 172.21.15.6:0/4119601829 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9415ca70 con 0x7fac94154520 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.817+0000 7fac9c312700 1 -- 172.21.15.6:0/4119601829 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac94068570 con 0x7fac94154520 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.817+0000 7fac8affd700 1 -- 172.21.15.6:0/4119601829 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fac8c005470 con 0x7fac94154520 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.817+0000 7fac8affd700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac780682f0 0x7fac7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.818+0000 7fac8affd700 1 -- 172.21.15.6:0/4119601829 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fac8c0860a0 con 0x7fac94154520 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.818+0000 7fac9a0ae700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac780682f0 0x7fac7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:26.823 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.818+0000 7fac9a0ae700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac780682f0 0x7fac7806a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fac840090b0 tx=0x7fac84008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:26.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:26.822+0000 7fac8affd700 1 -- 172.21.15.6:0/4119601829 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fac8c052780 con 0x7fac94154520 2023-11-28T13:04:27.031 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:04:27.034 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:04:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.060+0000 7fac9c312700 1 -- 172.21.15.6:0/4119601829 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fac9406b7d0 con 0x7fac94154520 2023-11-28T13:04:27.062 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.061+0000 7fac8affd700 1 -- 172.21.15.6:0/4119601829 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fac8c055da0 con 0x7fac94154520 2023-11-28T13:04:27.062 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:27.062 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 -- 172.21.15.6:0/4119601829 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac780682f0 msgr2=0x7fac7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac780682f0 0x7fac7806a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fac840090b0 tx=0x7fac84008030 comp rx=0 tx=0).stop 2023-11-28T13:04:27.064 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 -- 172.21.15.6:0/4119601829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac94154520 msgr2=0x7fac941549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac94154520 0x7fac941549a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fac940b5a20 tx=0x7fac8c00ef20 comp rx=0 tx=0).stop 2023-11-28T13:04:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 -- 172.21.15.6:0/4119601829 shutdown_connections 2023-11-28T13:04:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fac94154520 0x7fac941549a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fac780682f0 0x7fac7806a7a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fac80003a30 0x7fac941ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 --2- 172.21.15.6:0/4119601829 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fac94156740 0x7fac94156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 -- 172.21.15.6:0/4119601829 >> 172.21.15.6:0/4119601829 conn(0x7fac940d2f50 msgr2=0x7fac9415a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 -- 172.21.15.6:0/4119601829 shutdown_connections 2023-11-28T13:04:27.065 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:27.063+0000 7fac9c312700 1 -- 172.21.15.6:0/4119601829 wait complete. 2023-11-28T13:04:27.076 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:32.077 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:32.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.215+0000 7f27fdcd9700 1 Processor -- start 2023-11-28T13:04:32.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.215+0000 7f27fdcd9700 1 -- start start 2023-11-28T13:04:32.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.216+0000 7f27fdcd9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f27f80aefe0 0x7f27f80b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.216+0000 7f27fdcd9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80cc100 0x7f27f80cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.216+0000 7f27fdcd9700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f80b5880 con 0x7f27f80b4a70 2023-11-28T13:04:32.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.216+0000 7f27fdcd9700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f80d01e0 con 0x7f27f80aefe0 2023-11-28T13:04:32.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.216+0000 7f27fdcd9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80d04c0 con 0x7f27f80cc100 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.216+0000 7f27f6ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f27f80aefe0 0x7f27f80b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.216+0000 7f27f77fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f27f80b4a70 0x7f27f80cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50266/0 (socket says 172.21.15.6:50266) 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.216+0000 7f27f77fe700 1 -- 172.21.15.6:0/838897892 learned_addr learned my addr 172.21.15.6:0/838897892 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27f6ffd700 1 -- 172.21.15.6:0/838897892 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80cc100 msgr2=0x7f27f80cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.216+0000 7f27f7fff700 1 --2- 172.21.15.6:0/838897892 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80cc100 0x7f27f80cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27f6ffd700 1 --2- 172.21.15.6:0/838897892 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80cc100 0x7f27f80cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27f6ffd700 1 -- 172.21.15.6:0/838897892 >> v1:172.21.15.6:6789/0 conn(0x7f27f80b4a70 legacy=0x7f27f80cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27f67fc700 1 -- 172.21.15.6:0/838897892 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386345575 0 0) 0x7f27f80b5880 con 0x7f27f80b4a70 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27f6ffd700 1 -- 172.21.15.6:0/838897892 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80d07a0 con 0x7f27f80aefe0 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27f7fff700 1 --2- 172.21.15.6:0/838897892 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80cc100 0x7f27f80cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:32.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27f6ffd700 1 --2- 172.21.15.6:0/838897892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80aefe0 0x7f27f80b4e40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009840 tx=0x7f27e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffa2af1024bc9d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27f67fc700 1 -- 172.21.15.6:0/838897892 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8011830 con 0x7f27f80aefe0 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27f67fc700 1 -- 172.21.15.6:0/838897892 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e8011990 con 0x7f27f80aefe0 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27f67fc700 1 -- 172.21.15.6:0/838897892 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8011d20 con 0x7f27f80aefe0 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27fdcd9700 1 -- 172.21.15.6:0/838897892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80aefe0 msgr2=0x7f27f80b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.217+0000 7f27fdcd9700 1 --2- 172.21.15.6:0/838897892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80aefe0 0x7f27f80b4e40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009840 tx=0x7f27e8009b50 comp rx=0 tx=0).stop 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.218+0000 7f27fdcd9700 1 -- 172.21.15.6:0/838897892 shutdown_connections 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.218+0000 7f27fdcd9700 1 --2- 172.21.15.6:0/838897892 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80cc100 0x7f27f80cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.218+0000 7f27fdcd9700 1 --2- 172.21.15.6:0/838897892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80aefe0 0x7f27f80b4e40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.218+0000 7f27fdcd9700 1 -- 172.21.15.6:0/838897892 >> 172.21.15.6:0/838897892 conn(0x7f27f80b3610 msgr2=0x7f27f80cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.218+0000 7f27fdcd9700 1 -- 172.21.15.6:0/838897892 shutdown_connections 2023-11-28T13:04:32.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.218+0000 7f27fdcd9700 1 -- 172.21.15.6:0/838897892 wait complete. 2023-11-28T13:04:32.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.219+0000 7f27fdcd9700 1 Processor -- start 2023-11-28T13:04:32.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.219+0000 7f27fdcd9700 1 -- start start 2023-11-28T13:04:32.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27fdcd9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80aefe0 0x7f27f81d9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27fdcd9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80b4a70 0x7f27f81da480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27fdcd9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27f80cc100 0x7f27f81d9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27fdcd9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f81dcef0 con 0x7f27f80b4a70 2023-11-28T13:04:32.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27fdcd9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f81dac40 con 0x7f27f80cc100 2023-11-28T13:04:32.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27fdcd9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f81da9c0 con 0x7f27f80aefe0 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27f6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80b4a70 0x7f27f81da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27f6ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80b4a70 0x7f27f81da480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34722/0 (socket says 172.21.15.6:34722) 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27f6ffd700 1 -- 172.21.15.6:0/896232904 learned_addr learned my addr 172.21.15.6:0/896232904 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27f77fe700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80aefe0 0x7f27f81d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27f7fff700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27f80cc100 0x7f27f81d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27f6ffd700 1 -- 172.21.15.6:0/896232904 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80aefe0 msgr2=0x7f27f81d9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27f6ffd700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80aefe0 0x7f27f81d9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27f6ffd700 1 -- 172.21.15.6:0/896232904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27f80cc100 msgr2=0x7f27f81d9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27f6ffd700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27f80cc100 0x7f27f81d9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.220+0000 7f27f6ffd700 1 -- 172.21.15.6:0/896232904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80094a0 con 0x7f27f80b4a70 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.221+0000 7f27f6ffd700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80b4a70 0x7f27f81da480 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f27e8000c00 tx=0x7f27e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:32.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.221+0000 7f27effff700 1 -- 172.21.15.6:0/896232904 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e80036a0 con 0x7f27f80b4a70 2023-11-28T13:04:32.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.221+0000 7f27effff700 1 -- 172.21.15.6:0/896232904 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e801e070 con 0x7f27f80b4a70 2023-11-28T13:04:32.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.221+0000 7f27fdcd9700 1 -- 172.21.15.6:0/896232904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f80aead0 con 0x7f27f80b4a70 2023-11-28T13:04:32.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.221+0000 7f27effff700 1 -- 172.21.15.6:0/896232904 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8003a40 con 0x7f27f80b4a70 2023-11-28T13:04:32.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.221+0000 7f27fdcd9700 1 -- 172.21.15.6:0/896232904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f80cdd20 con 0x7f27f80b4a70 2023-11-28T13:04:32.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.222+0000 7f27fdcd9700 1 -- 172.21.15.6:0/896232904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f80aec60 con 0x7f27f80b4a70 2023-11-28T13:04:32.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.222+0000 7f27effff700 1 -- 172.21.15.6:0/896232904 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f27e800f050 con 0x7f27f80b4a70 2023-11-28T13:04:32.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.223+0000 7f27effff700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27d80682f0 0x7f27d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:32.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.223+0000 7f27f77fe700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27d80682f0 0x7f27d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:32.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.223+0000 7f27effff700 1 -- 172.21.15.6:0/896232904 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f27e8086f40 con 0x7f27f80b4a70 2023-11-28T13:04:32.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.223+0000 7f27f77fe700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27d80682f0 0x7f27d806a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f27e0007fd0 tx=0x7f27e0007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:32.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.227+0000 7f27effff700 1 -- 172.21.15.6:0/896232904 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f27e8053530 con 0x7f27f80b4a70 2023-11-28T13:04:32.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.462+0000 7f27fdcd9700 1 -- 172.21.15.6:0/896232904 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f27f81df1a0 con 0x7f27f80b4a70 2023-11-28T13:04:32.463 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.463+0000 7f27effff700 1 -- 172.21.15.6:0/896232904 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f27e80150c0 con 0x7f27f80b4a70 2023-11-28T13:04:32.463 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:32.463 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:32.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 -- 172.21.15.6:0/896232904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27d80682f0 msgr2=0x7f27d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.465 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27d80682f0 0x7f27d806a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f27e0007fd0 tx=0x7f27e0007de0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 -- 172.21.15.6:0/896232904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80b4a70 msgr2=0x7f27f81da480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:32.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80b4a70 0x7f27f81da480 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f27e8000c00 tx=0x7f27e8004750 comp rx=0 tx=0).stop 2023-11-28T13:04:32.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 -- 172.21.15.6:0/896232904 shutdown_connections 2023-11-28T13:04:32.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27f80cc100 0x7f27f81d9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27d80682f0 0x7f27d806a7a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27f80aefe0 0x7f27f81d9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 --2- 172.21.15.6:0/896232904 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27f80b4a70 0x7f27f81da480 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:32.466 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 -- 172.21.15.6:0/896232904 >> 172.21.15.6:0/896232904 conn(0x7f27f80b3610 msgr2=0x7f27f80cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:32.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 -- 172.21.15.6:0/896232904 shutdown_connections 2023-11-28T13:04:32.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:32.465+0000 7f27fdcd9700 1 -- 172.21.15.6:0/896232904 wait complete. 2023-11-28T13:04:32.477 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:37.478 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:37.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.614+0000 7f40a6942700 1 Processor -- start 2023-11-28T13:04:37.616 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.615+0000 7f40a6942700 1 -- start start 2023-11-28T13:04:37.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.615+0000 7f40a6942700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a0070520 0x7f40a00708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:37.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.615+0000 7f40a6942700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f40a00cb220 0x7f40a00b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:37.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.615+0000 7f40a6942700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f40a00cb690 0x7f40a00b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:37.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.615+0000 7f40a6942700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a015dbd0 con 0x7f40a00cb690 2023-11-28T13:04:37.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.615+0000 7f40a6942700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a00b1510 con 0x7f40a00cb220 2023-11-28T13:04:37.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.615+0000 7f40a6942700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a0070e30 con 0x7f40a0070520 2023-11-28T13:04:37.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f40a4edf700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f40a00cb690 0x7f40a00b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:37.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f40a4edf700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f40a00cb690 0x7f40a00b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34732/0 (socket says 172.21.15.6:34732) 2023-11-28T13:04:37.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f40a4edf700 1 -- 172.21.15.6:0/2567619132 learned_addr learned my addr 172.21.15.6:0/2567619132 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:37.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f409ffff700 1 --2- 172.21.15.6:0/2567619132 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a0070520 0x7f40a00708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:37.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f409f7fe700 1 --2- 172.21.15.6:0/2567619132 >> v2:172.21.15.61:3300/0 conn(0x7f40a00cb220 0x7f40a00b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:37.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f40a4edf700 1 -- 172.21.15.6:0/2567619132 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a0070520 msgr2=0x7f40a00708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:37.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f40a4edf700 1 --2- 172.21.15.6:0/2567619132 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a0070520 0x7f40a00708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f40a4edf700 1 -- 172.21.15.6:0/2567619132 >> v2:172.21.15.61:3300/0 conn(0x7f40a00cb220 msgr2=0x7f40a00b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:37.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f40a4edf700 1 --2- 172.21.15.6:0/2567619132 >> v2:172.21.15.61:3300/0 conn(0x7f40a00cb220 0x7f40a00b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f40a4edf700 1 -- 172.21.15.6:0/2567619132 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a015d610 con 0x7f40a00cb690 2023-11-28T13:04:37.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f40a4edf700 1 --2- 172.21.15.6:0/2567619132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 0x7f40a00b0fd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f40a006ad20 tx=0x7f409800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8deb5522f299e23c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:37.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f409effd700 1 -- 172.21.15.6:0/2567619132 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409800e070 con 0x7f40a00cb690 2023-11-28T13:04:37.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f409effd700 1 -- 172.21.15.6:0/2567619132 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409800b910 con 0x7f40a00cb690 2023-11-28T13:04:37.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.616+0000 7f409effd700 1 -- 172.21.15.6:0/2567619132 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409800bcc0 con 0x7f40a00cb690 2023-11-28T13:04:37.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.617+0000 7f40a6942700 1 -- 172.21.15.6:0/2567619132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 msgr2=0x7f40a00b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:37.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.617+0000 7f40a6942700 1 --2- 172.21.15.6:0/2567619132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 0x7f40a00b0fd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f40a006ad20 tx=0x7f409800b330 comp rx=0 tx=0).stop 2023-11-28T13:04:37.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.617+0000 7f40a6942700 1 -- 172.21.15.6:0/2567619132 shutdown_connections 2023-11-28T13:04:37.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.617+0000 7f40a6942700 1 --2- 172.21.15.6:0/2567619132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 0x7f40a00b0fd0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.617+0000 7f40a6942700 1 --2- 172.21.15.6:0/2567619132 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a0070520 0x7f40a00708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.617+0000 7f40a6942700 1 --2- 172.21.15.6:0/2567619132 >> v2:172.21.15.61:3300/0 conn(0x7f40a00cb220 0x7f40a00b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.617+0000 7f40a6942700 1 -- 172.21.15.6:0/2567619132 >> 172.21.15.6:0/2567619132 conn(0x7f40a00d0bd0 msgr2=0x7f40a00d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:37.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.617+0000 7f40a6942700 1 -- 172.21.15.6:0/2567619132 shutdown_connections 2023-11-28T13:04:37.622 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.617+0000 7f40a6942700 1 -- 172.21.15.6:0/2567619132 wait complete. 2023-11-28T13:04:37.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.618+0000 7f40a6942700 1 Processor -- start 2023-11-28T13:04:37.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.618+0000 7f40a6942700 1 -- start start 2023-11-28T13:04:37.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.619+0000 7f40a6942700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40a0070520 0x7f40a00ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:37.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.619+0000 7f40a6942700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a00cb220 0x7f40a00b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:37.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.619+0000 7f40a6942700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 0x7f40a015a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:37.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.619+0000 7f40a6942700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00c8800 con 0x7f40a00cb690 2023-11-28T13:04:37.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.619+0000 7f40a6942700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00b52c0 con 0x7f40a0070520 2023-11-28T13:04:37.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.619+0000 7f40a6942700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00b4410 con 0x7f40a00cb220 2023-11-28T13:04:37.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.619+0000 7f40a4edf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 0x7f40a015a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:37.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.619+0000 7f40a4edf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 0x7f40a015a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34738/0 (socket says 172.21.15.6:34738) 2023-11-28T13:04:37.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.619+0000 7f40a4edf700 1 -- 172.21.15.6:0/948602986 learned_addr learned my addr 172.21.15.6:0/948602986 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:37.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f409ffff700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40a0070520 0x7f40a00ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:37.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f409f7fe700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a00cb220 0x7f40a00b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:37.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f40a4edf700 1 -- 172.21.15.6:0/948602986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a00cb220 msgr2=0x7f40a00b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:37.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f40a4edf700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a00cb220 0x7f40a00b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f40a4edf700 1 -- 172.21.15.6:0/948602986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40a0070520 msgr2=0x7f40a00ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:37.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f40a4edf700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40a0070520 0x7f40a00ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f40a4edf700 1 -- 172.21.15.6:0/948602986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4098009b30 con 0x7f40a00cb690 2023-11-28T13:04:37.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f409f7fe700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a00cb220 0x7f40a00b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:37.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f409ffff700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40a0070520 0x7f40a00ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:37.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f40a4edf700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 0x7f40a015a4c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4098012000 tx=0x7f4098004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:37.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f409cff9700 1 -- 172.21.15.6:0/948602986 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409800ce80 con 0x7f40a00cb690 2023-11-28T13:04:37.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f409cff9700 1 -- 172.21.15.6:0/948602986 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098004250 con 0x7f40a00cb690 2023-11-28T13:04:37.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f409cff9700 1 -- 172.21.15.6:0/948602986 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098004600 con 0x7f40a00cb690 2023-11-28T13:04:37.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f40a6942700 1 -- 172.21.15.6:0/948602986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a00b0290 con 0x7f40a00cb690 2023-11-28T13:04:37.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.620+0000 7f40a6942700 1 -- 172.21.15.6:0/948602986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a015d8f0 con 0x7f40a00cb690 2023-11-28T13:04:37.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.621+0000 7f40a6942700 1 -- 172.21.15.6:0/948602986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a00b0570 con 0x7f40a00cb690 2023-11-28T13:04:37.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.622+0000 7f409cff9700 1 -- 172.21.15.6:0/948602986 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f409802c070 con 0x7f40a00cb690 2023-11-28T13:04:37.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.622+0000 7f409cff9700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f408c0682f0 0x7f408c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:37.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.622+0000 7f409cff9700 1 -- 172.21.15.6:0/948602986 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f40980870f0 con 0x7f40a00cb690 2023-11-28T13:04:37.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.622+0000 7f409ffff700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f408c0682f0 0x7f408c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:37.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.623+0000 7f409ffff700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f408c0682f0 0x7f408c06a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4090009a40 tx=0x7f4090008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:37.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.627+0000 7f409cff9700 1 -- 172.21.15.6:0/948602986 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f40980537d0 con 0x7f40a00cb690 2023-11-28T13:04:37.864 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.863+0000 7f40a6942700 1 -- 172.21.15.6:0/948602986 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f40a0159b10 con 0x7f40a00cb690 2023-11-28T13:04:37.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.863+0000 7f409cff9700 1 -- 172.21.15.6:0/948602986 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f4098056df0 con 0x7f40a00cb690 2023-11-28T13:04:37.865 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:37.865 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:37.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 -- 172.21.15.6:0/948602986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f408c0682f0 msgr2=0x7f408c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f408c0682f0 0x7f408c06a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4090009a40 tx=0x7f4090008040 comp rx=0 tx=0).stop 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 -- 172.21.15.6:0/948602986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 msgr2=0x7f40a015a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 0x7f40a015a4c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4098012000 tx=0x7f4098004ad0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 -- 172.21.15.6:0/948602986 shutdown_connections 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40a00cb690 0x7f40a015a4c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40a0070520 0x7f40a00ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f408c0682f0 0x7f408c06a7a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 --2- 172.21.15.6:0/948602986 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40a00cb220 0x7f40a00b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 -- 172.21.15.6:0/948602986 >> 172.21.15.6:0/948602986 conn(0x7f40a00d0bd0 msgr2=0x7f40a006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 -- 172.21.15.6:0/948602986 shutdown_connections 2023-11-28T13:04:37.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:37.865+0000 7f40a6942700 1 -- 172.21.15.6:0/948602986 wait complete. 2023-11-28T13:04:37.879 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:42.880 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.017+0000 7f3133804700 1 Processor -- start 2023-11-28T13:04:43.018 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.017+0000 7f3133804700 1 -- start start 2023-11-28T13:04:43.021 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f3133804700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f312c0b6510 0x7f312c0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f3133804700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f312c0ccfe0 0x7f312c070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f3133804700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f312c070a60 0x7f312c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f3133804700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312c070f00 con 0x7f312c0b6510 2023-11-28T13:04:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f3133804700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312c0ce9b0 con 0x7f312c070a60 2023-11-28T13:04:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f3133804700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0b0d30 con 0x7f312c0ccfe0 2023-11-28T13:04:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f31315a0700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f312c0b6510 0x7f312c0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.022 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f31315a0700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f312c0b6510 0x7f312c0cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34742/0 (socket says 172.21.15.6:34742) 2023-11-28T13:04:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f31315a0700 1 -- 172.21.15.6:0/2760521850 learned_addr learned my addr 172.21.15.6:0/2760521850 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f3130d9f700 1 --2- 172.21.15.6:0/2760521850 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f312c0ccfe0 0x7f312c070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f3131da1700 1 --2- 172.21.15.6:0/2760521850 >> v2:172.21.15.61:3300/0 conn(0x7f312c070a60 0x7f312c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f31315a0700 1 -- 172.21.15.6:0/2760521850 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f312c0ccfe0 msgr2=0x7f312c070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f31315a0700 1 --2- 172.21.15.6:0/2760521850 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f312c0ccfe0 0x7f312c070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f31315a0700 1 -- 172.21.15.6:0/2760521850 >> v2:172.21.15.61:3300/0 conn(0x7f312c070a60 msgr2=0x7f312c15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f31315a0700 1 --2- 172.21.15.6:0/2760521850 >> v2:172.21.15.61:3300/0 conn(0x7f312c070a60 0x7f312c15df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f31315a0700 1 -- 172.21.15.6:0/2760521850 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c0b0fd0 con 0x7f312c0b6510 2023-11-28T13:04:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.018+0000 7f31315a0700 1 --2- 172.21.15.6:0/2760521850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0b6510 0x7f312c0cb220 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f312c0af8e0 tx=0x7f3128009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7424e32072e798d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:43.023 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3123fff700 1 -- 172.21.15.6:0/2760521850 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312800b870 con 0x7f312c0b6510 2023-11-28T13:04:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3123fff700 1 -- 172.21.15.6:0/2760521850 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312800b9d0 con 0x7f312c0b6510 2023-11-28T13:04:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3123fff700 1 -- 172.21.15.6:0/2760521850 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312800bd60 con 0x7f312c0b6510 2023-11-28T13:04:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3133804700 1 -- 172.21.15.6:0/2760521850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0b6510 msgr2=0x7f312c0cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3133804700 1 --2- 172.21.15.6:0/2760521850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0b6510 0x7f312c0cb220 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f312c0af8e0 tx=0x7f3128009bd0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3133804700 1 -- 172.21.15.6:0/2760521850 shutdown_connections 2023-11-28T13:04:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3133804700 1 --2- 172.21.15.6:0/2760521850 >> v2:172.21.15.61:3300/0 conn(0x7f312c070a60 0x7f312c15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3133804700 1 --2- 172.21.15.6:0/2760521850 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0b6510 0x7f312c0cb220 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3133804700 1 --2- 172.21.15.6:0/2760521850 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f312c0ccfe0 0x7f312c070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3133804700 1 -- 172.21.15.6:0/2760521850 >> 172.21.15.6:0/2760521850 conn(0x7f312c0d0bd0 msgr2=0x7f312c0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:43.024 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3133804700 1 -- 172.21.15.6:0/2760521850 shutdown_connections 2023-11-28T13:04:43.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.019+0000 7f3133804700 1 -- 172.21.15.6:0/2760521850 wait complete. 2023-11-28T13:04:43.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.020+0000 7f3133804700 1 Processor -- start 2023-11-28T13:04:43.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.021+0000 7f3133804700 1 -- start start 2023-11-28T13:04:43.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.021+0000 7f3133804700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f312c070a60 0x7f312c1f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.021+0000 7f3133804700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312c0b6510 0x7f312c15d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.021+0000 7f3133804700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0ccfe0 0x7f312c1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.021+0000 7f3133804700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c1f2cb0 con 0x7f312c0ccfe0 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.021+0000 7f3133804700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0c9ba0 con 0x7f312c0b6510 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.021+0000 7f3133804700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0c9e80 con 0x7f312c070a60 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.021+0000 7f3131da1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0ccfe0 0x7f312c1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.021+0000 7f3131da1700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0ccfe0 0x7f312c1591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34752/0 (socket says 172.21.15.6:34752) 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.021+0000 7f3131da1700 1 -- 172.21.15.6:0/729341203 learned_addr learned my addr 172.21.15.6:0/729341203 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f31315a0700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f312c070a60 0x7f312c1f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3130d9f700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312c0b6510 0x7f312c15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3131da1700 1 -- 172.21.15.6:0/729341203 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f312c070a60 msgr2=0x7f312c1f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3131da1700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f312c070a60 0x7f312c1f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3131da1700 1 -- 172.21.15.6:0/729341203 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312c0b6510 msgr2=0x7f312c15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3131da1700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312c0b6510 0x7f312c15d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3131da1700 1 -- 172.21.15.6:0/729341203 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3128009550 con 0x7f312c0ccfe0 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3131da1700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0ccfe0 0x7f312c1591a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3124007e60 tx=0x7f312400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3121ffb700 1 -- 172.21.15.6:0/729341203 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312400ed30 con 0x7f312c0ccfe0 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3121ffb700 1 -- 172.21.15.6:0/729341203 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124014420 con 0x7f312c0ccfe0 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3133804700 1 -- 172.21.15.6:0/729341203 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312c0cb760 con 0x7f312c0ccfe0 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3121ffb700 1 -- 172.21.15.6:0/729341203 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240147d0 con 0x7f312c0ccfe0 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.022+0000 7f3133804700 1 -- 172.21.15.6:0/729341203 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c0b12b0 con 0x7f312c0ccfe0 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.024+0000 7f3133804700 1 -- 172.21.15.6:0/729341203 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312c15e4b0 con 0x7f312c0ccfe0 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.024+0000 7f3121ffb700 1 -- 172.21.15.6:0/729341203 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3124012050 con 0x7f312c0ccfe0 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.024+0000 7f3121ffb700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31100682f0 0x7f311006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.024+0000 7f3121ffb700 1 -- 172.21.15.6:0/729341203 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f31240863b0 con 0x7f312c0ccfe0 2023-11-28T13:04:43.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.024+0000 7f31315a0700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31100682f0 0x7f311006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:43.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.025+0000 7f31315a0700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31100682f0 0x7f311006a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3128000c00 tx=0x7f3128008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:43.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.029+0000 7f3121ffb700 1 -- 172.21.15.6:0/729341203 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3124052a10 con 0x7f312c0ccfe0 2023-11-28T13:04:43.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.267+0000 7f3133804700 1 -- 172.21.15.6:0/729341203 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f312c0b1e20 con 0x7f312c0ccfe0 2023-11-28T13:04:43.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.267+0000 7f3121ffb700 1 -- 172.21.15.6:0/729341203 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f3124056030 con 0x7f312c0ccfe0 2023-11-28T13:04:43.269 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:43.269 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 -- 172.21.15.6:0/729341203 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31100682f0 msgr2=0x7f311006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31100682f0 0x7f311006a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3128000c00 tx=0x7f3128008030 comp rx=0 tx=0).stop 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 -- 172.21.15.6:0/729341203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0ccfe0 msgr2=0x7f312c1591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0ccfe0 0x7f312c1591a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3124007e60 tx=0x7f312400e3c0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 -- 172.21.15.6:0/729341203 shutdown_connections 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f312c0ccfe0 0x7f312c1591a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f31100682f0 0x7f311006a7a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f312c070a60 0x7f312c1f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 --2- 172.21.15.6:0/729341203 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f312c0b6510 0x7f312c15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 -- 172.21.15.6:0/729341203 >> 172.21.15.6:0/729341203 conn(0x7f312c0d0bd0 msgr2=0x7f312c15a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 -- 172.21.15.6:0/729341203 shutdown_connections 2023-11-28T13:04:43.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:43.269+0000 7f3133804700 1 -- 172.21.15.6:0/729341203 wait complete. 2023-11-28T13:04:43.282 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:48.284 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:48.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.420+0000 7f6c8d2cc700 1 Processor -- start 2023-11-28T13:04:48.422 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.420+0000 7f6c8d2cc700 1 -- start start 2023-11-28T13:04:48.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.421+0000 7f6c8d2cc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f6c880b6510 0x7f6c880cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.421+0000 7f6c8d2cc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c880ccfe0 0x7f6c88070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.421+0000 7f6c8d2cc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c88070a60 0x7f6c8815df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.421+0000 7f6c8d2cc700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c88070f00 con 0x7f6c88070a60 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.421+0000 7f6c8d2cc700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c880ce9b0 con 0x7f6c880b6510 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.421+0000 7f6c8d2cc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c880b0d30 con 0x7f6c880ccfe0 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.421+0000 7f6c877fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c88070a60 0x7f6c8815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c877fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c88070a60 0x7f6c8815df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34758/0 (socket says 172.21.15.6:34758) 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c877fe700 1 -- 172.21.15.6:0/745901208 learned_addr learned my addr 172.21.15.6:0/745901208 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c86ffd700 1 --2- 172.21.15.6:0/745901208 >> v2:172.21.15.61:3300/0 conn(0x7f6c880b6510 0x7f6c880cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c867fc700 1 --2- 172.21.15.6:0/745901208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c880ccfe0 0x7f6c88070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c877fe700 1 -- 172.21.15.6:0/745901208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c880ccfe0 msgr2=0x7f6c88070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c877fe700 1 --2- 172.21.15.6:0/745901208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c880ccfe0 0x7f6c88070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c877fe700 1 -- 172.21.15.6:0/745901208 >> v2:172.21.15.61:3300/0 conn(0x7f6c880b6510 msgr2=0x7f6c880cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c877fe700 1 --2- 172.21.15.6:0/745901208 >> v2:172.21.15.61:3300/0 conn(0x7f6c880b6510 0x7f6c880cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c877fe700 1 -- 172.21.15.6:0/745901208 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c880b0fd0 con 0x7f6c88070a60 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c877fe700 1 --2- 172.21.15.6:0/745901208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c88070a60 0x7f6c8815df70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6c880af930 tx=0x7f6c8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a678da03f6985a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c85ffb700 1 -- 172.21.15.6:0/745901208 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000e070 con 0x7f6c88070a60 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c85ffb700 1 -- 172.21.15.6:0/745901208 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8000b910 con 0x7f6c88070a60 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.422+0000 7f6c85ffb700 1 -- 172.21.15.6:0/745901208 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000bcc0 con 0x7f6c88070a60 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.423+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/745901208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c88070a60 msgr2=0x7f6c8815df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.423+0000 7f6c8d2cc700 1 --2- 172.21.15.6:0/745901208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c88070a60 0x7f6c8815df70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6c880af930 tx=0x7f6c8000b330 comp rx=0 tx=0).stop 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.423+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/745901208 shutdown_connections 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.423+0000 7f6c8d2cc700 1 --2- 172.21.15.6:0/745901208 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c88070a60 0x7f6c8815df70 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.423+0000 7f6c8d2cc700 1 --2- 172.21.15.6:0/745901208 >> v2:172.21.15.61:3300/0 conn(0x7f6c880b6510 0x7f6c880cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.423+0000 7f6c8d2cc700 1 --2- 172.21.15.6:0/745901208 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c880ccfe0 0x7f6c88070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.423+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/745901208 >> 172.21.15.6:0/745901208 conn(0x7f6c880d0bd0 msgr2=0x7f6c880d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:48.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.423+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/745901208 shutdown_connections 2023-11-28T13:04:48.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.423+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/745901208 wait complete. 2023-11-28T13:04:48.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.424+0000 7f6c8d2cc700 1 Processor -- start 2023-11-28T13:04:48.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.424+0000 7f6c8d2cc700 1 -- start start 2023-11-28T13:04:48.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c8d2cc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c88070a60 0x7f6c8815c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.429 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c8d2cc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6c880b6510 0x7f6c880c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c8d2cc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c880ccfe0 0x7f6c880c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c8d2cc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8806c8a0 con 0x7f6c880ccfe0 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c8d2cc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8806bc90 con 0x7f6c880b6510 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c8d2cc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8806bf70 con 0x7f6c88070a60 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c877fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c880ccfe0 0x7f6c880c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c877fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c880ccfe0 0x7f6c880c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34762/0 (socket says 172.21.15.6:34762) 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c877fe700 1 -- 172.21.15.6:0/563552674 learned_addr learned my addr 172.21.15.6:0/563552674 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c86ffd700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c88070a60 0x7f6c8815c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.425+0000 7f6c867fc700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6c880b6510 0x7f6c880c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c877fe700 1 -- 172.21.15.6:0/563552674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c88070a60 msgr2=0x7f6c8815c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c877fe700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c88070a60 0x7f6c8815c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c877fe700 1 -- 172.21.15.6:0/563552674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6c880b6510 msgr2=0x7f6c880c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c877fe700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6c880b6510 0x7f6c880c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c877fe700 1 -- 172.21.15.6:0/563552674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c80009b30 con 0x7f6c880ccfe0 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c877fe700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c880ccfe0 0x7f6c880c8d20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6c8806ae00 tx=0x7f6c80004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c7f7fe700 1 -- 172.21.15.6:0/563552674 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c800099b0 con 0x7f6c880ccfe0 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c7f7fe700 1 -- 172.21.15.6:0/563552674 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8000ce80 con 0x7f6c880ccfe0 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/563552674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c880cb760 con 0x7f6c880ccfe0 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c7f7fe700 1 -- 172.21.15.6:0/563552674 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80004490 con 0x7f6c880ccfe0 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.426+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/563552674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c880b12b0 con 0x7f6c880ccfe0 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.427+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/563552674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8815e4b0 con 0x7f6c880ccfe0 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.428+0000 7f6c7f7fe700 1 -- 172.21.15.6:0/563552674 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6c800045f0 con 0x7f6c880ccfe0 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.428+0000 7f6c7f7fe700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c680682f0 0x7f6c6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:48.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.428+0000 7f6c7f7fe700 1 -- 172.21.15.6:0/563552674 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6c80087300 con 0x7f6c880ccfe0 2023-11-28T13:04:48.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.428+0000 7f6c86ffd700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c680682f0 0x7f6c6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:48.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.429+0000 7f6c86ffd700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c680682f0 0x7f6c6806a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6c880c4270 tx=0x7f6c78009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:48.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.433+0000 7f6c7f7fe700 1 -- 172.21.15.6:0/563552674 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6c80053a90 con 0x7f6c880ccfe0 2023-11-28T13:04:48.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.669+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/563552674 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6c880675a0 con 0x7f6c880ccfe0 2023-11-28T13:04:48.671 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.670+0000 7f6c7f7fe700 1 -- 172.21.15.6:0/563552674 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f6c80004880 con 0x7f6c880ccfe0 2023-11-28T13:04:48.671 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:48.671 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:48.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/563552674 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c680682f0 msgr2=0x7f6c6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:48.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c680682f0 0x7f6c6806a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6c880c4270 tx=0x7f6c78009210 comp rx=0 tx=0).stop 2023-11-28T13:04:48.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/563552674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c880ccfe0 msgr2=0x7f6c880c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:48.673 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c880ccfe0 0x7f6c880c8d20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6c8806ae00 tx=0x7f6c80004ad0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/563552674 shutdown_connections 2023-11-28T13:04:48.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c880ccfe0 0x7f6c880c8d20 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f6c680682f0 0x7f6c6806a7a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f6c88070a60 0x7f6c8815c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 --2- 172.21.15.6:0/563552674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6c880b6510 0x7f6c880c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:48.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/563552674 >> 172.21.15.6:0/563552674 conn(0x7f6c880d0bd0 msgr2=0x7f6c88156390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:48.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/563552674 shutdown_connections 2023-11-28T13:04:48.674 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:48.672+0000 7f6c8d2cc700 1 -- 172.21.15.6:0/563552674 wait complete. 2023-11-28T13:04:48.685 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:53.686 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:53.824 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.823+0000 7f3dd9d23700 1 Processor -- start 2023-11-28T13:04:53.825 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.823+0000 7f3dd9d23700 1 -- start start 2023-11-28T13:04:53.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.824+0000 7f3dd9d23700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3dd4070520 0x7f3dd40708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:53.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.824+0000 7f3dd9d23700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd415dbd0 con 0x7f3dd40cb6c0 2023-11-28T13:04:53.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.824+0000 7f3dd9d23700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd4070e30 con 0x7f3dd40cb220 2023-11-28T13:04:53.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.824+0000 7f3dd9d23700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd415d610 con 0x7f3dd4070520 2023-11-28T13:04:53.827 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.824+0000 7f3dd3fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f3dd40cb6c0 0x7f3dd415c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50324/0 (socket says 172.21.15.6:50324) 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.824+0000 7f3dd3fff700 1 -- 172.21.15.6:0/860107453 learned_addr learned my addr 172.21.15.6:0/860107453 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.824+0000 7f3dd37fe700 1 --2- 172.21.15.6:0/860107453 >> v2:172.21.15.61:3300/0 conn(0x7f3dd4070520 0x7f3dd40708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.824+0000 7f3dd27fc700 1 -- 172.21.15.6:0/860107453 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827882433 0 0) 0x7f3dd415dbd0 con 0x7f3dd40cb6c0 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.824+0000 7f3dd27fc700 1 -- 172.21.15.6:0/860107453 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dbc0035b0 con 0x7f3dd40cb6c0 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.825+0000 7f3dd37fe700 1 -- 172.21.15.6:0/860107453 >> v1:172.21.15.61:6789/0 conn(0x7f3dd40cb220 legacy=0x7f3dd40b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.825+0000 7f3dd37fe700 1 -- 172.21.15.6:0/860107453 >> v1:172.21.15.6:6789/0 conn(0x7f3dd40cb6c0 legacy=0x7f3dd415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.825+0000 7f3dd37fe700 1 -- 172.21.15.6:0/860107453 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd415d8f0 con 0x7f3dd4070520 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.825+0000 7f3dd37fe700 1 --2- 172.21.15.6:0/860107453 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3dd4070520 0x7f3dd40708f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009840 tx=0x7f3dc4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1ee54dae454629a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.825+0000 7f3dd27fc700 1 -- 172.21.15.6:0/860107453 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4011830 con 0x7f3dd4070520 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.825+0000 7f3dd27fc700 1 -- 172.21.15.6:0/860107453 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4011990 con 0x7f3dd4070520 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.825+0000 7f3dd27fc700 1 -- 172.21.15.6:0/860107453 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4011d00 con 0x7f3dd4070520 2023-11-28T13:04:53.828 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.826+0000 7f3dd9d23700 1 -- 172.21.15.6:0/860107453 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3dd4070520 msgr2=0x7f3dd40708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:53.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.826+0000 7f3dd9d23700 1 --2- 172.21.15.6:0/860107453 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3dd4070520 0x7f3dd40708f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009840 tx=0x7f3dc4009b50 comp rx=0 tx=0).stop 2023-11-28T13:04:53.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.826+0000 7f3dd9d23700 1 -- 172.21.15.6:0/860107453 shutdown_connections 2023-11-28T13:04:53.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.826+0000 7f3dd9d23700 1 --2- 172.21.15.6:0/860107453 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3dd4070520 0x7f3dd40708f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:53.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.826+0000 7f3dd9d23700 1 -- 172.21.15.6:0/860107453 >> 172.21.15.6:0/860107453 conn(0x7f3dd40d0bd0 msgr2=0x7f3dd40d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:53.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.826+0000 7f3dd9d23700 1 -- 172.21.15.6:0/860107453 shutdown_connections 2023-11-28T13:04:53.829 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.826+0000 7f3dd9d23700 1 -- 172.21.15.6:0/860107453 wait complete. 2023-11-28T13:04:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.827+0000 7f3dd9d23700 1 Processor -- start 2023-11-28T13:04:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.828+0000 7f3dd9d23700 1 -- start start 2023-11-28T13:04:53.831 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.828+0000 7f3dd9d23700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dd4070520 0x7f3dd414ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.828+0000 7f3dd9d23700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3dd40cb220 0x7f3dd415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.828+0000 7f3dd9d23700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3dd40cb6c0 0x7f3dd40c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.828+0000 7f3dd9d23700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd415f920 con 0x7f3dd4070520 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.828+0000 7f3dd9d23700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd415fd40 con 0x7f3dd40cb220 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.828+0000 7f3dd9d23700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd415eca0 con 0x7f3dd40cb6c0 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dd4070520 0x7f3dd414ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd37fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dd4070520 0x7f3dd414ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34774/0 (socket says 172.21.15.6:34774) 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd37fe700 1 -- 172.21.15.6:0/2890441049 learned_addr learned my addr 172.21.15.6:0/2890441049 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd3fff700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3dd40cb6c0 0x7f3dd40c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd2ffd700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3dd40cb220 0x7f3dd415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd37fe700 1 -- 172.21.15.6:0/2890441049 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3dd40cb6c0 msgr2=0x7f3dd40c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:53.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd37fe700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3dd40cb6c0 0x7f3dd40c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd37fe700 1 -- 172.21.15.6:0/2890441049 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3dd40cb220 msgr2=0x7f3dd415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd37fe700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3dd40cb220 0x7f3dd415e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd37fe700 1 -- 172.21.15.6:0/2890441049 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc40094a0 con 0x7f3dd4070520 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd3fff700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3dd40cb6c0 0x7f3dd40c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd2ffd700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3dd40cb220 0x7f3dd415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd37fe700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dd4070520 0x7f3dd414ff30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009fd0 tx=0x7f3dc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dbbfff700 1 -- 172.21.15.6:0/2890441049 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40036a0 con 0x7f3dd4070520 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dbbfff700 1 -- 172.21.15.6:0/2890441049 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc401e070 con 0x7f3dd4070520 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd9d23700 1 -- 172.21.15.6:0/2890441049 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd40b0570 con 0x7f3dd4070520 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dbbfff700 1 -- 172.21.15.6:0/2890441049 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4003a40 con 0x7f3dd4070520 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.829+0000 7f3dd9d23700 1 -- 172.21.15.6:0/2890441049 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd40b0290 con 0x7f3dd4070520 2023-11-28T13:04:53.833 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.830+0000 7f3dd9d23700 1 -- 172.21.15.6:0/2890441049 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd40b2360 con 0x7f3dd4070520 2023-11-28T13:04:53.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.831+0000 7f3dbbfff700 1 -- 172.21.15.6:0/2890441049 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3dc400f050 con 0x7f3dd4070520 2023-11-28T13:04:53.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.831+0000 7f3dbbfff700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3dc00682f0 0x7f3dc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:53.834 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.831+0000 7f3dbbfff700 1 -- 172.21.15.6:0/2890441049 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f3dc4086e80 con 0x7f3dd4070520 2023-11-28T13:04:53.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.831+0000 7f3dd2ffd700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3dc00682f0 0x7f3dc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:53.835 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.832+0000 7f3dd2ffd700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3dc00682f0 0x7f3dc006a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40c7f50 tx=0x7f3dc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:53.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:53.836+0000 7f3dbbfff700 1 -- 172.21.15.6:0/2890441049 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3dc4053560 con 0x7f3dd4070520 2023-11-28T13:04:54.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.076+0000 7f3dd9d23700 1 -- 172.21.15.6:0/2890441049 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3dd4066c60 con 0x7f3dd4070520 2023-11-28T13:04:54.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.077+0000 7f3dbbfff700 1 -- 172.21.15.6:0/2890441049 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f3dc40150c0 con 0x7f3dd4070520 2023-11-28T13:04:54.078 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:54.078 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:54.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.078+0000 7f3dd9d23700 1 -- 172.21.15.6:0/2890441049 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3dc00682f0 msgr2=0x7f3dc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:54.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.078+0000 7f3dd9d23700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3dc00682f0 0x7f3dc006a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40c7f50 tx=0x7f3dc8009040 comp rx=0 tx=0).stop 2023-11-28T13:04:54.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.078+0000 7f3dd9d23700 1 -- 172.21.15.6:0/2890441049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dd4070520 msgr2=0x7f3dd414ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:54.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.078+0000 7f3dd9d23700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dd4070520 0x7f3dd414ff30 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009fd0 tx=0x7f3dc4004750 comp rx=0 tx=0).stop 2023-11-28T13:04:54.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.079+0000 7f3dd9d23700 1 -- 172.21.15.6:0/2890441049 shutdown_connections 2023-11-28T13:04:54.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.079+0000 7f3dd9d23700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3dc00682f0 0x7f3dc006a7a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.079+0000 7f3dd9d23700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3dd40cb6c0 0x7f3dd40c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.080 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.079+0000 7f3dd9d23700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3dd4070520 0x7f3dd414ff30 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.079+0000 7f3dd9d23700 1 --2- 172.21.15.6:0/2890441049 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3dd40cb220 0x7f3dd415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:54.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.079+0000 7f3dd9d23700 1 -- 172.21.15.6:0/2890441049 >> 172.21.15.6:0/2890441049 conn(0x7f3dd40d0bd0 msgr2=0x7f3dd406cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:54.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.079+0000 7f3dd9d23700 1 -- 172.21.15.6:0/2890441049 shutdown_connections 2023-11-28T13:04:54.081 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:54.079+0000 7f3dd9d23700 1 -- 172.21.15.6:0/2890441049 wait complete. 2023-11-28T13:04:54.092 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:04:57.111 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:04:57.114 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:04:59.093 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:04:59.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.229+0000 7fa18dae2700 1 Processor -- start 2023-11-28T13:04:59.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.229+0000 7fa18dae2700 1 -- start start 2023-11-28T13:04:59.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.230+0000 7fa18dae2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b4a70 0x7fa1880cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.230+0000 7fa18dae2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa1880aefe0 0x7fa1880cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.230+0000 7fa18dae2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1880b5880 con 0x7fa1880cdb50 2023-11-28T13:04:59.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.230+0000 7fa18dae2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1880d01e0 con 0x7fa1880aefe0 2023-11-28T13:04:59.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.230+0000 7fa18dae2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880d04c0 con 0x7fa1880b4a70 2023-11-28T13:04:59.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa1877fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b4a70 0x7fa1880cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa187fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa1880aefe0 0x7fa1880cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.233 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa1877fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b4a70 0x7fa1880cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:52718/0 (socket says 172.21.15.6:52718) 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa187fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa1880aefe0 0x7fa1880cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:38504/0 (socket says 172.21.15.6:38504) 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa1877fe700 1 -- 172.21.15.6:0/1628539960 learned_addr learned my addr 172.21.15.6:0/1628539960 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa1877fe700 1 -- 172.21.15.6:0/1628539960 >> v2:172.21.15.61:3300/0 conn(0x7fa1880aefe0 msgr2=0x7fa1880cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa1877fe700 1 --2- 172.21.15.6:0/1628539960 >> v2:172.21.15.61:3300/0 conn(0x7fa1880aefe0 0x7fa1880cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa1877fe700 1 -- 172.21.15.6:0/1628539960 >> v1:172.21.15.61:6789/0 conn(0x7fa1880cdb50 legacy=0x7fa1880b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa1877fe700 1 -- 172.21.15.6:0/1628539960 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1880d07a0 con 0x7fa1880b4a70 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa187fff700 1 --2- 172.21.15.6:0/1628539960 >> v2:172.21.15.61:3300/0 conn(0x7fa1880aefe0 0x7fa1880cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa1877fe700 1 --2- 172.21.15.6:0/1628539960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b4a70 0x7fa1880cd610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1700098f0 tx=0x7fa170009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31dbf095f5d881b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.231+0000 7fa1867fc700 1 -- 172.21.15.6:0/1628539960 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170011830 con 0x7fa1880b4a70 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.232+0000 7fa1867fc700 1 -- 172.21.15.6:0/1628539960 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa170011990 con 0x7fa1880b4a70 2023-11-28T13:04:59.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.232+0000 7fa1867fc700 1 -- 172.21.15.6:0/1628539960 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170011d20 con 0x7fa1880b4a70 2023-11-28T13:04:59.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.232+0000 7fa18dae2700 1 -- 172.21.15.6:0/1628539960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b4a70 msgr2=0x7fa1880cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.232+0000 7fa18dae2700 1 --2- 172.21.15.6:0/1628539960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b4a70 0x7fa1880cd610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1700098f0 tx=0x7fa170009c00 comp rx=0 tx=0).stop 2023-11-28T13:04:59.235 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.232+0000 7fa18dae2700 1 -- 172.21.15.6:0/1628539960 shutdown_connections 2023-11-28T13:04:59.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.232+0000 7fa18dae2700 1 --2- 172.21.15.6:0/1628539960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880b4a70 0x7fa1880cd610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.232+0000 7fa18dae2700 1 --2- 172.21.15.6:0/1628539960 >> v2:172.21.15.61:3300/0 conn(0x7fa1880aefe0 0x7fa1880cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.232+0000 7fa18dae2700 1 -- 172.21.15.6:0/1628539960 >> 172.21.15.6:0/1628539960 conn(0x7fa1880b3610 msgr2=0x7fa1880cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:59.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.232+0000 7fa18dae2700 1 -- 172.21.15.6:0/1628539960 shutdown_connections 2023-11-28T13:04:59.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.232+0000 7fa18dae2700 1 -- 172.21.15.6:0/1628539960 wait complete. 2023-11-28T13:04:59.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.234+0000 7fa18dae2700 1 Processor -- start 2023-11-28T13:04:59.236 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.234+0000 7fa18dae2700 1 -- start start 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa18dae2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1880aefe0 0x7fa1881df910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa18dae2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1880b4a70 0x7fa1881ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa18dae2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880cdb50 0x7fa1881ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa18dae2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880c3530 con 0x7fa1880b4a70 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa18dae2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880c49c0 con 0x7fa1880aefe0 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa18dae2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880c4c60 con 0x7fa1880cdb50 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa186ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1880b4a70 0x7fa1881ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa186ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1880b4a70 0x7fa1881ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34790/0 (socket says 172.21.15.6:34790) 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa186ffd700 1 -- 172.21.15.6:0/1298823913 learned_addr learned my addr 172.21.15.6:0/1298823913 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa187fff700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880cdb50 0x7fa1881ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.239 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.235+0000 7fa1877fe700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1880aefe0 0x7fa1881df910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa186ffd700 1 -- 172.21.15.6:0/1298823913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880cdb50 msgr2=0x7fa1881ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa186ffd700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880cdb50 0x7fa1881ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa186ffd700 1 -- 172.21.15.6:0/1298823913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1880aefe0 msgr2=0x7fa1881df910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa186ffd700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1880aefe0 0x7fa1881df910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa186ffd700 1 -- 172.21.15.6:0/1298823913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180009a70 con 0x7fa1880b4a70 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa187fff700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880cdb50 0x7fa1881ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa186ffd700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1880b4a70 0x7fa1881ce140 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa17800c4e0 tx=0x7fa17800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa17ffff700 1 -- 172.21.15.6:0/1298823913 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780052d0 con 0x7fa1880b4a70 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa17ffff700 1 -- 172.21.15.6:0/1298823913 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa178005430 con 0x7fa1880b4a70 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa18dae2700 1 -- 172.21.15.6:0/1298823913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa170009550 con 0x7fa1880b4a70 2023-11-28T13:04:59.240 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa17ffff700 1 -- 172.21.15.6:0/1298823913 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780038c0 con 0x7fa1880b4a70 2023-11-28T13:04:59.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.236+0000 7fa18dae2700 1 -- 172.21.15.6:0/1298823913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1880aead0 con 0x7fa1880b4a70 2023-11-28T13:04:59.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.237+0000 7fa18dae2700 1 -- 172.21.15.6:0/1298823913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1880b5550 con 0x7fa1880b4a70 2023-11-28T13:04:59.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.238+0000 7fa17ffff700 1 -- 172.21.15.6:0/1298823913 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa178019030 con 0x7fa1880b4a70 2023-11-28T13:04:59.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.238+0000 7fa17ffff700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1680682f0 0x7fa16806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:04:59.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.238+0000 7fa17ffff700 1 -- 172.21.15.6:0/1298823913 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa1780851c0 con 0x7fa1880b4a70 2023-11-28T13:04:59.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.238+0000 7fa1877fe700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1680682f0 0x7fa16806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:04:59.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.238+0000 7fa1877fe700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1680682f0 0x7fa16806a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa170009c40 tx=0x7fa170008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:04:59.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.242+0000 7fa17ffff700 1 -- 172.21.15.6:0/1298823913 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa1780518a0 con 0x7fa1880b4a70 2023-11-28T13:04:59.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.480+0000 7fa18dae2700 1 -- 172.21.15.6:0/1298823913 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa1880aec60 con 0x7fa1880b4a70 2023-11-28T13:04:59.481 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.480+0000 7fa17ffff700 1 -- 172.21.15.6:0/1298823913 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fa178054ec0 con 0x7fa1880b4a70 2023-11-28T13:04:59.482 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:04:59.482 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:04:59.483 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 -- 172.21.15.6:0/1298823913 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1680682f0 msgr2=0x7fa16806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1680682f0 0x7fa16806a7a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa170009c40 tx=0x7fa170008030 comp rx=0 tx=0).stop 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 -- 172.21.15.6:0/1298823913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1880b4a70 msgr2=0x7fa1881ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1880b4a70 0x7fa1881ce140 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa17800c4e0 tx=0x7fa17800c8a0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 -- 172.21.15.6:0/1298823913 shutdown_connections 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1680682f0 0x7fa16806a7a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa1880cdb50 0x7fa1881ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa1880aefe0 0x7fa1881df910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 --2- 172.21.15.6:0/1298823913 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1880b4a70 0x7fa1881ce140 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 -- 172.21.15.6:0/1298823913 >> 172.21.15.6:0/1298823913 conn(0x7fa1880b3610 msgr2=0x7fa1880ac660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 -- 172.21.15.6:0/1298823913 shutdown_connections 2023-11-28T13:04:59.484 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:04:59.482+0000 7fa18dae2700 1 -- 172.21.15.6:0/1298823913 wait complete. 2023-11-28T13:04:59.495 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:04.496 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.632+0000 7ff23806f700 1 Processor -- start 2023-11-28T13:05:04.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.632+0000 7ff23806f700 1 -- start start 2023-11-28T13:05:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.633+0000 7ff23806f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff2300b4a70 0x7ff2300cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.633+0000 7ff23806f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2300cdb50 0x7ff2300b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.633+0000 7ff23806f700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2300b5880 con 0x7ff2300aefe0 2023-11-28T13:05:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.633+0000 7ff23806f700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2300d01e0 con 0x7ff2300b4a70 2023-11-28T13:05:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.633+0000 7ff23806f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300d04c0 con 0x7ff2300cdb50 2023-11-28T13:05:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.633+0000 7ff235e0b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff2300b4a70 0x7ff2300cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:04.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.633+0000 7ff235e0b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff2300b4a70 0x7ff2300cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34796/0 (socket says 172.21.15.6:34796) 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.633+0000 7ff235e0b700 1 -- 172.21.15.6:0/3731454011 learned_addr learned my addr 172.21.15.6:0/3731454011 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.634+0000 7ff23560a700 1 --2- 172.21.15.6:0/3731454011 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2300cdb50 0x7ff2300b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.634+0000 7ff235e0b700 1 -- 172.21.15.6:0/3731454011 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2300cdb50 msgr2=0x7ff2300b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.634+0000 7ff235e0b700 1 --2- 172.21.15.6:0/3731454011 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2300cdb50 0x7ff2300b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.634+0000 7ff235e0b700 1 -- 172.21.15.6:0/3731454011 >> v1:172.21.15.61:6789/0 conn(0x7ff2300aefe0 legacy=0x7ff2300cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.634+0000 7ff235e0b700 1 -- 172.21.15.6:0/3731454011 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2300d07a0 con 0x7ff2300b4a70 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.634+0000 7ff235e0b700 1 --2- 172.21.15.6:0/3731454011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 0x7ff2300cd610 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0098f0 tx=0x7ff22c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cd46bc588c53c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.634+0000 7ff234e09700 1 -- 172.21.15.6:0/3731454011 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c004230 con 0x7ff2300b4a70 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.634+0000 7ff234e09700 1 -- 172.21.15.6:0/3731454011 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c004390 con 0x7ff2300b4a70 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.634+0000 7ff234e09700 1 -- 172.21.15.6:0/3731454011 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0036a0 con 0x7ff2300b4a70 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.635+0000 7ff23806f700 1 -- 172.21.15.6:0/3731454011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 msgr2=0x7ff2300cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:04.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.635+0000 7ff23806f700 1 --2- 172.21.15.6:0/3731454011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 0x7ff2300cd610 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff22c0098f0 tx=0x7ff22c009c00 comp rx=0 tx=0).stop 2023-11-28T13:05:04.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.635+0000 7ff23806f700 1 -- 172.21.15.6:0/3731454011 shutdown_connections 2023-11-28T13:05:04.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.635+0000 7ff23806f700 1 --2- 172.21.15.6:0/3731454011 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 0x7ff2300cd610 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:04.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.635+0000 7ff23806f700 1 --2- 172.21.15.6:0/3731454011 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2300cdb50 0x7ff2300b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:04.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.635+0000 7ff23806f700 1 -- 172.21.15.6:0/3731454011 >> 172.21.15.6:0/3731454011 conn(0x7ff2300b3610 msgr2=0x7ff2300cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:04.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.635+0000 7ff23806f700 1 -- 172.21.15.6:0/3731454011 shutdown_connections 2023-11-28T13:05:04.638 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.635+0000 7ff23806f700 1 -- 172.21.15.6:0/3731454011 wait complete. 2023-11-28T13:05:04.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.636+0000 7ff23806f700 1 Processor -- start 2023-11-28T13:05:04.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.636+0000 7ff23806f700 1 -- start start 2023-11-28T13:05:04.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23806f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff2300aefe0 0x7ff2301d3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:04.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23806f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 0x7ff2301e1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:04.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23806f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2300cdb50 0x7ff2300c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:04.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23806f700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300c4220 con 0x7ff2300b4a70 2023-11-28T13:05:04.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23806f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300c75e0 con 0x7ff2300aefe0 2023-11-28T13:05:04.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23806f700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2301d4190 con 0x7ff2300cdb50 2023-11-28T13:05:04.641 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23560a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 0x7ff2301e1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23560a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 0x7ff2301e1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34804/0 (socket says 172.21.15.6:34804) 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23560a700 1 -- 172.21.15.6:0/1511768637 learned_addr learned my addr 172.21.15.6:0/1511768637 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff235e0b700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff2300aefe0 0x7ff2301d3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23660c700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2300cdb50 0x7ff2300c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23560a700 1 -- 172.21.15.6:0/1511768637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2300cdb50 msgr2=0x7ff2300c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23560a700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2300cdb50 0x7ff2300c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23560a700 1 -- 172.21.15.6:0/1511768637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff2300aefe0 msgr2=0x7ff2301d3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23560a700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff2300aefe0 0x7ff2301d3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23560a700 1 -- 172.21.15.6:0/1511768637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c009550 con 0x7ff2300b4a70 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.637+0000 7ff23560a700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 0x7ff2301e1410 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff22000c7f0 tx=0x7ff22000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.638+0000 7ff2267fc700 1 -- 172.21.15.6:0/1511768637 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220004230 con 0x7ff2300b4a70 2023-11-28T13:05:04.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.638+0000 7ff2267fc700 1 -- 172.21.15.6:0/1511768637 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff220004390 con 0x7ff2300b4a70 2023-11-28T13:05:04.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.638+0000 7ff2267fc700 1 -- 172.21.15.6:0/1511768637 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2200036a0 con 0x7ff2300b4a70 2023-11-28T13:05:04.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.638+0000 7ff23806f700 1 -- 172.21.15.6:0/1511768637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2300aec60 con 0x7ff2300b4a70 2023-11-28T13:05:04.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.638+0000 7ff23806f700 1 -- 172.21.15.6:0/1511768637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2300aead0 con 0x7ff2300b4a70 2023-11-28T13:05:04.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.639+0000 7ff23806f700 1 -- 172.21.15.6:0/1511768637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2300cbad0 con 0x7ff2300b4a70 2023-11-28T13:05:04.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.639+0000 7ff2267fc700 1 -- 172.21.15.6:0/1511768637 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff220020070 con 0x7ff2300b4a70 2023-11-28T13:05:04.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.639+0000 7ff2267fc700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff2140682c0 0x7ff21406a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:04.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.640+0000 7ff2267fc700 1 -- 172.21.15.6:0/1511768637 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff2200859b0 con 0x7ff2300b4a70 2023-11-28T13:05:04.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.640+0000 7ff235e0b700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff2140682c0 0x7ff21406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:04.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.640+0000 7ff235e0b700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff2140682c0 0x7ff21406a770 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009130 tx=0x7ff22c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:04.645 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.644+0000 7ff2267fc700 1 -- 172.21.15.6:0/1511768637 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff220052090 con 0x7ff2300b4a70 2023-11-28T13:05:04.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.881+0000 7ff23806f700 1 -- 172.21.15.6:0/1511768637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff2300c7750 con 0x7ff2300b4a70 2023-11-28T13:05:04.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.881+0000 7ff2267fc700 1 -- 172.21.15.6:0/1511768637 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ff22001e0c0 con 0x7ff2300b4a70 2023-11-28T13:05:04.920 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:04.920 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:04.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 -- 172.21.15.6:0/1511768637 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff2140682c0 msgr2=0x7ff21406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:04.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff2140682c0 0x7ff21406a770 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009130 tx=0x7ff22c008030 comp rx=0 tx=0).stop 2023-11-28T13:05:04.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 -- 172.21.15.6:0/1511768637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 msgr2=0x7ff2301e1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:04.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 0x7ff2301e1410 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff22000c7f0 tx=0x7ff22000cb00 comp rx=0 tx=0).stop 2023-11-28T13:05:04.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 -- 172.21.15.6:0/1511768637 shutdown_connections 2023-11-28T13:05:04.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff2140682c0 0x7ff21406a770 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:04.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff2300cdb50 0x7ff2300c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:04.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff2300aefe0 0x7ff2301d3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:04.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 --2- 172.21.15.6:0/1511768637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2300b4a70 0x7ff2301e1410 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:04.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 -- 172.21.15.6:0/1511768637 >> 172.21.15.6:0/1511768637 conn(0x7ff2300b3610 msgr2=0x7ff2301e0430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:04.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 -- 172.21.15.6:0/1511768637 shutdown_connections 2023-11-28T13:05:04.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:04.883+0000 7ff23806f700 1 -- 172.21.15.6:0/1511768637 wait complete. 2023-11-28T13:05:04.923 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:09.924 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:10.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.062+0000 7f53f1492700 1 Processor -- start 2023-11-28T13:05:10.063 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.062+0000 7f53f1492700 1 -- start start 2023-11-28T13:05:10.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53f1492700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f53ec0b49d0 0x7f53ec0d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53f1492700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec0cd5a0 0x7f53ec0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53f1492700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ec0b4fb0 con 0x7f53ec15c6a0 2023-11-28T13:05:10.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53f1492700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec0b5570 con 0x7f53ec0b49d0 2023-11-28T13:05:10.067 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53f1492700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0cdf60 con 0x7f53ec0cd5a0 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53ea7fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f53ec0b49d0 0x7f53ec0d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53eaffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f53ec15c6a0 0x7f53ec06abf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50362/0 (socket says 172.21.15.6:50362) 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53eaffd700 1 -- 172.21.15.6:0/4213116660 learned_addr learned my addr 172.21.15.6:0/4213116660 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53eb7fe700 1 --2- 172.21.15.6:0/4213116660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec0cd5a0 0x7f53ec0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53ea7fc700 1 -- 172.21.15.6:0/4213116660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec0cd5a0 msgr2=0x7f53ec0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53ea7fc700 1 --2- 172.21.15.6:0/4213116660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec0cd5a0 0x7f53ec0cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53e9ffb700 1 -- 172.21.15.6:0/4213116660 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608958394 0 0) 0x7f53ec0b4fb0 con 0x7f53ec15c6a0 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53ea7fc700 1 -- 172.21.15.6:0/4213116660 >> v1:172.21.15.6:6789/0 conn(0x7f53ec15c6a0 legacy=0x7f53ec06abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53ea7fc700 1 -- 172.21.15.6:0/4213116660 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0b5290 con 0x7f53ec0b49d0 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53eb7fe700 1 --2- 172.21.15.6:0/4213116660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec0cd5a0 0x7f53ec0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.063+0000 7f53ea7fc700 1 --2- 172.21.15.6:0/4213116660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 0x7f53ec0d19d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009840 tx=0x7f53e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f98cbe0b2baa08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:10.068 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53e9ffb700 1 -- 172.21.15.6:0/4213116660 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0011830 con 0x7f53ec0b49d0 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53e9ffb700 1 -- 172.21.15.6:0/4213116660 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0011990 con 0x7f53ec0b49d0 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53e9ffb700 1 -- 172.21.15.6:0/4213116660 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0011d20 con 0x7f53ec0b49d0 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53f1492700 1 -- 172.21.15.6:0/4213116660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 msgr2=0x7f53ec0d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53f1492700 1 --2- 172.21.15.6:0/4213116660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 0x7f53ec0d19d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009840 tx=0x7f53e0009b50 comp rx=0 tx=0).stop 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53f1492700 1 -- 172.21.15.6:0/4213116660 shutdown_connections 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53f1492700 1 --2- 172.21.15.6:0/4213116660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec0cd5a0 0x7f53ec0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53f1492700 1 --2- 172.21.15.6:0/4213116660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 0x7f53ec0d19d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53f1492700 1 -- 172.21.15.6:0/4213116660 >> 172.21.15.6:0/4213116660 conn(0x7f53ec0d2f50 msgr2=0x7f53ec0d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53f1492700 1 -- 172.21.15.6:0/4213116660 shutdown_connections 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.064+0000 7f53f1492700 1 -- 172.21.15.6:0/4213116660 wait complete. 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.066+0000 7f53f1492700 1 Processor -- start 2023-11-28T13:05:10.069 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.066+0000 7f53f1492700 1 -- start start 2023-11-28T13:05:10.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53f1492700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 0x7f53ec0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53f1492700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53ec0cd5a0 0x7f53ec0c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53f1492700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec15c6a0 0x7f53ec0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53f1492700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec151d60 con 0x7f53ec0b49d0 2023-11-28T13:05:10.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53f1492700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1514c0 con 0x7f53ec0cd5a0 2023-11-28T13:05:10.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53f1492700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1517a0 con 0x7f53ec15c6a0 2023-11-28T13:05:10.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 0x7f53ec0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 0x7f53ec0cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34816/0 (socket says 172.21.15.6:34816) 2023-11-28T13:05:10.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53eaffd700 1 -- 172.21.15.6:0/648842801 learned_addr learned my addr 172.21.15.6:0/648842801 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:10.072 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53ea7fc700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53ec0cd5a0 0x7f53ec0c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53eb7fe700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec15c6a0 0x7f53ec0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53eaffd700 1 -- 172.21.15.6:0/648842801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec15c6a0 msgr2=0x7f53ec0c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53eaffd700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec15c6a0 0x7f53ec0c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53eaffd700 1 -- 172.21.15.6:0/648842801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53ec0cd5a0 msgr2=0x7f53ec0c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53eaffd700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53ec0cd5a0 0x7f53ec0c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53eaffd700 1 -- 172.21.15.6:0/648842801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00094a0 con 0x7f53ec0b49d0 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53eaffd700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 0x7f53ec0cb8b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00c550 tx=0x7f53dc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.067+0000 7f53d37fe700 1 -- 172.21.15.6:0/648842801 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc004c80 con 0x7f53ec0b49d0 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.068+0000 7f53d37fe700 1 -- 172.21.15.6:0/648842801 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc004e00 con 0x7f53ec0b49d0 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.068+0000 7f53f1492700 1 -- 172.21.15.6:0/648842801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec0d14b0 con 0x7f53ec0b49d0 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.068+0000 7f53d37fe700 1 -- 172.21.15.6:0/648842801 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc007310 con 0x7f53ec0b49d0 2023-11-28T13:05:10.073 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.068+0000 7f53f1492700 1 -- 172.21.15.6:0/648842801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec0d11d0 con 0x7f53ec0b49d0 2023-11-28T13:05:10.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.069+0000 7f53f1492700 1 -- 172.21.15.6:0/648842801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec15ca70 con 0x7f53ec0b49d0 2023-11-28T13:05:10.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.069+0000 7f53d37fe700 1 -- 172.21.15.6:0/648842801 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f53dc01a020 con 0x7f53ec0b49d0 2023-11-28T13:05:10.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.069+0000 7f53d37fe700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f53d80682f0 0x7f53d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:10.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.069+0000 7f53d37fe700 1 -- 172.21.15.6:0/648842801 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f53dc085830 con 0x7f53ec0b49d0 2023-11-28T13:05:10.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.069+0000 7f53ea7fc700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f53d80682f0 0x7f53d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:10.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.070+0000 7f53ea7fc700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f53d80682f0 0x7f53d806a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f53e00090b0 tx=0x7f53e0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:10.075 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.074+0000 7f53d37fe700 1 -- 172.21.15.6:0/648842801 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f53dc051f10 con 0x7f53ec0b49d0 2023-11-28T13:05:10.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.312+0000 7f53f1492700 1 -- 172.21.15.6:0/648842801 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f53ec068570 con 0x7f53ec0b49d0 2023-11-28T13:05:10.314 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.312+0000 7f53d37fe700 1 -- 172.21.15.6:0/648842801 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f53dc055530 con 0x7f53ec0b49d0 2023-11-28T13:05:10.314 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:10.315 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:10.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.314+0000 7f53f1492700 1 -- 172.21.15.6:0/648842801 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f53d80682f0 msgr2=0x7f53d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.314+0000 7f53f1492700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f53d80682f0 0x7f53d806a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f53e00090b0 tx=0x7f53e0008030 comp rx=0 tx=0).stop 2023-11-28T13:05:10.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.314+0000 7f53f1492700 1 -- 172.21.15.6:0/648842801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 msgr2=0x7f53ec0cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:10.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.314+0000 7f53f1492700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 0x7f53ec0cb8b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00c550 tx=0x7f53dc00c860 comp rx=0 tx=0).stop 2023-11-28T13:05:10.316 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.314+0000 7f53f1492700 1 -- 172.21.15.6:0/648842801 shutdown_connections 2023-11-28T13:05:10.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.314+0000 7f53f1492700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f53d80682f0 0x7f53d806a7a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.314+0000 7f53f1492700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f53ec15c6a0 0x7f53ec0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.314+0000 7f53f1492700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f53ec0b49d0 0x7f53ec0cb8b0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.314+0000 7f53f1492700 1 --2- 172.21.15.6:0/648842801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f53ec0cd5a0 0x7f53ec0c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:10.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.315+0000 7f53f1492700 1 -- 172.21.15.6:0/648842801 >> 172.21.15.6:0/648842801 conn(0x7f53ec0d2f50 msgr2=0x7f53ec0b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:10.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.315+0000 7f53f1492700 1 -- 172.21.15.6:0/648842801 shutdown_connections 2023-11-28T13:05:10.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:10.315+0000 7f53f1492700 1 -- 172.21.15.6:0/648842801 wait complete. 2023-11-28T13:05:10.328 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:15.329 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:15.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.466+0000 7fe856f58700 1 Processor -- start 2023-11-28T13:05:15.467 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.466+0000 7fe856f58700 1 -- start start 2023-11-28T13:05:15.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe856f58700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500b4a70 0x7fe8500cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe856f58700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fe8500aefe0 0x7fe8500cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe856f58700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8500b5880 con 0x7fe8500cdb50 2023-11-28T13:05:15.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe856f58700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8500d01e0 con 0x7fe8500aefe0 2023-11-28T13:05:15.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe856f58700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500d04c0 con 0x7fe8500b4a70 2023-11-28T13:05:15.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe84ffff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe8500cdb50 0x7fe8500b4e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50378/0 (socket says 172.21.15.6:50378) 2023-11-28T13:05:15.470 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe84ffff700 1 -- 172.21.15.6:0/2519913461 learned_addr learned my addr 172.21.15.6:0/2519913461 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe8554f5700 1 --2- 172.21.15.6:0/2519913461 >> v2:172.21.15.61:3300/0 conn(0x7fe8500aefe0 0x7fe8500cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe854cf4700 1 --2- 172.21.15.6:0/2519913461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500b4a70 0x7fe8500cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe84f7fe700 1 -- 172.21.15.6:0/2519913461 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828060287 0 0) 0x7fe8500b5880 con 0x7fe8500cdb50 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe84f7fe700 1 -- 172.21.15.6:0/2519913461 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8400035b0 con 0x7fe8500cdb50 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe8554f5700 1 -- 172.21.15.6:0/2519913461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500b4a70 msgr2=0x7fe8500cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe8554f5700 1 --2- 172.21.15.6:0/2519913461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500b4a70 0x7fe8500cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe8554f5700 1 -- 172.21.15.6:0/2519913461 >> v1:172.21.15.6:6789/0 conn(0x7fe8500cdb50 legacy=0x7fe8500b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.467+0000 7fe8554f5700 1 -- 172.21.15.6:0/2519913461 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8500d07a0 con 0x7fe8500aefe0 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.468+0000 7fe854cf4700 1 --2- 172.21.15.6:0/2519913461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500b4a70 0x7fe8500cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.468+0000 7fe8554f5700 1 --2- 172.21.15.6:0/2519913461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8500aefe0 0x7fe8500cb3c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe848009d90 tx=0x7fe84800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=881bcedf12e8409e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.468+0000 7fe84f7fe700 1 -- 172.21.15.6:0/2519913461 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84800e070 con 0x7fe8500aefe0 2023-11-28T13:05:15.471 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.468+0000 7fe84f7fe700 1 -- 172.21.15.6:0/2519913461 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84800bb20 con 0x7fe8500aefe0 2023-11-28T13:05:15.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.468+0000 7fe84f7fe700 1 -- 172.21.15.6:0/2519913461 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84800be90 con 0x7fe8500aefe0 2023-11-28T13:05:15.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.468+0000 7fe856f58700 1 -- 172.21.15.6:0/2519913461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8500aefe0 msgr2=0x7fe8500cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:15.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.468+0000 7fe856f58700 1 --2- 172.21.15.6:0/2519913461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8500aefe0 0x7fe8500cb3c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe848009d90 tx=0x7fe84800b330 comp rx=0 tx=0).stop 2023-11-28T13:05:15.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.468+0000 7fe856f58700 1 -- 172.21.15.6:0/2519913461 shutdown_connections 2023-11-28T13:05:15.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.469+0000 7fe856f58700 1 --2- 172.21.15.6:0/2519913461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8500aefe0 0x7fe8500cb3c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.469+0000 7fe856f58700 1 --2- 172.21.15.6:0/2519913461 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500b4a70 0x7fe8500cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.469+0000 7fe856f58700 1 -- 172.21.15.6:0/2519913461 >> 172.21.15.6:0/2519913461 conn(0x7fe8500b3610 msgr2=0x7fe8500cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:15.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.469+0000 7fe856f58700 1 -- 172.21.15.6:0/2519913461 shutdown_connections 2023-11-28T13:05:15.472 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.469+0000 7fe856f58700 1 -- 172.21.15.6:0/2519913461 wait complete. 2023-11-28T13:05:15.475 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.470+0000 7fe856f58700 1 Processor -- start 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.470+0000 7fe856f58700 1 -- start start 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe856f58700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8500aefe0 0x7fe8501e1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe856f58700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8500b4a70 0x7fe8501daf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe856f58700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500cdb50 0x7fe8501dcf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe856f58700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500c7460 con 0x7fe8500aefe0 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe856f58700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8500c49b0 con 0x7fe8500b4a70 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe856f58700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8501e2160 con 0x7fe8500cdb50 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe854cf4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8500aefe0 0x7fe8501e1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe854cf4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8500aefe0 0x7fe8501e1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34830/0 (socket says 172.21.15.6:34830) 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe854cf4700 1 -- 172.21.15.6:0/672740149 learned_addr learned my addr 172.21.15.6:0/672740149 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe84ffff700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8500b4a70 0x7fe8501daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe8554f5700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500cdb50 0x7fe8501dcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.476 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe854cf4700 1 -- 172.21.15.6:0/672740149 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500cdb50 msgr2=0x7fe8501dcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe854cf4700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500cdb50 0x7fe8501dcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe854cf4700 1 -- 172.21.15.6:0/672740149 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8500b4a70 msgr2=0x7fe8501daf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe854cf4700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8500b4a70 0x7fe8501daf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe854cf4700 1 -- 172.21.15.6:0/672740149 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8440094a0 con 0x7fe8500aefe0 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.471+0000 7fe854cf4700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8500aefe0 0x7fe8501e1580 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe84400ec10 tx=0x7fe844009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.472+0000 7fe84d7fa700 1 -- 172.21.15.6:0/672740149 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400c940 con 0x7fe8500aefe0 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.472+0000 7fe84d7fa700 1 -- 172.21.15.6:0/672740149 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84400caa0 con 0x7fe8500aefe0 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.472+0000 7fe84d7fa700 1 -- 172.21.15.6:0/672740149 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84400ce10 con 0x7fe8500aefe0 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.472+0000 7fe856f58700 1 -- 172.21.15.6:0/672740149 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe848009a70 con 0x7fe8500aefe0 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.472+0000 7fe856f58700 1 -- 172.21.15.6:0/672740149 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8500aead0 con 0x7fe8500aefe0 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.473+0000 7fe856f58700 1 -- 172.21.15.6:0/672740149 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8500b5550 con 0x7fe8500aefe0 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.473+0000 7fe84d7fa700 1 -- 172.21.15.6:0/672740149 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe844019030 con 0x7fe8500aefe0 2023-11-28T13:05:15.477 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.474+0000 7fe84d7fa700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8340682f0 0x7fe83406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:15.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.474+0000 7fe84d7fa700 1 -- 172.21.15.6:0/672740149 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fe844085f80 con 0x7fe8500aefe0 2023-11-28T13:05:15.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.474+0000 7fe84ffff700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8340682f0 0x7fe83406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:15.478 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.474+0000 7fe84ffff700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8340682f0 0x7fe83406a7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8501ddc50 tx=0x7fe83c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:15.479 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.478+0000 7fe84d7fa700 1 -- 172.21.15.6:0/672740149 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe844052570 con 0x7fe8500aefe0 2023-11-28T13:05:15.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.716+0000 7fe856f58700 1 -- 172.21.15.6:0/672740149 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe8500aec60 con 0x7fe8500aefe0 2023-11-28T13:05:15.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.716+0000 7fe84d7fa700 1 -- 172.21.15.6:0/672740149 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fe8440170c0 con 0x7fe8500aefe0 2023-11-28T13:05:15.717 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:15.718 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:15.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 -- 172.21.15.6:0/672740149 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8340682f0 msgr2=0x7fe83406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:15.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8340682f0 0x7fe83406a7a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8501ddc50 tx=0x7fe83c009040 comp rx=0 tx=0).stop 2023-11-28T13:05:15.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 -- 172.21.15.6:0/672740149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8500aefe0 msgr2=0x7fe8501e1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:15.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8500aefe0 0x7fe8501e1580 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe84400ec10 tx=0x7fe844009ec0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 -- 172.21.15.6:0/672740149 shutdown_connections 2023-11-28T13:05:15.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8340682f0 0x7fe83406a7a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe8500cdb50 0x7fe8501dcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8500aefe0 0x7fe8501e1580 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 --2- 172.21.15.6:0/672740149 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8500b4a70 0x7fe8501daf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:15.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 -- 172.21.15.6:0/672740149 >> 172.21.15.6:0/672740149 conn(0x7fe8500b3610 msgr2=0x7fe8500addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:15.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 -- 172.21.15.6:0/672740149 shutdown_connections 2023-11-28T13:05:15.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:15.718+0000 7fe856f58700 1 -- 172.21.15.6:0/672740149 wait complete. 2023-11-28T13:05:15.731 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:20.732 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:20.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.870+0000 7f27ce4c0700 1 Processor -- start 2023-11-28T13:05:20.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.870+0000 7f27ce4c0700 1 -- start start 2023-11-28T13:05:20.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27ce4c0700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f27c80ca7e0 0x7f27c80cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:20.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27ce4c0700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f27c806f290 0x7f27c80b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:20.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27ce4c0700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c80cb0f0 con 0x7f27c80b4fb0 2023-11-28T13:05:20.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27ce4c0700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c80ca410 con 0x7f27c80ca7e0 2023-11-28T13:05:20.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27ce4c0700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c80ca130 con 0x7f27c806f290 2023-11-28T13:05:20.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27c7fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f27c80ca7e0 0x7f27c80cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:20.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27c7fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f27c80ca7e0 0x7f27c80cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34838/0 (socket says 172.21.15.6:34838) 2023-11-28T13:05:20.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27c7fff700 1 -- 172.21.15.6:0/1869862491 learned_addr learned my addr 172.21.15.6:0/1869862491 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27c77fe700 1 --2- 172.21.15.6:0/1869862491 >> v2:172.21.15.61:3300/0 conn(0x7f27c806f290 0x7f27c80b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27c7fff700 1 -- 172.21.15.6:0/1869862491 >> v2:172.21.15.61:3300/0 conn(0x7f27c806f290 msgr2=0x7f27c80b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27c7fff700 1 --2- 172.21.15.6:0/1869862491 >> v2:172.21.15.61:3300/0 conn(0x7f27c806f290 0x7f27c80b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27c7fff700 1 -- 172.21.15.6:0/1869862491 >> v1:172.21.15.61:6789/0 conn(0x7f27c80b4fb0 legacy=0x7f27c80aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27c7fff700 1 -- 172.21.15.6:0/1869862491 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80b2100 con 0x7f27c80ca7e0 2023-11-28T13:05:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27c7fff700 1 --2- 172.21.15.6:0/1869862491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c80ca7e0 0x7f27c80cabb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f27c8070ac0 tx=0x7f27b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209fdd83bc90f746 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.871+0000 7f27c6ffd700 1 -- 172.21.15.6:0/1869862491 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b800b870 con 0x7f27c80ca7e0 2023-11-28T13:05:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.872+0000 7f27c6ffd700 1 -- 172.21.15.6:0/1869862491 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b800b9d0 con 0x7f27c80ca7e0 2023-11-28T13:05:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.872+0000 7f27c6ffd700 1 -- 172.21.15.6:0/1869862491 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b800bd60 con 0x7f27c80ca7e0 2023-11-28T13:05:20.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.872+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1869862491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c80ca7e0 msgr2=0x7f27c80cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.872+0000 7f27ce4c0700 1 --2- 172.21.15.6:0/1869862491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c80ca7e0 0x7f27c80cabb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f27c8070ac0 tx=0x7f27b8009bd0 comp rx=0 tx=0).stop 2023-11-28T13:05:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.872+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1869862491 shutdown_connections 2023-11-28T13:05:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.872+0000 7f27ce4c0700 1 --2- 172.21.15.6:0/1869862491 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c80ca7e0 0x7f27c80cabb0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.872+0000 7f27ce4c0700 1 --2- 172.21.15.6:0/1869862491 >> v2:172.21.15.61:3300/0 conn(0x7f27c806f290 0x7f27c80b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.872+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1869862491 >> 172.21.15.6:0/1869862491 conn(0x7f27c80b3610 msgr2=0x7f27c80cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.872+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1869862491 shutdown_connections 2023-11-28T13:05:20.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.872+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1869862491 wait complete. 2023-11-28T13:05:20.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.873+0000 7f27ce4c0700 1 Processor -- start 2023-11-28T13:05:20.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.874+0000 7f27ce4c0700 1 -- start start 2023-11-28T13:05:20.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.874+0000 7f27ce4c0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c806f290 0x7f27c81569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:20.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.874+0000 7f27ce4c0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27c80b4fb0 0x7f27c80c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:20.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.874+0000 7f27ce4c0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27c80ca7e0 0x7f27c815c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:20.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.874+0000 7f27ce4c0700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c815dab0 con 0x7f27c806f290 2023-11-28T13:05:20.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.874+0000 7f27ce4c0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8156f20 con 0x7f27c80ca7e0 2023-11-28T13:05:20.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.874+0000 7f27ce4c0700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c815cf30 con 0x7f27c80b4fb0 2023-11-28T13:05:20.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.874+0000 7f27c7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c806f290 0x7f27c81569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:20.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.874+0000 7f27c7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c806f290 0x7f27c81569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34844/0 (socket says 172.21.15.6:34844) 2023-11-28T13:05:20.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.874+0000 7f27c7fff700 1 -- 172.21.15.6:0/1370916164 learned_addr learned my addr 172.21.15.6:0/1370916164 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:20.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c77fe700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27c80b4fb0 0x7f27c80c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:20.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27cca5d700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27c80ca7e0 0x7f27c815c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:20.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c7fff700 1 -- 172.21.15.6:0/1370916164 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27c80b4fb0 msgr2=0x7f27c80c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:20.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c7fff700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27c80b4fb0 0x7f27c80c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:20.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c7fff700 1 -- 172.21.15.6:0/1370916164 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27c80ca7e0 msgr2=0x7f27c815c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:20.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c7fff700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27c80ca7e0 0x7f27c815c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:20.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c7fff700 1 -- 172.21.15.6:0/1370916164 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8009550 con 0x7f27c806f290 2023-11-28T13:05:20.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c77fe700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27c80b4fb0 0x7f27c80c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:20.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27cca5d700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27c80ca7e0 0x7f27c815c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:20.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c7fff700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c806f290 0x7f27c81569e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009130 tx=0x7f27b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:20.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c4ff9700 1 -- 172.21.15.6:0/1370916164 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004220 con 0x7f27c806f290 2023-11-28T13:05:20.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c4ff9700 1 -- 172.21.15.6:0/1370916164 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b801f070 con 0x7f27c806f290 2023-11-28T13:05:20.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27c4ff9700 1 -- 172.21.15.6:0/1370916164 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004580 con 0x7f27c806f290 2023-11-28T13:05:20.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1370916164 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c80d0280 con 0x7f27c806f290 2023-11-28T13:05:20.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.875+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1370916164 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c80b23e0 con 0x7f27c806f290 2023-11-28T13:05:20.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.876+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1370916164 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c8071cc0 con 0x7f27c806f290 2023-11-28T13:05:20.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.877+0000 7f27c4ff9700 1 -- 172.21.15.6:0/1370916164 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f27b800f050 con 0x7f27c806f290 2023-11-28T13:05:20.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.877+0000 7f27c4ff9700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27b40682f0 0x7f27b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:20.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.877+0000 7f27c4ff9700 1 -- 172.21.15.6:0/1370916164 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f27b8021070 con 0x7f27c806f290 2023-11-28T13:05:20.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.877+0000 7f27c77fe700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27b40682f0 0x7f27b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:20.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.878+0000 7f27c77fe700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27b40682f0 0x7f27b406a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f27c8157ad0 tx=0x7f27bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:20.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:20.882+0000 7f27c4ff9700 1 -- 172.21.15.6:0/1370916164 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f27b8053a50 con 0x7f27c806f290 2023-11-28T13:05:21.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.121+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1370916164 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f27c8066c60 con 0x7f27c806f290 2023-11-28T13:05:21.123 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.122+0000 7f27c4ff9700 1 -- 172.21.15.6:0/1370916164 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f27b8057070 con 0x7f27c806f290 2023-11-28T13:05:21.123 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:21.123 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:21.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1370916164 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27b40682f0 msgr2=0x7f27b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:21.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27b40682f0 0x7f27b406a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f27c8157ad0 tx=0x7f27bc008040 comp rx=0 tx=0).stop 2023-11-28T13:05:21.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1370916164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c806f290 msgr2=0x7f27c81569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:21.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c806f290 0x7f27c81569e0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009130 tx=0x7f27b8004750 comp rx=0 tx=0).stop 2023-11-28T13:05:21.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1370916164 shutdown_connections 2023-11-28T13:05:21.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27c80ca7e0 0x7f27c815c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f27c806f290 0x7f27c81569e0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f27b40682f0 0x7f27b406a7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 --2- 172.21.15.6:0/1370916164 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f27c80b4fb0 0x7f27c80c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:21.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1370916164 >> 172.21.15.6:0/1370916164 conn(0x7f27c80b3610 msgr2=0x7f27c80b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:21.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1370916164 shutdown_connections 2023-11-28T13:05:21.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:21.124+0000 7f27ce4c0700 1 -- 172.21.15.6:0/1370916164 wait complete. 2023-11-28T13:05:21.137 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:26.138 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:26.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.275+0000 7f9dc8479700 1 Processor -- start 2023-11-28T13:05:26.276 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.275+0000 7f9dc8479700 1 -- start start 2023-11-28T13:05:26.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.275+0000 7f9dc8479700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 0x7f9dc00cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.275+0000 7f9dc8479700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9dc006f290 0x7f9dc00b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc8479700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc00cb0f0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc8479700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc00ca410 con 0x7f9dc006f290 2023-11-28T13:05:26.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc8479700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00ca130 con 0x7f9dc00ca7e0 2023-11-28T13:05:26.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc5a14700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9dc006f290 0x7f9dc00b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:26.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc6215700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 0x7f9dc00cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:26.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc6a16700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f9dc00b4fb0 0x7f9dc00aca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:40438/0 (socket says 172.21.15.6:40438) 2023-11-28T13:05:26.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc6a16700 1 -- 172.21.15.6:0/2817015075 learned_addr learned my addr 172.21.15.6:0/2817015075 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:05:26.279 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc5213700 1 -- 172.21.15.6:0/2817015075 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810748780 0 0) 0x7f9dc00cb0f0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc5213700 1 -- 172.21.15.6:0/2817015075 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dac0035b0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc6215700 1 -- 172.21.15.6:0/2817015075 >> v2:172.21.15.61:3300/0 conn(0x7f9dc006f290 msgr2=0x7f9dc00b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc6215700 1 --2- 172.21.15.6:0/2817015075 >> v2:172.21.15.61:3300/0 conn(0x7f9dc006f290 0x7f9dc00b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc6215700 1 -- 172.21.15.6:0/2817015075 >> v1:172.21.15.61:6789/0 conn(0x7f9dc00b4fb0 legacy=0x7f9dc00aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc6215700 1 -- 172.21.15.6:0/2817015075 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00b2100 con 0x7f9dc00ca7e0 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.276+0000 7f9dc5a14700 1 --2- 172.21.15.6:0/2817015075 >> v2:172.21.15.61:3300/0 conn(0x7f9dc006f290 0x7f9dc00b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.277+0000 7f9dc6215700 1 --2- 172.21.15.6:0/2817015075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 0x7f9dc00cabb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009840 tx=0x7f9db0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d73118ae53e99a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.277+0000 7f9dc5213700 1 -- 172.21.15.6:0/2817015075 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011830 con 0x7f9dc00ca7e0 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.277+0000 7f9dc5213700 1 -- 172.21.15.6:0/2817015075 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db0011990 con 0x7f9dc00ca7e0 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.277+0000 7f9dc5213700 1 -- 172.21.15.6:0/2817015075 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011d20 con 0x7f9dc00ca7e0 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.277+0000 7f9dc8479700 1 -- 172.21.15.6:0/2817015075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 msgr2=0x7f9dc00cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.280 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.277+0000 7f9dc8479700 1 --2- 172.21.15.6:0/2817015075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 0x7f9dc00cabb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009840 tx=0x7f9db0009b50 comp rx=0 tx=0).stop 2023-11-28T13:05:26.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.277+0000 7f9dc8479700 1 -- 172.21.15.6:0/2817015075 shutdown_connections 2023-11-28T13:05:26.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.277+0000 7f9dc8479700 1 --2- 172.21.15.6:0/2817015075 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 0x7f9dc00cabb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.277+0000 7f9dc8479700 1 --2- 172.21.15.6:0/2817015075 >> v2:172.21.15.61:3300/0 conn(0x7f9dc006f290 0x7f9dc00b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.278+0000 7f9dc8479700 1 -- 172.21.15.6:0/2817015075 >> 172.21.15.6:0/2817015075 conn(0x7f9dc00b3610 msgr2=0x7f9dc00cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:26.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.278+0000 7f9dc8479700 1 -- 172.21.15.6:0/2817015075 shutdown_connections 2023-11-28T13:05:26.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.278+0000 7f9dc8479700 1 -- 172.21.15.6:0/2817015075 wait complete. 2023-11-28T13:05:26.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.279+0000 7f9dc8479700 1 Processor -- start 2023-11-28T13:05:26.283 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.279+0000 7f9dc8479700 1 -- start start 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.279+0000 7f9dc8479700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dc006f290 0x7f9dc01567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.279+0000 7f9dc8479700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9dc00b4fb0 0x7f9dc0155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.279+0000 7f9dc8479700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 0x7f9dc01f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc8479700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00c5180 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc8479700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00c5460 con 0x7f9dc006f290 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc8479700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00c5740 con 0x7f9dc00ca7e0 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc5a14700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9dc00b4fb0 0x7f9dc0155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc5a14700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9dc00b4fb0 0x7f9dc0155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34858/0 (socket says 172.21.15.6:34858) 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc5a14700 1 -- 172.21.15.6:0/2901721577 learned_addr learned my addr 172.21.15.6:0/2901721577 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc6a16700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 0x7f9dc01f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:26.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc6215700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dc006f290 0x7f9dc01567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc5a14700 1 -- 172.21.15.6:0/2901721577 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 msgr2=0x7f9dc01f00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc5a14700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 0x7f9dc01f00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc5a14700 1 -- 172.21.15.6:0/2901721577 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dc006f290 msgr2=0x7f9dc01567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc5a14700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dc006f290 0x7f9dc01567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc5a14700 1 -- 172.21.15.6:0/2901721577 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc0094a0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc6215700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dc006f290 0x7f9dc01567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc5a14700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9dc00b4fb0 0x7f9dc0155f40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00ebe0 tx=0x7f9dbc009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9db6ffd700 1 -- 172.21.15.6:0/2901721577 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00c8b0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9db6ffd700 1 -- 172.21.15.6:0/2901721577 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc00ca10 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9db6ffd700 1 -- 172.21.15.6:0/2901721577 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00cda0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.280+0000 7f9dc8479700 1 -- 172.21.15.6:0/2901721577 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db00094a0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.281+0000 7f9dc8479700 1 -- 172.21.15.6:0/2901721577 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc00d0280 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.282+0000 7f9dc8479700 1 -- 172.21.15.6:0/2901721577 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc00b23e0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.282+0000 7f9db6ffd700 1 -- 172.21.15.6:0/2901721577 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9dbc019030 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.282+0000 7f9db6ffd700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9da40682f0 0x7f9da406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.282+0000 7f9db6ffd700 1 -- 172.21.15.6:0/2901721577 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9dbc085e10 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.282+0000 7f9dc6215700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9da40682f0 0x7f9da406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:26.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.283+0000 7f9dc6215700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9da40682f0 0x7f9da406a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0157620 tx=0x7f9db0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:26.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.287+0000 7f9db6ffd700 1 -- 172.21.15.6:0/2901721577 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9dbc0524f0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.526+0000 7f9dc8479700 1 -- 172.21.15.6:0/2901721577 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9dc0071cc0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.526+0000 7f9db6ffd700 1 -- 172.21.15.6:0/2901721577 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9dbc0210c0 con 0x7f9dc00b4fb0 2023-11-28T13:05:26.527 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:26.528 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:26.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 -- 172.21.15.6:0/2901721577 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9da40682f0 msgr2=0x7f9da406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9da40682f0 0x7f9da406a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0157620 tx=0x7f9db0008030 comp rx=0 tx=0).stop 2023-11-28T13:05:26.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 -- 172.21.15.6:0/2901721577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9dc00b4fb0 msgr2=0x7f9dc0155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:26.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9dc00b4fb0 0x7f9dc0155f40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00ebe0 tx=0x7f9dbc009e20 comp rx=0 tx=0).stop 2023-11-28T13:05:26.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 -- 172.21.15.6:0/2901721577 shutdown_connections 2023-11-28T13:05:26.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9da40682f0 0x7f9da406a7a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9dc00ca7e0 0x7f9dc01f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9dc006f290 0x7f9dc01567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 --2- 172.21.15.6:0/2901721577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9dc00b4fb0 0x7f9dc0155f40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:26.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 -- 172.21.15.6:0/2901721577 >> 172.21.15.6:0/2901721577 conn(0x7f9dc00b3610 msgr2=0x7f9dc00b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:26.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 -- 172.21.15.6:0/2901721577 shutdown_connections 2023-11-28T13:05:26.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:26.528+0000 7f9dc8479700 1 -- 172.21.15.6:0/2901721577 wait complete. 2023-11-28T13:05:26.541 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:27.175 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:05:27.178 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:05:31.543 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:31.680 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.679+0000 7ffa74194700 1 Processor -- start 2023-11-28T13:05:31.681 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.679+0000 7ffa74194700 1 -- start start 2023-11-28T13:05:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.680+0000 7ffa74194700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c15c8a0 0x7ffa6c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.680+0000 7ffa74194700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ffa6c0b2290 0x7ffa6c0ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.680+0000 7ffa74194700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6c0d0280 con 0x7ffa6c06f120 2023-11-28T13:05:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.680+0000 7ffa74194700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6c0d0560 con 0x7ffa6c0b2290 2023-11-28T13:05:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.680+0000 7ffa74194700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0d0840 con 0x7ffa6c15c8a0 2023-11-28T13:05:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.680+0000 7ffa72731700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ffa6c06f120 0x7ffa6c0cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50422/0 (socket says 172.21.15.6:50422) 2023-11-28T13:05:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.680+0000 7ffa72731700 1 -- 172.21.15.6:0/3593906814 learned_addr learned my addr 172.21.15.6:0/3593906814 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:05:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa71f30700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c15c8a0 0x7ffa6c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa7172f700 1 --2- 172.21.15.6:0/3593906814 >> v2:172.21.15.61:3300/0 conn(0x7ffa6c0b2290 0x7ffa6c0ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.684 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389346743 0 0) 0x7ffa6c0d0280 con 0x7ffa6c06f120 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa580035b0 con 0x7ffa6c06f120 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2996129301 0 0) 0x7ffa580035b0 con 0x7ffa6c06f120 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c15c8a0 msgr2=0x7ffa6c0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c15c8a0 0x7ffa6c0b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 >> v2:172.21.15.61:3300/0 conn(0x7ffa6c0b2290 msgr2=0x7ffa6c0ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 --2- 172.21.15.6:0/3593906814 >> v2:172.21.15.61:3300/0 conn(0x7ffa6c0b2290 0x7ffa6c0ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c0b1550 con 0x7ffa6c06f120 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7ffa640019f0 con 0x7ffa6c06f120 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 >> v1:172.21.15.6:6789/0 conn(0x7ffa6c06f120 legacy=0x7ffa6c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c15c8a0 0x7ffa58003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa58003f80 0x7ffa58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.685 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa58004d90 0x7ffa58005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0d0560 con 0x7ffa58003f80 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0d0840 con 0x7ffa58004d90 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5800ba10 con 0x7ffa6c15c8a0 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa7172f700 1 --2- 172.21.15.6:0/3593906814 >> v2:172.21.15.61:3300/0 conn(0x7ffa6c0b2290 0x7ffa6c0ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa71f30700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c15c8a0 0x7ffa58003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa7172f700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa58003f80 0x7ffa58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa72731700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa58004d90 0x7ffa58005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa7172f700 1 -- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c15c8a0 msgr2=0x7ffa58003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.681+0000 7ffa7172f700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c15c8a0 0x7ffa58003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa7172f700 1 -- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa58004d90 msgr2=0x7ffa58005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa7172f700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa58004d90 0x7ffa58005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa7172f700 1 -- 172.21.15.6:0/3593906814 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580035b0 con 0x7ffa58003f80 2023-11-28T13:05:31.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa7172f700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa58003f80 0x7ffa58004850 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009d10 tx=0x7ffa58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7aaeb880e0a56d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68009e60 con 0x7ffa58003f80 2023-11-28T13:05:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6800e9a0 con 0x7ffa58003f80 2023-11-28T13:05:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa70f2e700 1 -- 172.21.15.6:0/3593906814 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800ed10 con 0x7ffa58003f80 2023-11-28T13:05:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa74194700 1 -- 172.21.15.6:0/3593906814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa58003f80 msgr2=0x7ffa58004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa74194700 1 --2- 172.21.15.6:0/3593906814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa58003f80 0x7ffa58004850 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009d10 tx=0x7ffa58005880 comp rx=0 tx=0).stop 2023-11-28T13:05:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa72731700 1 -- 172.21.15.6:0/3593906814 reap_dead start 2023-11-28T13:05:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa74194700 1 -- 172.21.15.6:0/3593906814 shutdown_connections 2023-11-28T13:05:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.682+0000 7ffa74194700 1 -- 172.21.15.6:0/3593906814 >> 172.21.15.6:0/3593906814 conn(0x7ffa6c0b3610 msgr2=0x7ffa6c0cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.683+0000 7ffa74194700 1 -- 172.21.15.6:0/3593906814 shutdown_connections 2023-11-28T13:05:31.687 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.683+0000 7ffa74194700 1 -- 172.21.15.6:0/3593906814 wait complete. 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.684+0000 7ffa74194700 1 Processor -- start 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.684+0000 7ffa74194700 1 -- start start 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa74194700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa58003f80 0x7ffa6c1e6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa74194700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa6c0c7bd0 0x7ffa6c158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa74194700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c1e2630 0x7ffa6c1e2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa74194700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c1e2ff0 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa74194700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0caf00 con 0x7ffa58003f80 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa74194700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0c86a0 con 0x7ffa6c1e2630 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa7172f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa6c0c7bd0 0x7ffa6c158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa7172f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa6c0c7bd0 0x7ffa6c158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34880/0 (socket says 172.21.15.6:34880) 2023-11-28T13:05:31.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa7172f700 1 -- 172.21.15.6:0/1761027068 learned_addr learned my addr 172.21.15.6:0/1761027068 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa71f30700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa58003f80 0x7ffa6c1e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa72731700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c1e2630 0x7ffa6c1e2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa7172f700 1 -- 172.21.15.6:0/1761027068 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c1e2630 msgr2=0x7ffa6c1e2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa7172f700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c1e2630 0x7ffa6c1e2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa7172f700 1 -- 172.21.15.6:0/1761027068 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa58003f80 msgr2=0x7ffa6c1e6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa7172f700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa58003f80 0x7ffa6c1e6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa7172f700 1 -- 172.21.15.6:0/1761027068 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6c15cc70 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa7172f700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa6c0c7bd0 0x7ffa6c158fe0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ffa580046d0 tx=0x7ffa680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa627fc700 1 -- 172.21.15.6:0/1761027068 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68020070 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa627fc700 1 -- 172.21.15.6:0/1761027068 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa68009e60 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa74194700 1 -- 172.21.15.6:0/1761027068 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c0b1830 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.685+0000 7ffa627fc700 1 -- 172.21.15.6:0/1761027068 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68003ad0 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.686+0000 7ffa74194700 1 -- 172.21.15.6:0/1761027068 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6c06a6c0 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.686+0000 7ffa74194700 1 -- 172.21.15.6:0/1761027068 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6c066c60 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.687+0000 7ffa627fc700 1 -- 172.21.15.6:0/1761027068 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffa6802c030 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.687+0000 7ffa627fc700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffa500682f0 0x7ffa5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:31.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.688+0000 7ffa71f30700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffa500682f0 0x7ffa5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:31.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.688+0000 7ffa627fc700 1 -- 172.21.15.6:0/1761027068 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ffa68087150 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.692 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.688+0000 7ffa71f30700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffa500682f0 0x7ffa5006a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009b80 tx=0x7ffa5c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:31.693 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.692+0000 7ffa627fc700 1 -- 172.21.15.6:0/1761027068 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffa680538e0 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.933+0000 7ffa74194700 1 -- 172.21.15.6:0/1761027068 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffa6c0675a0 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.934+0000 7ffa627fc700 1 -- 172.21.15.6:0/1761027068 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ffa68056f00 con 0x7ffa6c0c7bd0 2023-11-28T13:05:31.935 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:31.935 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 -- 172.21.15.6:0/1761027068 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffa500682f0 msgr2=0x7ffa5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffa500682f0 0x7ffa5006a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009b80 tx=0x7ffa5c009210 comp rx=0 tx=0).stop 2023-11-28T13:05:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 -- 172.21.15.6:0/1761027068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa6c0c7bd0 msgr2=0x7ffa6c158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:31.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa6c0c7bd0 0x7ffa6c158fe0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ffa580046d0 tx=0x7ffa680047c0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 -- 172.21.15.6:0/1761027068 shutdown_connections 2023-11-28T13:05:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ffa500682f0 0x7ffa5006a7a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ffa6c1e2630 0x7ffa6c1e2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffa58003f80 0x7ffa6c1e6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 --2- 172.21.15.6:0/1761027068 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa6c0c7bd0 0x7ffa6c158fe0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 -- 172.21.15.6:0/1761027068 >> 172.21.15.6:0/1761027068 conn(0x7ffa6c0b3610 msgr2=0x7ffa6c0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 -- 172.21.15.6:0/1761027068 shutdown_connections 2023-11-28T13:05:31.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:31.936+0000 7ffa74194700 1 -- 172.21.15.6:0/1761027068 wait complete. 2023-11-28T13:05:31.949 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:36.950 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:37.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.087+0000 7f5152b7c700 1 Processor -- start 2023-11-28T13:05:37.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.088+0000 7f5152b7c700 1 -- start start 2023-11-28T13:05:37.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.088+0000 7f5152b7c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f514c0ccfe0 0x7f514c070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.088+0000 7f5152b7c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f514c070a60 0x7f514c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.088+0000 7f5152b7c700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c0ce9b0 con 0x7f514c0b6510 2023-11-28T13:05:37.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.088+0000 7f5152b7c700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514c070f00 con 0x7f514c070a60 2023-11-28T13:05:37.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.088+0000 7f5152b7c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0b0d30 con 0x7f514c0ccfe0 2023-11-28T13:05:37.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f5151119700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f514c070a60 0x7f514c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f5150918700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f514c0b6510 0x7f514c0cb220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50438/0 (socket says 172.21.15.6:50438) 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f5150918700 1 -- 172.21.15.6:0/811261527 learned_addr learned my addr 172.21.15.6:0/811261527 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f514bfff700 1 --2- 172.21.15.6:0/811261527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f514c0ccfe0 0x7f514c070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f514b7fe700 1 -- 172.21.15.6:0/811261527 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9028757 0 0) 0x7f514c0ce9b0 con 0x7f514c0b6510 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f514b7fe700 1 -- 172.21.15.6:0/811261527 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f513c0035b0 con 0x7f514c0b6510 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f5151119700 1 -- 172.21.15.6:0/811261527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f514c0ccfe0 msgr2=0x7f514c070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f5151119700 1 --2- 172.21.15.6:0/811261527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f514c0ccfe0 0x7f514c070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f5151119700 1 -- 172.21.15.6:0/811261527 >> v1:172.21.15.6:6789/0 conn(0x7f514c0b6510 legacy=0x7f514c0cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f5151119700 1 -- 172.21.15.6:0/811261527 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0b0fd0 con 0x7f514c070a60 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f5151119700 1 --2- 172.21.15.6:0/811261527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c070a60 0x7f514c0cee40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5144009d90 tx=0x7f514400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1045c531e813e8cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f514b7fe700 1 -- 172.21.15.6:0/811261527 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514400e070 con 0x7f514c070a60 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f514b7fe700 1 -- 172.21.15.6:0/811261527 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514400bb20 con 0x7f514c070a60 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.089+0000 7f514b7fe700 1 -- 172.21.15.6:0/811261527 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144012430 con 0x7f514c070a60 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.090+0000 7f5152b7c700 1 -- 172.21.15.6:0/811261527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c070a60 msgr2=0x7f514c0cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.090+0000 7f5152b7c700 1 --2- 172.21.15.6:0/811261527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c070a60 0x7f514c0cee40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5144009d90 tx=0x7f514400b330 comp rx=0 tx=0).stop 2023-11-28T13:05:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.090+0000 7f5152b7c700 1 -- 172.21.15.6:0/811261527 shutdown_connections 2023-11-28T13:05:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.090+0000 7f5152b7c700 1 --2- 172.21.15.6:0/811261527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c070a60 0x7f514c0cee40 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.090+0000 7f5152b7c700 1 --2- 172.21.15.6:0/811261527 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f514c0ccfe0 0x7f514c070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.090+0000 7f5152b7c700 1 -- 172.21.15.6:0/811261527 >> 172.21.15.6:0/811261527 conn(0x7f514c0d0bd0 msgr2=0x7f514c0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.090+0000 7f5152b7c700 1 -- 172.21.15.6:0/811261527 shutdown_connections 2023-11-28T13:05:37.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.090+0000 7f5152b7c700 1 -- 172.21.15.6:0/811261527 wait complete. 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.091+0000 7f5152b7c700 1 Processor -- start 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.091+0000 7f5152b7c700 1 -- start start 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f5152b7c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f514c070a60 0x7f514c0af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f5152b7c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c0b6510 0x7f514c0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f5152b7c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f514c0ccfe0 0x7f514c0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f5152b7c700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0af5d0 con 0x7f514c0b6510 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f5152b7c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c159620 con 0x7f514c0ccfe0 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f5152b7c700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0cac80 con 0x7f514c070a60 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f514bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c0b6510 0x7f514c0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f514bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c0b6510 0x7f514c0ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34894/0 (socket says 172.21.15.6:34894) 2023-11-28T13:05:37.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f514bfff700 1 -- 172.21.15.6:0/2012656963 learned_addr learned my addr 172.21.15.6:0/2012656963 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f5150918700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f514c070a60 0x7f514c0af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f5151119700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f514c0ccfe0 0x7f514c0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f514bfff700 1 -- 172.21.15.6:0/2012656963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f514c070a60 msgr2=0x7f514c0af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.092+0000 7f514bfff700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f514c070a60 0x7f514c0af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.093+0000 7f514bfff700 1 -- 172.21.15.6:0/2012656963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f514c0ccfe0 msgr2=0x7f514c0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.093+0000 7f514bfff700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f514c0ccfe0 0x7f514c0c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.093+0000 7f514bfff700 1 -- 172.21.15.6:0/2012656963 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5144009a70 con 0x7f514c0b6510 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.093+0000 7f5151119700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f514c0ccfe0 0x7f514c0c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.093+0000 7f514bfff700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c0b6510 0x7f514c0ca740 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f513800e870 tx=0x7f513800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.093+0000 7f51497fa700 1 -- 172.21.15.6:0/2012656963 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138009590 con 0x7f514c0b6510 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.093+0000 7f51497fa700 1 -- 172.21.15.6:0/2012656963 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5138013680 con 0x7f514c0b6510 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.093+0000 7f5152b7c700 1 -- 172.21.15.6:0/2012656963 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c0cf380 con 0x7f514c0b6510 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.093+0000 7f51497fa700 1 -- 172.21.15.6:0/2012656963 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138013a10 con 0x7f514c0b6510 2023-11-28T13:05:37.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.093+0000 7f5152b7c700 1 -- 172.21.15.6:0/2012656963 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c0b12b0 con 0x7f514c0b6510 2023-11-28T13:05:37.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.094+0000 7f5152b7c700 1 -- 172.21.15.6:0/2012656963 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c0cf510 con 0x7f514c0b6510 2023-11-28T13:05:37.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.094+0000 7f51497fa700 1 -- 172.21.15.6:0/2012656963 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5138019030 con 0x7f514c0b6510 2023-11-28T13:05:37.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.095+0000 7f51497fa700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f51300682f0 0x7f513006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:37.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.095+0000 7f51497fa700 1 -- 172.21.15.6:0/2012656963 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5138085b00 con 0x7f514c0b6510 2023-11-28T13:05:37.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.095+0000 7f5150918700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f51300682f0 0x7f513006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:37.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.095+0000 7f5150918700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f51300682f0 0x7f513006a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5140007fd0 tx=0x7f5140007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:37.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.099+0000 7f51497fa700 1 -- 172.21.15.6:0/2012656963 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f51380521e0 con 0x7f514c0b6510 2023-11-28T13:05:37.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.340+0000 7f5152b7c700 1 -- 172.21.15.6:0/2012656963 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f514c0675a0 con 0x7f514c0b6510 2023-11-28T13:05:37.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.340+0000 7f51497fa700 1 -- 172.21.15.6:0/2012656963 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f5138021080 con 0x7f514c0b6510 2023-11-28T13:05:37.342 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:37.342 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:37.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.342+0000 7f5152b7c700 1 -- 172.21.15.6:0/2012656963 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f51300682f0 msgr2=0x7f513006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.342+0000 7f5152b7c700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f51300682f0 0x7f513006a7a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5140007fd0 tx=0x7f5140007dc0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.343+0000 7f5152b7c700 1 -- 172.21.15.6:0/2012656963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c0b6510 msgr2=0x7f514c0ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:37.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.343+0000 7f5152b7c700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c0b6510 0x7f514c0ca740 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f513800e870 tx=0x7f513800eb80 comp rx=0 tx=0).stop 2023-11-28T13:05:37.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.343+0000 7f5152b7c700 1 -- 172.21.15.6:0/2012656963 shutdown_connections 2023-11-28T13:05:37.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.343+0000 7f5152b7c700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f514c0ccfe0 0x7f514c0c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.343+0000 7f5152b7c700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f51300682f0 0x7f513006a7a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.343+0000 7f5152b7c700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f514c070a60 0x7f514c0af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.343+0000 7f5152b7c700 1 --2- 172.21.15.6:0/2012656963 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f514c0b6510 0x7f514c0ca740 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:37.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.343+0000 7f5152b7c700 1 -- 172.21.15.6:0/2012656963 >> 172.21.15.6:0/2012656963 conn(0x7f514c0d0bd0 msgr2=0x7f514c06f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:37.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.343+0000 7f5152b7c700 1 -- 172.21.15.6:0/2012656963 shutdown_connections 2023-11-28T13:05:37.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:37.343+0000 7f5152b7c700 1 -- 172.21.15.6:0/2012656963 wait complete. 2023-11-28T13:05:37.356 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:42.356 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:42.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.494+0000 7fd05c80e700 1 Processor -- start 2023-11-28T13:05:42.495 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.494+0000 7fd05c80e700 1 -- start start 2023-11-28T13:05:42.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.494+0000 7fd05c80e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0580ca7e0 0x7fd0580cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.494+0000 7fd05c80e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 0x7fd0580b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.494+0000 7fd05c80e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0580cb0f0 con 0x7fd0580b4fb0 2023-11-28T13:05:42.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.494+0000 7fd05c80e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0580ca410 con 0x7fd0580ca7e0 2023-11-28T13:05:42.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.494+0000 7fd05c80e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0580ca130 con 0x7fd05806f290 2023-11-28T13:05:42.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05659c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0580ca7e0 0x7fd0580cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05659c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0580ca7e0 0x7fd0580cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34900/0 (socket says 172.21.15.6:34900) 2023-11-28T13:05:42.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05659c700 1 -- 172.21.15.6:0/2229055577 learned_addr learned my addr 172.21.15.6:0/2229055577 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:42.498 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd055d9b700 1 --2- 172.21.15.6:0/2229055577 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 0x7fd0580b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05659c700 1 -- 172.21.15.6:0/2229055577 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 msgr2=0x7fd0580b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05659c700 1 --2- 172.21.15.6:0/2229055577 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 0x7fd0580b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05659c700 1 -- 172.21.15.6:0/2229055577 >> v1:172.21.15.61:6789/0 conn(0x7fd0580b4fb0 legacy=0x7fd0580aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05659c700 1 -- 172.21.15.6:0/2229055577 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0580b2100 con 0x7fd0580ca7e0 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd055d9b700 1 --2- 172.21.15.6:0/2229055577 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 0x7fd0580b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05659c700 1 --2- 172.21.15.6:0/2229055577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 0x7fd0580cabb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fd0480098f0 tx=0x7fd048009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d41f68416392a98 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05559a700 1 -- 172.21.15.6:0/2229055577 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048004230 con 0x7fd0580ca7e0 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05559a700 1 -- 172.21.15.6:0/2229055577 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd048004390 con 0x7fd0580ca7e0 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.495+0000 7fd05559a700 1 -- 172.21.15.6:0/2229055577 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0480036a0 con 0x7fd0580ca7e0 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.496+0000 7fd05c80e700 1 -- 172.21.15.6:0/2229055577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 msgr2=0x7fd0580cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.496+0000 7fd05c80e700 1 --2- 172.21.15.6:0/2229055577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 0x7fd0580cabb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fd0480098f0 tx=0x7fd048009c00 comp rx=0 tx=0).stop 2023-11-28T13:05:42.499 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.496+0000 7fd05c80e700 1 -- 172.21.15.6:0/2229055577 shutdown_connections 2023-11-28T13:05:42.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.496+0000 7fd05c80e700 1 --2- 172.21.15.6:0/2229055577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 0x7fd0580cabb0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.496+0000 7fd05c80e700 1 --2- 172.21.15.6:0/2229055577 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 0x7fd0580b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.496+0000 7fd05c80e700 1 -- 172.21.15.6:0/2229055577 >> 172.21.15.6:0/2229055577 conn(0x7fd0580b3610 msgr2=0x7fd0580cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:42.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.496+0000 7fd05c80e700 1 -- 172.21.15.6:0/2229055577 shutdown_connections 2023-11-28T13:05:42.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.496+0000 7fd05c80e700 1 -- 172.21.15.6:0/2229055577 wait complete. 2023-11-28T13:05:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.497+0000 7fd05c80e700 1 Processor -- start 2023-11-28T13:05:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.497+0000 7fd05c80e700 1 -- start start 2023-11-28T13:05:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.498+0000 7fd05c80e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 0x7fd0581e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.498+0000 7fd05c80e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0580b4fb0 0x7fd0580c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.498+0000 7fd05c80e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 0x7fd0581e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.498+0000 7fd05c80e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0580c7650 con 0x7fd0580ca7e0 2023-11-28T13:05:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.498+0000 7fd05c80e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0580c7040 con 0x7fd0580b4fb0 2023-11-28T13:05:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.498+0000 7fd05c80e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0581e5820 con 0x7fd05806f290 2023-11-28T13:05:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.498+0000 7fd056d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 0x7fd0581e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.498+0000 7fd056d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 0x7fd0581e2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34910/0 (socket says 172.21.15.6:34910) 2023-11-28T13:05:42.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.498+0000 7fd056d9d700 1 -- 172.21.15.6:0/1919136294 learned_addr learned my addr 172.21.15.6:0/1919136294 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:42.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd05659c700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 0x7fd0581e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd055d9b700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0580b4fb0 0x7fd0580c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd056d9d700 1 -- 172.21.15.6:0/1919136294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 msgr2=0x7fd0581e51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd056d9d700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 0x7fd0581e51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd056d9d700 1 -- 172.21.15.6:0/1919136294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0580b4fb0 msgr2=0x7fd0580c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd056d9d700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0580b4fb0 0x7fd0580c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd056d9d700 1 -- 172.21.15.6:0/1919136294 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd048009550 con 0x7fd0580ca7e0 2023-11-28T13:05:42.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd056d9d700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 0x7fd0581e2c50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd05000cc10 tx=0x7fd05000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:42.505 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd04effd700 1 -- 172.21.15.6:0/1919136294 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05000abb0 con 0x7fd0580ca7e0 2023-11-28T13:05:42.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd04effd700 1 -- 172.21.15.6:0/1919136294 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05000ad10 con 0x7fd0580ca7e0 2023-11-28T13:05:42.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd05c80e700 1 -- 172.21.15.6:0/1919136294 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0580d0280 con 0x7fd0580ca7e0 2023-11-28T13:05:42.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd04effd700 1 -- 172.21.15.6:0/1919136294 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050012660 con 0x7fd0580ca7e0 2023-11-28T13:05:42.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.499+0000 7fd05c80e700 1 -- 172.21.15.6:0/1919136294 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0580b23e0 con 0x7fd0580ca7e0 2023-11-28T13:05:42.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.500+0000 7fd05c80e700 1 -- 172.21.15.6:0/1919136294 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd058071cc0 con 0x7fd0580ca7e0 2023-11-28T13:05:42.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.500+0000 7fd04effd700 1 -- 172.21.15.6:0/1919136294 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd050022070 con 0x7fd0580ca7e0 2023-11-28T13:05:42.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.501+0000 7fd04effd700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0380682f0 0x7fd03806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:42.506 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.501+0000 7fd04effd700 1 -- 172.21.15.6:0/1919136294 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fd050086530 con 0x7fd0580ca7e0 2023-11-28T13:05:42.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.501+0000 7fd05659c700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0380682f0 0x7fd03806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:42.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.501+0000 7fd05659c700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0380682f0 0x7fd03806a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd048009840 tx=0x7fd048008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:42.507 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.506+0000 7fd04effd700 1 -- 172.21.15.6:0/1919136294 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd050052bb0 con 0x7fd0580ca7e0 2023-11-28T13:05:42.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.744+0000 7fd05c80e700 1 -- 172.21.15.6:0/1919136294 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd058066c60 con 0x7fd0580ca7e0 2023-11-28T13:05:42.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.744+0000 7fd04effd700 1 -- 172.21.15.6:0/1919136294 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fd0500561d0 con 0x7fd0580ca7e0 2023-11-28T13:05:42.746 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:42.746 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:42.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 -- 172.21.15.6:0/1919136294 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0380682f0 msgr2=0x7fd03806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0380682f0 0x7fd03806a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd048009840 tx=0x7fd048008030 comp rx=0 tx=0).stop 2023-11-28T13:05:42.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 -- 172.21.15.6:0/1919136294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 msgr2=0x7fd0581e2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:42.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 0x7fd0581e2c50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd05000cc10 tx=0x7fd05000a060 comp rx=0 tx=0).stop 2023-11-28T13:05:42.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 -- 172.21.15.6:0/1919136294 shutdown_connections 2023-11-28T13:05:42.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0580ca7e0 0x7fd0581e2c50 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd0380682f0 0x7fd03806a7a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd05806f290 0x7fd0581e51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 --2- 172.21.15.6:0/1919136294 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd0580b4fb0 0x7fd0580c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:42.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 -- 172.21.15.6:0/1919136294 >> 172.21.15.6:0/1919136294 conn(0x7fd0580b3610 msgr2=0x7fd0580b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:42.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 -- 172.21.15.6:0/1919136294 shutdown_connections 2023-11-28T13:05:42.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:42.746+0000 7fd05c80e700 1 -- 172.21.15.6:0/1919136294 wait complete. 2023-11-28T13:05:42.759 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:47.760 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:47.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.898+0000 7f8048385700 1 Processor -- start 2023-11-28T13:05:47.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.898+0000 7f8048385700 1 -- start start 2023-11-28T13:05:47.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f8048385700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80400cb220 0x7f80400b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:47.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f8048385700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f80400cb6c0 0x7f80400b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f8048385700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804015dbd0 con 0x7f8040070520 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f8048385700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80400b1510 con 0x7f80400cb6c0 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f8048385700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804015d610 con 0x7f80400cb220 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f8046121700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f8040070520 0x7f80400708f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50466/0 (socket says 172.21.15.6:50466) 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f8046121700 1 -- 172.21.15.6:0/1825924477 learned_addr learned my addr 172.21.15.6:0/1825924477 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f8046922700 1 --2- 172.21.15.6:0/1825924477 >> v2:172.21.15.61:3300/0 conn(0x7f80400cb6c0 0x7f80400b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749168749 0 0) 0x7f804015dbd0 con 0x7f8040070520 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c0035b0 con 0x7f8040070520 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f8045920700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80400cb220 0x7f80400b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2480149402 0 0) 0x7f802c0035b0 con 0x7f8040070520 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80400cb220 msgr2=0x7f80400b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:47.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f804511f700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80400cb220 0x7f80400b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:47.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 >> v2:172.21.15.61:3300/0 conn(0x7f80400cb6c0 msgr2=0x7f80400b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:47.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.899+0000 7f804511f700 1 --2- 172.21.15.6:0/1825924477 >> v2:172.21.15.61:3300/0 conn(0x7f80400cb6c0 0x7f80400b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:47.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804015d8f0 con 0x7f8040070520 2023-11-28T13:05:47.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f803c001670 con 0x7f8040070520 2023-11-28T13:05:47.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 >> v1:172.21.15.6:6789/0 conn(0x7f8040070520 legacy=0x7f80400708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:47.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f802c003a00 0x7f802c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:47.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f802c004850 0x7f802c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:47.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80400cb220 0x7f802c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:47.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80400b1510 con 0x7f802c004850 2023-11-28T13:05:47.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804015d610 con 0x7f802c003a00 2023-11-28T13:05:47.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c00ba60 con 0x7f80400cb220 2023-11-28T13:05:47.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f8045920700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f802c004850 0x7f802c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:47.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f8046121700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f802c003a00 0x7f802c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:47.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f8046922700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80400cb220 0x7f802c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:47.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f8045920700 1 -- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80400cb220 msgr2=0x7f802c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:47.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f8045920700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80400cb220 0x7f802c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f8045920700 1 -- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f802c003a00 msgr2=0x7f802c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f8045920700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f802c003a00 0x7f802c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f8045920700 1 -- 172.21.15.6:0/1825924477 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0035b0 con 0x7f802c004850 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f8045920700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f802c004850 0x7f802c004cc0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8030009b90 tx=0x7f802c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440b74b208081dd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803000e6c0 con 0x7f802c004850 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803000e820 con 0x7f802c004850 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.900+0000 7f804511f700 1 -- 172.21.15.6:0/1825924477 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803000eb90 con 0x7f802c004850 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.901+0000 7f8048385700 1 -- 172.21.15.6:0/1825924477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f802c004850 msgr2=0x7f802c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.901+0000 7f8048385700 1 --2- 172.21.15.6:0/1825924477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f802c004850 0x7f802c004cc0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8030009b90 tx=0x7f802c005880 comp rx=0 tx=0).stop 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.901+0000 7f8046922700 1 -- 172.21.15.6:0/1825924477 reap_dead start 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.901+0000 7f8048385700 1 -- 172.21.15.6:0/1825924477 shutdown_connections 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.901+0000 7f8048385700 1 -- 172.21.15.6:0/1825924477 >> 172.21.15.6:0/1825924477 conn(0x7f80400d0bd0 msgr2=0x7f80400d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:47.907 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.901+0000 7f8048385700 1 -- 172.21.15.6:0/1825924477 shutdown_connections 2023-11-28T13:05:47.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.901+0000 7f8048385700 1 -- 172.21.15.6:0/1825924477 wait complete. 2023-11-28T13:05:47.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.903+0000 7f8048385700 1 Processor -- start 2023-11-28T13:05:47.908 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.903+0000 7f8048385700 1 -- start start 2023-11-28T13:05:47.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8048385700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f802c004850 0x7f804015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:47.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8048385700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80401e5c90 0x7f80401e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:47.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8048385700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80400c9700 0x7f80400c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:47.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8048385700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401e6640 con 0x7f80400c9700 2023-11-28T13:05:47.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8048385700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401453b0 con 0x7f802c004850 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8048385700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80400ca6d0 con 0x7f80401e5c90 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8046922700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80400c9700 0x7f80400c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8046922700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80400c9700 0x7f80400c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34930/0 (socket says 172.21.15.6:34930) 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8046922700 1 -- 172.21.15.6:0/2884159360 learned_addr learned my addr 172.21.15.6:0/2884159360 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8045920700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80401e5c90 0x7f80401e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8046121700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f802c004850 0x7f804015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8046922700 1 -- 172.21.15.6:0/2884159360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80401e5c90 msgr2=0x7f80401e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8046922700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80401e5c90 0x7f80401e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8046922700 1 -- 172.21.15.6:0/2884159360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f802c004850 msgr2=0x7f804015bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8046922700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f802c004850 0x7f804015bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8046922700 1 -- 172.21.15.6:0/2884159360 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80400b0570 con 0x7f80400c9700 2023-11-28T13:05:47.911 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.904+0000 7f8046922700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80400c9700 0x7f80400c9b70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f80400d0b30 tx=0x7f8038011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.905+0000 7f8036ffd700 1 -- 172.21.15.6:0/2884159360 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800e040 con 0x7f80400c9700 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.905+0000 7f8036ffd700 1 -- 172.21.15.6:0/2884159360 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803800f040 con 0x7f80400c9700 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.905+0000 7f8048385700 1 -- 172.21.15.6:0/2884159360 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80400b0290 con 0x7f80400c9700 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.905+0000 7f8036ffd700 1 -- 172.21.15.6:0/2884159360 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803800fc40 con 0x7f80400c9700 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.905+0000 7f8048385700 1 -- 172.21.15.6:0/2884159360 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80400b2360 con 0x7f80400c9700 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.906+0000 7f8048385700 1 -- 172.21.15.6:0/2884159360 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8040145560 con 0x7f80400c9700 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.906+0000 7f8036ffd700 1 -- 172.21.15.6:0/2884159360 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f803800fda0 con 0x7f80400c9700 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.906+0000 7f8036ffd700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80240682f0 0x7f802406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.906+0000 7f8036ffd700 1 -- 172.21.15.6:0/2884159360 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f80380867c0 con 0x7f80400c9700 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.907+0000 7f8046121700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80240682f0 0x7f802406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:47.912 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.907+0000 7f8046121700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80240682f0 0x7f802406a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f803c000fb0 tx=0x7f803c001e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:47.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:47.911+0000 7f8036ffd700 1 -- 172.21.15.6:0/2884159360 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8040145560 con 0x7f80400c9700 2023-11-28T13:05:48.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.150+0000 7f8048385700 1 -- 172.21.15.6:0/2884159360 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8040145560 con 0x7f80400c9700 2023-11-28T13:05:48.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.151+0000 7f8036ffd700 1 -- 172.21.15.6:0/2884159360 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8040145560 con 0x7f80400c9700 2023-11-28T13:05:48.151 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:48.151 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:48.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 -- 172.21.15.6:0/2884159360 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80240682f0 msgr2=0x7f802406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:48.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80240682f0 0x7f802406a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f803c000fb0 tx=0x7f803c001e00 comp rx=0 tx=0).stop 2023-11-28T13:05:48.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 -- 172.21.15.6:0/2884159360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80400c9700 msgr2=0x7f80400c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:48.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80400c9700 0x7f80400c9b70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f80400d0b30 tx=0x7f8038011ed0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 -- 172.21.15.6:0/2884159360 shutdown_connections 2023-11-28T13:05:48.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80400c9700 0x7f80400c9b70 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f802c004850 0x7f804015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f80240682f0 0x7f802406a7a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 --2- 172.21.15.6:0/2884159360 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f80401e5c90 0x7f80401e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:48.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 -- 172.21.15.6:0/2884159360 >> 172.21.15.6:0/2884159360 conn(0x7f80400d0bd0 msgr2=0x7f80400cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:48.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 -- 172.21.15.6:0/2884159360 shutdown_connections 2023-11-28T13:05:48.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:48.153+0000 7f8048385700 1 -- 172.21.15.6:0/2884159360 wait complete. 2023-11-28T13:05:48.165 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:53.166 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:53.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.303+0000 7f859c426700 1 Processor -- start 2023-11-28T13:05:53.305 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.303+0000 7f859c426700 1 -- start start 2023-11-28T13:05:53.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.304+0000 7f859c426700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940b49d0 0x7f85940d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.307 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.304+0000 7f859c426700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85940cd5a0 0x7f85940cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.304+0000 7f859c426700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85940b4fb0 con 0x7f859415c6a0 2023-11-28T13:05:53.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.304+0000 7f859c426700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85940b5570 con 0x7f85940cd5a0 2023-11-28T13:05:53.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.304+0000 7f859c426700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85940cdf60 con 0x7f85940b49d0 2023-11-28T13:05:53.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.304+0000 7f859a9c3700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85940cd5a0 0x7f85940cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.304+0000 7f859a9c3700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85940cd5a0 0x7f85940cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34938/0 (socket says 172.21.15.6:34938) 2023-11-28T13:05:53.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.304+0000 7f859a9c3700 1 -- 172.21.15.6:0/4245864180 learned_addr learned my addr 172.21.15.6:0/4245864180 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:53.308 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.304+0000 7f85999c1700 1 --2- 172.21.15.6:0/4245864180 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940b49d0 0x7f85940d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.305+0000 7f859a9c3700 1 -- 172.21.15.6:0/4245864180 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940b49d0 msgr2=0x7f85940d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:53.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.305+0000 7f859a9c3700 1 --2- 172.21.15.6:0/4245864180 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940b49d0 0x7f85940d19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.305+0000 7f859a9c3700 1 -- 172.21.15.6:0/4245864180 >> v1:172.21.15.61:6789/0 conn(0x7f859415c6a0 legacy=0x7f859406abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.305+0000 7f859a9c3700 1 -- 172.21.15.6:0/4245864180 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85940b5290 con 0x7f85940cd5a0 2023-11-28T13:05:53.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.305+0000 7f85999c1700 1 --2- 172.21.15.6:0/4245864180 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940b49d0 0x7f85940d19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:05:53.309 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.305+0000 7f859a9c3700 1 --2- 172.21.15.6:0/4245864180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85940cd5a0 0x7f85940cda20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f858c009e50 tx=0x7f858c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f818e53676684 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.305+0000 7f85991c0700 1 -- 172.21.15.6:0/4245864180 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c00e070 con 0x7f85940cd5a0 2023-11-28T13:05:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.305+0000 7f85991c0700 1 -- 172.21.15.6:0/4245864180 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c00b9c0 con 0x7f85940cd5a0 2023-11-28T13:05:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.305+0000 7f85991c0700 1 -- 172.21.15.6:0/4245864180 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c00bd70 con 0x7f85940cd5a0 2023-11-28T13:05:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.306+0000 7f859c426700 1 -- 172.21.15.6:0/4245864180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85940cd5a0 msgr2=0x7f85940cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.306+0000 7f859c426700 1 --2- 172.21.15.6:0/4245864180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85940cd5a0 0x7f85940cda20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f858c009e50 tx=0x7f858c00b330 comp rx=0 tx=0).stop 2023-11-28T13:05:53.310 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.306+0000 7f859c426700 1 -- 172.21.15.6:0/4245864180 shutdown_connections 2023-11-28T13:05:53.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.306+0000 7f859c426700 1 --2- 172.21.15.6:0/4245864180 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85940cd5a0 0x7f85940cda20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.306+0000 7f859c426700 1 --2- 172.21.15.6:0/4245864180 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940b49d0 0x7f85940d19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.306+0000 7f859c426700 1 -- 172.21.15.6:0/4245864180 >> 172.21.15.6:0/4245864180 conn(0x7f85940d2f50 msgr2=0x7f85940d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:53.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.306+0000 7f859c426700 1 -- 172.21.15.6:0/4245864180 shutdown_connections 2023-11-28T13:05:53.311 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.306+0000 7f859c426700 1 -- 172.21.15.6:0/4245864180 wait complete. 2023-11-28T13:05:53.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.307+0000 7f859c426700 1 Processor -- start 2023-11-28T13:05:53.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.307+0000 7f859c426700 1 -- start start 2023-11-28T13:05:53.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859c426700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85940b49d0 0x7f85940c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.317 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859c426700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940cd5a0 0x7f85940cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859c426700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f859415c6a0 0x7f8594144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859c426700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85940cc520 con 0x7f859415c6a0 2023-11-28T13:05:53.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859c426700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85940cb950 con 0x7f85940b49d0 2023-11-28T13:05:53.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859c426700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85940c9290 con 0x7f85940cd5a0 2023-11-28T13:05:53.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859a9c3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f859415c6a0 0x7f8594144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.318 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859a9c3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f859415c6a0 0x7f8594144c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34944/0 (socket says 172.21.15.6:34944) 2023-11-28T13:05:53.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859a9c3700 1 -- 172.21.15.6:0/3167143769 learned_addr learned my addr 172.21.15.6:0/3167143769 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:53.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859a1c2700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85940b49d0 0x7f85940c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f85999c1700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940cd5a0 0x7f85940cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859a9c3700 1 -- 172.21.15.6:0/3167143769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940cd5a0 msgr2=0x7f85940cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.319 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859a9c3700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940cd5a0 0x7f85940cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859a9c3700 1 -- 172.21.15.6:0/3167143769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85940b49d0 msgr2=0x7f85940c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859a9c3700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85940b49d0 0x7f85940c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859a9c3700 1 -- 172.21.15.6:0/3167143769 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c009b30 con 0x7f859415c6a0 2023-11-28T13:05:53.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.308+0000 7f859a9c3700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f859415c6a0 0x7f8594144c30 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f859408ae10 tx=0x7f858c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:53.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.309+0000 7f8586ffd700 1 -- 172.21.15.6:0/3167143769 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c004800 con 0x7f859415c6a0 2023-11-28T13:05:53.320 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.309+0000 7f8586ffd700 1 -- 172.21.15.6:0/3167143769 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c004960 con 0x7f859415c6a0 2023-11-28T13:05:53.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.309+0000 7f859c426700 1 -- 172.21.15.6:0/3167143769 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85940d14b0 con 0x7f859415c6a0 2023-11-28T13:05:53.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.309+0000 7f8586ffd700 1 -- 172.21.15.6:0/3167143769 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c01c680 con 0x7f859415c6a0 2023-11-28T13:05:53.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.309+0000 7f859c426700 1 -- 172.21.15.6:0/3167143769 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85940d11d0 con 0x7f859415c6a0 2023-11-28T13:05:53.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.310+0000 7f859c426700 1 -- 172.21.15.6:0/3167143769 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859415ca70 con 0x7f859415c6a0 2023-11-28T13:05:53.321 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.310+0000 7f8586ffd700 1 -- 172.21.15.6:0/3167143769 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f858c02c070 con 0x7f859415c6a0 2023-11-28T13:05:53.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.310+0000 7f8586ffd700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85780682f0 0x7f857806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:53.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.311+0000 7f8586ffd700 1 -- 172.21.15.6:0/3167143769 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f858c087880 con 0x7f859415c6a0 2023-11-28T13:05:53.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.311+0000 7f859a1c2700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85780682f0 0x7f857806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:53.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.311+0000 7f859a1c2700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85780682f0 0x7f857806a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f859415b120 tx=0x7f8588009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:53.322 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.315+0000 7f8586ffd700 1 -- 172.21.15.6:0/3167143769 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f858c053f60 con 0x7f859415c6a0 2023-11-28T13:05:53.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.557+0000 7f859c426700 1 -- 172.21.15.6:0/3167143769 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8594068570 con 0x7f859415c6a0 2023-11-28T13:05:53.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.557+0000 7f8586ffd700 1 -- 172.21.15.6:0/3167143769 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f858c057580 con 0x7f859415c6a0 2023-11-28T13:05:53.559 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:53.559 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:53.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 -- 172.21.15.6:0/3167143769 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85780682f0 msgr2=0x7f857806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85780682f0 0x7f857806a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f859415b120 tx=0x7f8588009040 comp rx=0 tx=0).stop 2023-11-28T13:05:53.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 -- 172.21.15.6:0/3167143769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f859415c6a0 msgr2=0x7f8594144c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:53.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f859415c6a0 0x7f8594144c30 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f859408ae10 tx=0x7f858c004ad0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 -- 172.21.15.6:0/3167143769 shutdown_connections 2023-11-28T13:05:53.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f859415c6a0 0x7f8594144c30 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85940b49d0 0x7f85940c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85780682f0 0x7f857806a7a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 --2- 172.21.15.6:0/3167143769 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85940cd5a0 0x7f85940cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:53.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 -- 172.21.15.6:0/3167143769 >> 172.21.15.6:0/3167143769 conn(0x7f85940d2f50 msgr2=0x7f859406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:53.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 -- 172.21.15.6:0/3167143769 shutdown_connections 2023-11-28T13:05:53.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:53.559+0000 7f859c426700 1 -- 172.21.15.6:0/3167143769 wait complete. 2023-11-28T13:05:53.573 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:05:57.240 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:05:57.243 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:05:58.573 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:05:58.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.710+0000 7eff64778700 1 Processor -- start 2023-11-28T13:05:58.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.710+0000 7eff64778700 1 -- start start 2023-11-28T13:05:58.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.711+0000 7eff64778700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c15c8a0 0x7eff5c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.711+0000 7eff64778700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7eff5c0acd70 0x7eff5c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.711+0000 7eff64778700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5c0d0280 con 0x7eff5c0b2290 2023-11-28T13:05:58.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.711+0000 7eff64778700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5c0d0560 con 0x7eff5c0acd70 2023-11-28T13:05:58.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.711+0000 7eff64778700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c0d0840 con 0x7eff5c15c8a0 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.711+0000 7eff62d15700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7eff5c0acd70 0x7eff5c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.711+0000 7eff61d13700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7eff5c0b2290 0x7eff5c0ac660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50500/0 (socket says 172.21.15.6:50500) 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.711+0000 7eff61d13700 1 -- 172.21.15.6:0/378514815 learned_addr learned my addr 172.21.15.6:0/378514815 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff62514700 1 --2- 172.21.15.6:0/378514815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c15c8a0 0x7eff5c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff62d15700 1 -- 172.21.15.6:0/378514815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c15c8a0 msgr2=0x7eff5c0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff62d15700 1 --2- 172.21.15.6:0/378514815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c15c8a0 0x7eff5c0b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff62d15700 1 -- 172.21.15.6:0/378514815 >> v1:172.21.15.6:6789/0 conn(0x7eff5c0b2290 legacy=0x7eff5c0ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff62d15700 1 -- 172.21.15.6:0/378514815 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c0b1550 con 0x7eff5c0acd70 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff61512700 1 -- 172.21.15.6:0/378514815 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588027160 0 0) 0x7eff5c0d0280 con 0x7eff5c0b2290 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff62514700 1 --2- 172.21.15.6:0/378514815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c15c8a0 0x7eff5c0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff62d15700 1 --2- 172.21.15.6:0/378514815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0acd70 0x7eff5c0cd610 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7eff54009d90 tx=0x7eff5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4d8fe67f0f3cdf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:58.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff61512700 1 -- 172.21.15.6:0/378514815 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5400e070 con 0x7eff5c0acd70 2023-11-28T13:05:58.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff61512700 1 -- 172.21.15.6:0/378514815 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5400bbd0 con 0x7eff5c0acd70 2023-11-28T13:05:58.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff61512700 1 -- 172.21.15.6:0/378514815 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54012430 con 0x7eff5c0acd70 2023-11-28T13:05:58.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff64778700 1 -- 172.21.15.6:0/378514815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0acd70 msgr2=0x7eff5c0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:58.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.712+0000 7eff64778700 1 --2- 172.21.15.6:0/378514815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0acd70 0x7eff5c0cd610 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7eff54009d90 tx=0x7eff5400b330 comp rx=0 tx=0).stop 2023-11-28T13:05:58.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.713+0000 7eff64778700 1 -- 172.21.15.6:0/378514815 shutdown_connections 2023-11-28T13:05:58.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.713+0000 7eff64778700 1 --2- 172.21.15.6:0/378514815 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0acd70 0x7eff5c0cd610 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.713+0000 7eff64778700 1 --2- 172.21.15.6:0/378514815 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c15c8a0 0x7eff5c0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.713+0000 7eff64778700 1 -- 172.21.15.6:0/378514815 >> 172.21.15.6:0/378514815 conn(0x7eff5c0b3610 msgr2=0x7eff5c0cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.713+0000 7eff64778700 1 -- 172.21.15.6:0/378514815 shutdown_connections 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.713+0000 7eff64778700 1 -- 172.21.15.6:0/378514815 wait complete. 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.714+0000 7eff64778700 1 Processor -- start 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.714+0000 7eff64778700 1 -- start start 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff64778700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c0acd70 0x7eff5c158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff64778700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0b2290 0x7eff5c1531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff64778700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff5c15c8a0 0x7eff5c0c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff64778700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c0cada0 con 0x7eff5c0b2290 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff64778700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c159160 con 0x7eff5c15c8a0 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff64778700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c0c9210 con 0x7eff5c0acd70 2023-11-28T13:05:58.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff61d13700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0b2290 0x7eff5c1531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff61d13700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0b2290 0x7eff5c1531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34954/0 (socket says 172.21.15.6:34954) 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff61d13700 1 -- 172.21.15.6:0/2598533522 learned_addr learned my addr 172.21.15.6:0/2598533522 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff62d15700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff5c15c8a0 0x7eff5c0c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff62514700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c0acd70 0x7eff5c158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff61d13700 1 -- 172.21.15.6:0/2598533522 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c0acd70 msgr2=0x7eff5c158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff61d13700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c0acd70 0x7eff5c158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff61d13700 1 -- 172.21.15.6:0/2598533522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff5c15c8a0 msgr2=0x7eff5c0c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff61d13700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff5c15c8a0 0x7eff5c0c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff61d13700 1 -- 172.21.15.6:0/2598533522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff54009a70 con 0x7eff5c0b2290 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff61d13700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0b2290 0x7eff5c1531c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7eff580011e0 tx=0x7eff5800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff4effd700 1 -- 172.21.15.6:0/2598533522 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58004c80 con 0x7eff5c0b2290 2023-11-28T13:05:58.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff4effd700 1 -- 172.21.15.6:0/2598533522 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff58004e00 con 0x7eff5c0b2290 2023-11-28T13:05:58.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff64778700 1 -- 172.21.15.6:0/2598533522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5c15cc70 con 0x7eff5c0b2290 2023-11-28T13:05:58.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff4effd700 1 -- 172.21.15.6:0/2598533522 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff58007310 con 0x7eff5c0b2290 2023-11-28T13:05:58.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.715+0000 7eff64778700 1 -- 172.21.15.6:0/2598533522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c0b1830 con 0x7eff5c0b2290 2023-11-28T13:05:58.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.716+0000 7eff64778700 1 -- 172.21.15.6:0/2598533522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5c06a6c0 con 0x7eff5c0b2290 2023-11-28T13:05:58.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.717+0000 7eff4effd700 1 -- 172.21.15.6:0/2598533522 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff5801a020 con 0x7eff5c0b2290 2023-11-28T13:05:58.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.717+0000 7eff4effd700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff400682f0 0x7eff4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:05:58.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.717+0000 7eff4effd700 1 -- 172.21.15.6:0/2598533522 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7eff58085bd0 con 0x7eff5c0b2290 2023-11-28T13:05:58.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.717+0000 7eff62514700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff400682f0 0x7eff4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:05:58.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.718+0000 7eff62514700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff400682f0 0x7eff4006a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7eff50006fd0 tx=0x7eff50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:05:58.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.722+0000 7eff4effd700 1 -- 172.21.15.6:0/2598533522 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7eff580521c0 con 0x7eff5c0b2290 2023-11-28T13:05:58.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.958+0000 7eff64778700 1 -- 172.21.15.6:0/2598533522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7eff5c0675a0 con 0x7eff5c0b2290 2023-11-28T13:05:58.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.958+0000 7eff4effd700 1 -- 172.21.15.6:0/2598533522 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7eff580557e0 con 0x7eff5c0b2290 2023-11-28T13:05:58.960 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:05:58.960 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:05:58.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 -- 172.21.15.6:0/2598533522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff400682f0 msgr2=0x7eff4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:58.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff400682f0 0x7eff4006a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7eff50006fd0 tx=0x7eff50008040 comp rx=0 tx=0).stop 2023-11-28T13:05:58.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 -- 172.21.15.6:0/2598533522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0b2290 msgr2=0x7eff5c1531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:05:58.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0b2290 0x7eff5c1531c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7eff580011e0 tx=0x7eff5800cae0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 -- 172.21.15.6:0/2598533522 shutdown_connections 2023-11-28T13:05:58.962 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff5c15c8a0 0x7eff5c0c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff400682f0 0x7eff4006a7a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff5c0acd70 0x7eff5c158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 --2- 172.21.15.6:0/2598533522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff5c0b2290 0x7eff5c1531c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:05:58.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 -- 172.21.15.6:0/2598533522 >> 172.21.15.6:0/2598533522 conn(0x7eff5c0b3610 msgr2=0x7eff5c0cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:05:58.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.960+0000 7eff64778700 1 -- 172.21.15.6:0/2598533522 shutdown_connections 2023-11-28T13:05:58.963 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:05:58.961+0000 7eff64778700 1 -- 172.21.15.6:0/2598533522 wait complete. 2023-11-28T13:05:58.974 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:03.974 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:04.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.112+0000 7f117f5b7700 1 Processor -- start 2023-11-28T13:06:04.114 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.112+0000 7f117f5b7700 1 -- start start 2023-11-28T13:06:04.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.113+0000 7f117f5b7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780b6510 0x7f11780cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.113+0000 7f117f5b7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f11780ccfe0 0x7f1178070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.113+0000 7f117f5b7700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11780ce9b0 con 0x7f1178070a60 2023-11-28T13:06:04.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.113+0000 7f117f5b7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1178070f00 con 0x7f11780ccfe0 2023-11-28T13:06:04.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.113+0000 7f117f5b7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11780b0d30 con 0x7f11780b6510 2023-11-28T13:06:04.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.113+0000 7f117db54700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1178070a60 0x7f117815df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50516/0 (socket says 172.21.15.6:50516) 2023-11-28T13:06:04.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.113+0000 7f117db54700 1 -- 172.21.15.6:0/3100479106 learned_addr learned my addr 172.21.15.6:0/3100479106 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:06:04.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.113+0000 7f117d353700 1 --2- 172.21.15.6:0/3100479106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780b6510 0x7f11780cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.113+0000 7f117cb52700 1 --2- 172.21.15.6:0/3100479106 >> v2:172.21.15.61:3300/0 conn(0x7f11780ccfe0 0x7f1178070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f116ffff700 1 -- 172.21.15.6:0/3100479106 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785918879 0 0) 0x7f11780ce9b0 con 0x7f1178070a60 2023-11-28T13:06:04.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f116ffff700 1 -- 172.21.15.6:0/3100479106 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11640035b0 con 0x7f1178070a60 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f117d353700 1 -- 172.21.15.6:0/3100479106 >> v2:172.21.15.61:3300/0 conn(0x7f11780ccfe0 msgr2=0x7f1178070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f117d353700 1 --2- 172.21.15.6:0/3100479106 >> v2:172.21.15.61:3300/0 conn(0x7f11780ccfe0 0x7f1178070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f117d353700 1 -- 172.21.15.6:0/3100479106 >> v1:172.21.15.6:6789/0 conn(0x7f1178070a60 legacy=0x7f117815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f117d353700 1 -- 172.21.15.6:0/3100479106 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11780b0fd0 con 0x7f11780b6510 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f117cb52700 1 --2- 172.21.15.6:0/3100479106 >> v2:172.21.15.61:3300/0 conn(0x7f11780ccfe0 0x7f1178070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f117d353700 1 --2- 172.21.15.6:0/3100479106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780b6510 0x7f11780cb220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1168009840 tx=0x7f1168009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22be84e48913fe7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f116ffff700 1 -- 172.21.15.6:0/3100479106 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168011830 con 0x7f11780b6510 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f116ffff700 1 -- 172.21.15.6:0/3100479106 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1168011990 con 0x7f11780b6510 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.114+0000 7f116ffff700 1 -- 172.21.15.6:0/3100479106 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168011d20 con 0x7f11780b6510 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.115+0000 7f117f5b7700 1 -- 172.21.15.6:0/3100479106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780b6510 msgr2=0x7f11780cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.115+0000 7f117f5b7700 1 --2- 172.21.15.6:0/3100479106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780b6510 0x7f11780cb220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1168009840 tx=0x7f1168009b50 comp rx=0 tx=0).stop 2023-11-28T13:06:04.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.115+0000 7f117f5b7700 1 -- 172.21.15.6:0/3100479106 shutdown_connections 2023-11-28T13:06:04.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.115+0000 7f117f5b7700 1 --2- 172.21.15.6:0/3100479106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780b6510 0x7f11780cb220 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.115+0000 7f117f5b7700 1 --2- 172.21.15.6:0/3100479106 >> v2:172.21.15.61:3300/0 conn(0x7f11780ccfe0 0x7f1178070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.115+0000 7f117f5b7700 1 -- 172.21.15.6:0/3100479106 >> 172.21.15.6:0/3100479106 conn(0x7f11780d0bd0 msgr2=0x7f11780d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:04.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.115+0000 7f117f5b7700 1 -- 172.21.15.6:0/3100479106 shutdown_connections 2023-11-28T13:06:04.119 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.115+0000 7f117f5b7700 1 -- 172.21.15.6:0/3100479106 wait complete. 2023-11-28T13:06:04.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.116+0000 7f117f5b7700 1 Processor -- start 2023-11-28T13:06:04.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.116+0000 7f117f5b7700 1 -- start start 2023-11-28T13:06:04.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117f5b7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1178070a60 0x7f11780ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117f5b7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11780b6510 0x7f11780ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117f5b7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780ccfe0 0x7f11781f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117f5b7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11780c7f20 con 0x7f11780b6510 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117f5b7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11780cac90 con 0x7f1178070a60 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117f5b7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11780ac8f0 con 0x7f11780ccfe0 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117cb52700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11780b6510 0x7f11780ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117cb52700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11780b6510 0x7f11780ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34968/0 (socket says 172.21.15.6:34968) 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117cb52700 1 -- 172.21.15.6:0/209020106 learned_addr learned my addr 172.21.15.6:0/209020106 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117d353700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1178070a60 0x7f11780ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117db54700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780ccfe0 0x7f11781f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117cb52700 1 -- 172.21.15.6:0/209020106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780ccfe0 msgr2=0x7f11781f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117cb52700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780ccfe0 0x7f11781f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117cb52700 1 -- 172.21.15.6:0/209020106 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1178070a60 msgr2=0x7f11780ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117cb52700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1178070a60 0x7f11780ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117cb52700 1 -- 172.21.15.6:0/209020106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740094a0 con 0x7f11780b6510 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.117+0000 7f117cb52700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11780b6510 0x7f11780ac3b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f117400ebb0 tx=0x7f117400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.118+0000 7f116dffb700 1 -- 172.21.15.6:0/209020106 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400c850 con 0x7f11780b6510 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.118+0000 7f116dffb700 1 -- 172.21.15.6:0/209020106 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117400c9b0 con 0x7f11780b6510 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.118+0000 7f117f5b7700 1 -- 172.21.15.6:0/209020106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11680094a0 con 0x7f11780b6510 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.118+0000 7f116dffb700 1 -- 172.21.15.6:0/209020106 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400cd40 con 0x7f11780b6510 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.118+0000 7f117f5b7700 1 -- 172.21.15.6:0/209020106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11780cb760 con 0x7f11780b6510 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.119+0000 7f117f5b7700 1 -- 172.21.15.6:0/209020106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11780b12b0 con 0x7f11780b6510 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.119+0000 7f116dffb700 1 -- 172.21.15.6:0/209020106 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1174019030 con 0x7f11780b6510 2023-11-28T13:06:04.126 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.120+0000 7f116dffb700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f115c0682f0 0x7f115c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.120+0000 7f116dffb700 1 -- 172.21.15.6:0/209020106 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f1174085de0 con 0x7f11780b6510 2023-11-28T13:06:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.120+0000 7f117d353700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f115c0682f0 0x7f115c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.120+0000 7f117d353700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f115c0682f0 0x7f115c06a7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1168009fd0 tx=0x7f1168008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:04.127 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.124+0000 7f116dffb700 1 -- 172.21.15.6:0/209020106 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f11740523d0 con 0x7f11780b6510 2023-11-28T13:06:04.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.366+0000 7f117f5b7700 1 -- 172.21.15.6:0/209020106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f11780cfe50 con 0x7f11780b6510 2023-11-28T13:06:04.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.366+0000 7f116dffb700 1 -- 172.21.15.6:0/209020106 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f11740210c0 con 0x7f11780b6510 2023-11-28T13:06:04.367 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:04.368 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:04.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 -- 172.21.15.6:0/209020106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f115c0682f0 msgr2=0x7f115c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f115c0682f0 0x7f115c06a7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1168009fd0 tx=0x7f1168008030 comp rx=0 tx=0).stop 2023-11-28T13:06:04.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 -- 172.21.15.6:0/209020106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11780b6510 msgr2=0x7f11780ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:04.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11780b6510 0x7f11780ac3b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f117400ebb0 tx=0x7f117400eec0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 -- 172.21.15.6:0/209020106 shutdown_connections 2023-11-28T13:06:04.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f115c0682f0 0x7f115c06a7a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11780ccfe0 0x7f11781f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1178070a60 0x7f11780ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 --2- 172.21.15.6:0/209020106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11780b6510 0x7f11780ac3b0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:04.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 -- 172.21.15.6:0/209020106 >> 172.21.15.6:0/209020106 conn(0x7f11780d0bd0 msgr2=0x7f117815ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:04.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 -- 172.21.15.6:0/209020106 shutdown_connections 2023-11-28T13:06:04.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:04.368+0000 7f117f5b7700 1 -- 172.21.15.6:0/209020106 wait complete. 2023-11-28T13:06:04.382 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:09.382 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:09.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.519+0000 7f8c79e4e700 1 Processor -- start 2023-11-28T13:06:09.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.519+0000 7f8c79e4e700 1 -- start start 2023-11-28T13:06:09.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c79e4e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c7415c8a0 0x7f8c740b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c79e4e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8c740b2290 0x7f8c740ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c79e4e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f8c7406f120 0x7f8c740acba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c79e4e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c740d0280 con 0x7f8c740b2290 2023-11-28T13:06:09.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c79e4e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c740d0560 con 0x7f8c7406f120 2023-11-28T13:06:09.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c79e4e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740d0840 con 0x7f8c7415c8a0 2023-11-28T13:06:09.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c72ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8c740b2290 0x7f8c740ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c72ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8c740b2290 0x7f8c740ac660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34976/0 (socket says 172.21.15.6:34976) 2023-11-28T13:06:09.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c72ffd700 1 -- 172.21.15.6:0/2291705140 learned_addr learned my addr 172.21.15.6:0/2291705140 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:09.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c737fe700 1 --2- 172.21.15.6:0/2291705140 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c7415c8a0 0x7f8c740b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c73fff700 1 --2- 172.21.15.6:0/2291705140 >> v2:172.21.15.61:3300/0 conn(0x7f8c7406f120 0x7f8c740acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c72ffd700 1 -- 172.21.15.6:0/2291705140 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c7415c8a0 msgr2=0x7f8c740b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:09.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c72ffd700 1 --2- 172.21.15.6:0/2291705140 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c7415c8a0 0x7f8c740b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c72ffd700 1 -- 172.21.15.6:0/2291705140 >> v2:172.21.15.61:3300/0 conn(0x7f8c7406f120 msgr2=0x7f8c740acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:09.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c72ffd700 1 --2- 172.21.15.6:0/2291705140 >> v2:172.21.15.61:3300/0 conn(0x7f8c7406f120 0x7f8c740acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c72ffd700 1 -- 172.21.15.6:0/2291705140 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c740b1550 con 0x7f8c740b2290 2023-11-28T13:06:09.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c72ffd700 1 --2- 172.21.15.6:0/2291705140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c740b2290 0x7f8c740ac660 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8c74070be0 tx=0x7f8c68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34884e4eefa5e49 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:09.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c727fc700 1 -- 172.21.15.6:0/2291705140 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800b870 con 0x7f8c740b2290 2023-11-28T13:06:09.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.520+0000 7f8c727fc700 1 -- 172.21.15.6:0/2291705140 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6800b9d0 con 0x7f8c740b2290 2023-11-28T13:06:09.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.521+0000 7f8c727fc700 1 -- 172.21.15.6:0/2291705140 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800bd60 con 0x7f8c740b2290 2023-11-28T13:06:09.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.521+0000 7f8c79e4e700 1 -- 172.21.15.6:0/2291705140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c740b2290 msgr2=0x7f8c740ac660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.521+0000 7f8c79e4e700 1 --2- 172.21.15.6:0/2291705140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c740b2290 0x7f8c740ac660 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8c74070be0 tx=0x7f8c68009bd0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.521+0000 7f8c79e4e700 1 -- 172.21.15.6:0/2291705140 shutdown_connections 2023-11-28T13:06:09.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.521+0000 7f8c79e4e700 1 --2- 172.21.15.6:0/2291705140 >> v2:172.21.15.61:3300/0 conn(0x7f8c7406f120 0x7f8c740acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.521+0000 7f8c79e4e700 1 --2- 172.21.15.6:0/2291705140 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c7415c8a0 0x7f8c740b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.521+0000 7f8c79e4e700 1 --2- 172.21.15.6:0/2291705140 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c740b2290 0x7f8c740ac660 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.521+0000 7f8c79e4e700 1 -- 172.21.15.6:0/2291705140 >> 172.21.15.6:0/2291705140 conn(0x7f8c740b3610 msgr2=0x7f8c740cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:09.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.521+0000 7f8c79e4e700 1 -- 172.21.15.6:0/2291705140 shutdown_connections 2023-11-28T13:06:09.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.521+0000 7f8c79e4e700 1 -- 172.21.15.6:0/2291705140 wait complete. 2023-11-28T13:06:09.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.522+0000 7f8c79e4e700 1 Processor -- start 2023-11-28T13:06:09.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.523+0000 7f8c79e4e700 1 -- start start 2023-11-28T13:06:09.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.523+0000 7f8c79e4e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c7406f120 0x7f8c741f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.523+0000 7f8c79e4e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c740b2290 0x7f8c741583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.523+0000 7f8c79e4e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c7415c8a0 0x7f8c74143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.523+0000 7f8c79e4e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c74142d90 con 0x7f8c7406f120 2023-11-28T13:06:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.523+0000 7f8c79e4e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c74142180 con 0x7f8c7415c8a0 2023-11-28T13:06:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.523+0000 7f8c79e4e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c74142460 con 0x7f8c740b2290 2023-11-28T13:06:09.533 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.523+0000 7f8c737fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c7406f120 0x7f8c741f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.523+0000 7f8c737fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c7406f120 0x7f8c741f1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34980/0 (socket says 172.21.15.6:34980) 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.523+0000 7f8c737fe700 1 -- 172.21.15.6:0/374742522 learned_addr learned my addr 172.21.15.6:0/374742522 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c72ffd700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c740b2290 0x7f8c741583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c73fff700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c7415c8a0 0x7f8c74143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c737fe700 1 -- 172.21.15.6:0/374742522 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c740b2290 msgr2=0x7f8c741583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c737fe700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c740b2290 0x7f8c741583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c737fe700 1 -- 172.21.15.6:0/374742522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c7415c8a0 msgr2=0x7f8c74143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c737fe700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c7415c8a0 0x7f8c74143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c737fe700 1 -- 172.21.15.6:0/374742522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68009550 con 0x7f8c7406f120 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c73fff700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c7415c8a0 0x7f8c74143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c737fe700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c7406f120 0x7f8c741f1780 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400d800 tx=0x7f8c6400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c5bfff700 1 -- 172.21.15.6:0/374742522 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c640125f0 con 0x7f8c7406f120 2023-11-28T13:06:09.534 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c5bfff700 1 -- 172.21.15.6:0/374742522 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c64012750 con 0x7f8c7406f120 2023-11-28T13:06:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c79e4e700 1 -- 172.21.15.6:0/374742522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7415cc70 con 0x7f8c7406f120 2023-11-28T13:06:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c5bfff700 1 -- 172.21.15.6:0/374742522 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64012b00 con 0x7f8c7406f120 2023-11-28T13:06:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.524+0000 7f8c79e4e700 1 -- 172.21.15.6:0/374742522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c740b1830 con 0x7f8c7406f120 2023-11-28T13:06:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.525+0000 7f8c79e4e700 1 -- 172.21.15.6:0/374742522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7406a6c0 con 0x7f8c7406f120 2023-11-28T13:06:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.525+0000 7f8c5bfff700 1 -- 172.21.15.6:0/374742522 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c64012c60 con 0x7f8c7406f120 2023-11-28T13:06:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.526+0000 7f8c5bfff700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8c600682f0 0x7f8c6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.526+0000 7f8c5bfff700 1 -- 172.21.15.6:0/374742522 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f8c640862e0 con 0x7f8c7406f120 2023-11-28T13:06:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.526+0000 7f8c72ffd700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8c600682f0 0x7f8c6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.526+0000 7f8c72ffd700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8c600682f0 0x7f8c6006a7a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8c68000c00 tx=0x7f8c68008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:09.535 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.530+0000 7f8c5bfff700 1 -- 172.21.15.6:0/374742522 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8c64052940 con 0x7f8c7406f120 2023-11-28T13:06:09.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.770+0000 7f8c79e4e700 1 -- 172.21.15.6:0/374742522 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8c74143070 con 0x7f8c7406f120 2023-11-28T13:06:09.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.770+0000 7f8c5bfff700 1 -- 172.21.15.6:0/374742522 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f8c64055f60 con 0x7f8c7406f120 2023-11-28T13:06:09.772 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:09.772 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:09.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 -- 172.21.15.6:0/374742522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8c600682f0 msgr2=0x7f8c6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8c600682f0 0x7f8c6006a7a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8c68000c00 tx=0x7f8c68008030 comp rx=0 tx=0).stop 2023-11-28T13:06:09.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 -- 172.21.15.6:0/374742522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c7406f120 msgr2=0x7f8c741f1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:09.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c7406f120 0x7f8c741f1780 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400d800 tx=0x7f8c6400db10 comp rx=0 tx=0).stop 2023-11-28T13:06:09.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 -- 172.21.15.6:0/374742522 shutdown_connections 2023-11-28T13:06:09.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c7415c8a0 0x7f8c74143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c7406f120 0x7f8c741f1780 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f8c600682f0 0x7f8c6006a7a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 --2- 172.21.15.6:0/374742522 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f8c740b2290 0x7f8c741583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:09.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 -- 172.21.15.6:0/374742522 >> 172.21.15.6:0/374742522 conn(0x7f8c740b3610 msgr2=0x7f8c740cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:09.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 -- 172.21.15.6:0/374742522 shutdown_connections 2023-11-28T13:06:09.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:09.772+0000 7f8c79e4e700 1 -- 172.21.15.6:0/374742522 wait complete. 2023-11-28T13:06:09.785 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:14.786 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:14.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.923+0000 7f85f9964700 1 Processor -- start 2023-11-28T13:06:14.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.923+0000 7f85f9964700 1 -- start start 2023-11-28T13:06:14.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f9964700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85f40b49d0 0x7f85f40d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:14.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f9964700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f40b4fb0 con 0x7f85f415c6a0 2023-11-28T13:06:14.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f9964700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f40b5290 con 0x7f85f40cd5a0 2023-11-28T13:06:14.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f9964700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f40b5570 con 0x7f85f40b49d0 2023-11-28T13:06:14.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f27fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85f40b49d0 0x7f85f40d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:14.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f27fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85f40b49d0 0x7f85f40d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34990/0 (socket says 172.21.15.6:34990) 2023-11-28T13:06:14.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f2ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f85f415c6a0 0x7f85f406abf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50540/0 (socket says 172.21.15.6:50540) 2023-11-28T13:06:14.927 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f27fc700 1 -- 172.21.15.6:0/3116511086 learned_addr learned my addr 172.21.15.6:0/3116511086 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:14.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f27fc700 1 -- 172.21.15.6:0/3116511086 >> v1:172.21.15.61:6789/0 conn(0x7f85f40cd5a0 legacy=0x7f85f40b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:14.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f27fc700 1 -- 172.21.15.6:0/3116511086 >> v1:172.21.15.6:6789/0 conn(0x7f85f415c6a0 legacy=0x7f85f406abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:14.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f27fc700 1 -- 172.21.15.6:0/3116511086 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40d11d0 con 0x7f85f40b49d0 2023-11-28T13:06:14.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.924+0000 7f85f1ffb700 1 -- 172.21.15.6:0/3116511086 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842838824 0 0) 0x7f85f40b4fb0 con 0x7f85f415c6a0 2023-11-28T13:06:14.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f27fc700 1 --2- 172.21.15.6:0/3116511086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40b49d0 0x7f85f40d19d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f85e40098f0 tx=0x7f85e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8b907d3548bf35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:14.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f1ffb700 1 -- 172.21.15.6:0/3116511086 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400b8a0 con 0x7f85f40b49d0 2023-11-28T13:06:14.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f1ffb700 1 -- 172.21.15.6:0/3116511086 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e400ba00 con 0x7f85f40b49d0 2023-11-28T13:06:14.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f1ffb700 1 -- 172.21.15.6:0/3116511086 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400bd90 con 0x7f85f40b49d0 2023-11-28T13:06:14.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f9964700 1 -- 172.21.15.6:0/3116511086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40b49d0 msgr2=0x7f85f40d19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:14.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f9964700 1 --2- 172.21.15.6:0/3116511086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40b49d0 0x7f85f40d19d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f85e40098f0 tx=0x7f85e4009c00 comp rx=0 tx=0).stop 2023-11-28T13:06:14.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f9964700 1 -- 172.21.15.6:0/3116511086 shutdown_connections 2023-11-28T13:06:14.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f9964700 1 --2- 172.21.15.6:0/3116511086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40b49d0 0x7f85f40d19d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:14.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f9964700 1 -- 172.21.15.6:0/3116511086 >> 172.21.15.6:0/3116511086 conn(0x7f85f40d2f50 msgr2=0x7f85f40d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:14.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f9964700 1 -- 172.21.15.6:0/3116511086 shutdown_connections 2023-11-28T13:06:14.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.925+0000 7f85f9964700 1 -- 172.21.15.6:0/3116511086 wait complete. 2023-11-28T13:06:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.927+0000 7f85f9964700 1 Processor -- start 2023-11-28T13:06:14.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.927+0000 7f85f9964700 1 -- start start 2023-11-28T13:06:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f9964700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85f40b49d0 0x7f85f40c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f9964700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40cd5a0 0x7f85f41e7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f9964700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85f415c6a0 0x7f85f4144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f9964700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f415e800 con 0x7f85f40cd5a0 2023-11-28T13:06:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f9964700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f41e8440 con 0x7f85f40b49d0 2023-11-28T13:06:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f9964700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f41e7e80 con 0x7f85f415c6a0 2023-11-28T13:06:14.933 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40cd5a0 0x7f85f41e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f27fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40cd5a0 0x7f85f41e7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:34996/0 (socket says 172.21.15.6:34996) 2023-11-28T13:06:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f27fc700 1 -- 172.21.15.6:0/4283385802 learned_addr learned my addr 172.21.15.6:0/4283385802 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f2ffd700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85f40b49d0 0x7f85f40c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f37fe700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85f415c6a0 0x7f85f4144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f27fc700 1 -- 172.21.15.6:0/4283385802 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85f415c6a0 msgr2=0x7f85f4144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f27fc700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85f415c6a0 0x7f85f4144a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:14.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f27fc700 1 -- 172.21.15.6:0/4283385802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85f40b49d0 msgr2=0x7f85f40c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f27fc700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85f40b49d0 0x7f85f40c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f27fc700 1 -- 172.21.15.6:0/4283385802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4009550 con 0x7f85f40cd5a0 2023-11-28T13:06:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f27fc700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40cd5a0 0x7f85f41e7430 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f85e400b0e0 tx=0x7f85e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85dbfff700 1 -- 172.21.15.6:0/4283385802 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4004380 con 0x7f85f40cd5a0 2023-11-28T13:06:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85dbfff700 1 -- 172.21.15.6:0/4283385802 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e401f070 con 0x7f85f40cd5a0 2023-11-28T13:06:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85f9964700 1 -- 172.21.15.6:0/4283385802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f415ca70 con 0x7f85f40cd5a0 2023-11-28T13:06:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.928+0000 7f85dbfff700 1 -- 172.21.15.6:0/4283385802 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e40036a0 con 0x7f85f40cd5a0 2023-11-28T13:06:14.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.929+0000 7f85f9964700 1 -- 172.21.15.6:0/4283385802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f40d14b0 con 0x7f85f40cd5a0 2023-11-28T13:06:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.930+0000 7f85f9964700 1 -- 172.21.15.6:0/4283385802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f40d1f10 con 0x7f85f40cd5a0 2023-11-28T13:06:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.930+0000 7f85dbfff700 1 -- 172.21.15.6:0/4283385802 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f85e402a020 con 0x7f85f40cd5a0 2023-11-28T13:06:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.930+0000 7f85dbfff700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85e00682f0 0x7f85e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.930+0000 7f85dbfff700 1 -- 172.21.15.6:0/4283385802 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f85e4087330 con 0x7f85f40cd5a0 2023-11-28T13:06:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.930+0000 7f85f2ffd700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85e00682f0 0x7f85e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:14.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.931+0000 7f85f2ffd700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85e00682f0 0x7f85e006a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f85e8000f60 tx=0x7f85e8004a50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:14.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:14.935+0000 7f85dbfff700 1 -- 172.21.15.6:0/4283385802 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f85e4053a10 con 0x7f85f40cd5a0 2023-11-28T13:06:15.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.176+0000 7f85f9964700 1 -- 172.21.15.6:0/4283385802 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f85f4068570 con 0x7f85f40cd5a0 2023-11-28T13:06:15.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.176+0000 7f85dbfff700 1 -- 172.21.15.6:0/4283385802 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f85e4057030 con 0x7f85f40cd5a0 2023-11-28T13:06:15.177 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:15.178 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:15.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 -- 172.21.15.6:0/4283385802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85e00682f0 msgr2=0x7f85e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85e00682f0 0x7f85e006a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f85e8000f60 tx=0x7f85e8004a50 comp rx=0 tx=0).stop 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 -- 172.21.15.6:0/4283385802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40cd5a0 msgr2=0x7f85f41e7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40cd5a0 0x7f85f41e7430 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f85e400b0e0 tx=0x7f85e4004750 comp rx=0 tx=0).stop 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 -- 172.21.15.6:0/4283385802 shutdown_connections 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f85e00682f0 0x7f85e006a7a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f85f415c6a0 0x7f85f4144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85f40b49d0 0x7f85f40c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 --2- 172.21.15.6:0/4283385802 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f40cd5a0 0x7f85f41e7430 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 -- 172.21.15.6:0/4283385802 >> 172.21.15.6:0/4283385802 conn(0x7f85f40d2f50 msgr2=0x7f85f40d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 -- 172.21.15.6:0/4283385802 shutdown_connections 2023-11-28T13:06:15.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:15.178+0000 7f85f9964700 1 -- 172.21.15.6:0/4283385802 wait complete. 2023-11-28T13:06:15.191 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:20.193 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:20.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.331+0000 7fede4264700 1 Processor -- start 2023-11-28T13:06:20.332 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.331+0000 7fede4264700 1 -- start start 2023-11-28T13:06:20.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede4264700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7feddc070c30 0x7feddc1652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede4264700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feddc0ce9b0 con 0x7feddc0b6510 2023-11-28T13:06:20.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede4264700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feddc0b0d30 con 0x7feddc0ccfe0 2023-11-28T13:06:20.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede4264700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feddc0b0fd0 con 0x7feddc070c30 2023-11-28T13:06:20.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede2000700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7feddc0b6510 0x7feddc0cb220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50554/0 (socket says 172.21.15.6:50554) 2023-11-28T13:06:20.333 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede2000700 1 -- 172.21.15.6:0/1562576554 learned_addr learned my addr 172.21.15.6:0/1562576554 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:06:20.334 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede2801700 1 --2- 172.21.15.6:0/1562576554 >> v2:172.21.15.6:3300/0 conn(0x7feddc070c30 0x7feddc1652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede0ffe700 1 -- 172.21.15.6:0/1562576554 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921417233 0 0) 0x7feddc0ce9b0 con 0x7feddc0b6510 2023-11-28T13:06:20.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede0ffe700 1 -- 172.21.15.6:0/1562576554 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedc80035b0 con 0x7feddc0b6510 2023-11-28T13:06:20.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede2801700 1 -- 172.21.15.6:0/1562576554 >> v1:172.21.15.61:6789/0 conn(0x7feddc0ccfe0 legacy=0x7feddc070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede2801700 1 -- 172.21.15.6:0/1562576554 >> v1:172.21.15.6:6789/0 conn(0x7feddc0b6510 legacy=0x7feddc0cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.332+0000 7fede2801700 1 -- 172.21.15.6:0/1562576554 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc0b12b0 con 0x7feddc070c30 2023-11-28T13:06:20.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.333+0000 7fede2801700 1 --2- 172.21.15.6:0/1562576554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc070c30 0x7feddc1652e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009d90 tx=0x7fedd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920cdf8c4e1fe578 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:20.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.333+0000 7fede0ffe700 1 -- 172.21.15.6:0/1562576554 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd400e070 con 0x7feddc070c30 2023-11-28T13:06:20.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.333+0000 7fede0ffe700 1 -- 172.21.15.6:0/1562576554 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd400bb20 con 0x7feddc070c30 2023-11-28T13:06:20.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.333+0000 7fede0ffe700 1 -- 172.21.15.6:0/1562576554 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4012430 con 0x7feddc070c30 2023-11-28T13:06:20.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.333+0000 7fede4264700 1 -- 172.21.15.6:0/1562576554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc070c30 msgr2=0x7feddc1652e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.333+0000 7fede4264700 1 --2- 172.21.15.6:0/1562576554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc070c30 0x7feddc1652e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009d90 tx=0x7fedd400b330 comp rx=0 tx=0).stop 2023-11-28T13:06:20.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.333+0000 7fede4264700 1 -- 172.21.15.6:0/1562576554 shutdown_connections 2023-11-28T13:06:20.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.333+0000 7fede4264700 1 --2- 172.21.15.6:0/1562576554 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc070c30 0x7feddc1652e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.333+0000 7fede4264700 1 -- 172.21.15.6:0/1562576554 >> 172.21.15.6:0/1562576554 conn(0x7feddc0d0bd0 msgr2=0x7feddc0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:20.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.334+0000 7fede4264700 1 -- 172.21.15.6:0/1562576554 shutdown_connections 2023-11-28T13:06:20.336 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.334+0000 7fede4264700 1 -- 172.21.15.6:0/1562576554 wait complete. 2023-11-28T13:06:20.338 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.335+0000 7fede4264700 1 Processor -- start 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.335+0000 7fede4264700 1 -- start start 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede4264700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feddc070c30 0x7feddc154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede4264700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc0b6510 0x7feddc06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede4264700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feddc0ccfe0 0x7feddc06f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede4264700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0ca250 con 0x7feddc0b6510 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede4264700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0c9640 con 0x7feddc0ccfe0 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede4264700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0c9920 con 0x7feddc070c30 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede17ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc0b6510 0x7feddc06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede17ff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc0b6510 0x7feddc06f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35010/0 (socket says 172.21.15.6:35010) 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede17ff700 1 -- 172.21.15.6:0/2112096370 learned_addr learned my addr 172.21.15.6:0/2112096370 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede2801700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feddc0ccfe0 0x7feddc06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede2000700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feddc070c30 0x7feddc154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede17ff700 1 -- 172.21.15.6:0/2112096370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feddc070c30 msgr2=0x7feddc154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede17ff700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feddc070c30 0x7feddc154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede17ff700 1 -- 172.21.15.6:0/2112096370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feddc0ccfe0 msgr2=0x7feddc06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede17ff700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feddc0ccfe0 0x7feddc06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede17ff700 1 -- 172.21.15.6:0/2112096370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd4009a70 con 0x7feddc0b6510 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fede17ff700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc0b6510 0x7feddc06f250 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00c4b0 tx=0x7fedcc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.336+0000 7fedd27fc700 1 -- 172.21.15.6:0/2112096370 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0052d0 con 0x7feddc0b6510 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.337+0000 7fedd27fc700 1 -- 172.21.15.6:0/2112096370 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc005430 con 0x7feddc0b6510 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.337+0000 7fede4264700 1 -- 172.21.15.6:0/2112096370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feddc0cffe0 con 0x7feddc0b6510 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.337+0000 7fedd27fc700 1 -- 172.21.15.6:0/2112096370 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0038c0 con 0x7feddc0b6510 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.337+0000 7fede4264700 1 -- 172.21.15.6:0/2112096370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc0cfe50 con 0x7feddc0b6510 2023-11-28T13:06:20.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.338+0000 7fede4264700 1 -- 172.21.15.6:0/2112096370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feddc0d0170 con 0x7feddc0b6510 2023-11-28T13:06:20.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.338+0000 7fedd27fc700 1 -- 172.21.15.6:0/2112096370 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fedcc019030 con 0x7feddc0b6510 2023-11-28T13:06:20.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.338+0000 7fedd27fc700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fedc00682f0 0x7fedc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:20.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.338+0000 7fedd27fc700 1 -- 172.21.15.6:0/2112096370 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fedcc084fe0 con 0x7feddc0b6510 2023-11-28T13:06:20.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.339+0000 7fede2000700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fedc00682f0 0x7fedc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:20.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.339+0000 7fede2000700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fedc00682f0 0x7fedc006a7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fedd8007fd0 tx=0x7fedd8007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:20.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.343+0000 7fedd27fc700 1 -- 172.21.15.6:0/2112096370 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7feddc0d0170 con 0x7feddc0b6510 2023-11-28T13:06:20.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.582+0000 7fede4264700 1 -- 172.21.15.6:0/2112096370 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7feddc0d0170 con 0x7feddc0b6510 2023-11-28T13:06:20.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.583+0000 7fedd27fc700 1 -- 172.21.15.6:0/2112096370 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fedcc0170c0 con 0x7feddc0b6510 2023-11-28T13:06:20.585 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:20.585 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:20.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 -- 172.21.15.6:0/2112096370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fedc00682f0 msgr2=0x7fedc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fedc00682f0 0x7fedc006a7a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fedd8007fd0 tx=0x7fedd8007dc0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 -- 172.21.15.6:0/2112096370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc0b6510 msgr2=0x7feddc06f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:20.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc0b6510 0x7feddc06f250 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00c4b0 tx=0x7fedcc00c7c0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 -- 172.21.15.6:0/2112096370 shutdown_connections 2023-11-28T13:06:20.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7feddc0ccfe0 0x7feddc06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fedc00682f0 0x7fedc006a7a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7feddc070c30 0x7feddc154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 --2- 172.21.15.6:0/2112096370 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feddc0b6510 0x7feddc06f250 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:20.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 -- 172.21.15.6:0/2112096370 >> 172.21.15.6:0/2112096370 conn(0x7feddc0d0bd0 msgr2=0x7feddc0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:20.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 -- 172.21.15.6:0/2112096370 shutdown_connections 2023-11-28T13:06:20.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:20.585+0000 7fede4264700 1 -- 172.21.15.6:0/2112096370 wait complete. 2023-11-28T13:06:20.598 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:25.599 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.735+0000 7ff7e34ae700 1 Processor -- start 2023-11-28T13:06:25.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.735+0000 7ff7e34ae700 1 -- start start 2023-11-28T13:06:25.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.736+0000 7ff7e34ae700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7ff7dc0b4a70 0x7ff7dc0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.736+0000 7ff7e34ae700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0aefe0 0x7ff7dc0cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.736+0000 7ff7e34ae700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7dc0b5880 con 0x7ff7dc0cdb50 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.736+0000 7ff7e34ae700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7dc0d01e0 con 0x7ff7dc0b4a70 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.736+0000 7ff7e34ae700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0d04c0 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.736+0000 7ff7e0a49700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff7dc0cdb50 0x7ff7dc0b4e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50570/0 (socket says 172.21.15.6:50570) 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.736+0000 7ff7e0a49700 1 -- 172.21.15.6:0/2926998623 learned_addr learned my addr 172.21.15.6:0/2926998623 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.736+0000 7ff7e124a700 1 --2- 172.21.15.6:0/2926998623 >> v2:172.21.15.61:3300/0 conn(0x7ff7dc0b4a70 0x7ff7dc0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.736+0000 7ff7e1a4b700 1 --2- 172.21.15.6:0/2926998623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0aefe0 0x7ff7dc0cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.736+0000 7ff7d3fff700 1 -- 172.21.15.6:0/2926998623 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67451057 0 0) 0x7ff7dc0b5880 con 0x7ff7dc0cdb50 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.737+0000 7ff7d3fff700 1 -- 172.21.15.6:0/2926998623 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c80035b0 con 0x7ff7dc0cdb50 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.737+0000 7ff7e124a700 1 -- 172.21.15.6:0/2926998623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0aefe0 msgr2=0x7ff7dc0cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.737+0000 7ff7e124a700 1 --2- 172.21.15.6:0/2926998623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0aefe0 0x7ff7dc0cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.737+0000 7ff7e124a700 1 -- 172.21.15.6:0/2926998623 >> v1:172.21.15.6:6789/0 conn(0x7ff7dc0cdb50 legacy=0x7ff7dc0b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:25.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.737+0000 7ff7e124a700 1 -- 172.21.15.6:0/2926998623 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc0d07a0 con 0x7ff7dc0b4a70 2023-11-28T13:06:25.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.737+0000 7ff7e1a4b700 1 --2- 172.21.15.6:0/2926998623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0aefe0 0x7ff7dc0cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:06:25.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.737+0000 7ff7e124a700 1 --2- 172.21.15.6:0/2926998623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7dc0b4a70 0x7ff7dc0cd610 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009840 tx=0x7ff7cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1b39c91070cdc81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:25.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.737+0000 7ff7d3fff700 1 -- 172.21.15.6:0/2926998623 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc011830 con 0x7ff7dc0b4a70 2023-11-28T13:06:25.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.737+0000 7ff7d3fff700 1 -- 172.21.15.6:0/2926998623 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc011990 con 0x7ff7dc0b4a70 2023-11-28T13:06:25.740 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.737+0000 7ff7d3fff700 1 -- 172.21.15.6:0/2926998623 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc011d00 con 0x7ff7dc0b4a70 2023-11-28T13:06:25.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.738+0000 7ff7e34ae700 1 -- 172.21.15.6:0/2926998623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7dc0b4a70 msgr2=0x7ff7dc0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:25.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.738+0000 7ff7e34ae700 1 --2- 172.21.15.6:0/2926998623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7dc0b4a70 0x7ff7dc0cd610 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009840 tx=0x7ff7cc009b50 comp rx=0 tx=0).stop 2023-11-28T13:06:25.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.738+0000 7ff7e34ae700 1 -- 172.21.15.6:0/2926998623 shutdown_connections 2023-11-28T13:06:25.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.738+0000 7ff7e34ae700 1 --2- 172.21.15.6:0/2926998623 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0aefe0 0x7ff7dc0cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.741 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.738+0000 7ff7e34ae700 1 --2- 172.21.15.6:0/2926998623 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7dc0b4a70 0x7ff7dc0cd610 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.738+0000 7ff7e34ae700 1 -- 172.21.15.6:0/2926998623 >> 172.21.15.6:0/2926998623 conn(0x7ff7dc0b3610 msgr2=0x7ff7dc0cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:25.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.738+0000 7ff7e34ae700 1 -- 172.21.15.6:0/2926998623 shutdown_connections 2023-11-28T13:06:25.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.738+0000 7ff7e34ae700 1 -- 172.21.15.6:0/2926998623 wait complete. 2023-11-28T13:06:25.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.739+0000 7ff7e34ae700 1 Processor -- start 2023-11-28T13:06:25.742 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.740+0000 7ff7e34ae700 1 -- start start 2023-11-28T13:06:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.740+0000 7ff7e34ae700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7dc0aefe0 0x7ff7dc0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.740+0000 7ff7e34ae700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0b4a70 0x7ff7dc06aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e34ae700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7dc0cdb50 0x7ff7dc1d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e34ae700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc06afb0 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e34ae700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc099770 con 0x7ff7dc0cdb50 2023-11-28T13:06:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e34ae700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc09b490 con 0x7ff7dc0b4a70 2023-11-28T13:06:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e124a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7dc0aefe0 0x7ff7dc0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e124a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7dc0aefe0 0x7ff7dc0afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35022/0 (socket says 172.21.15.6:35022) 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e124a700 1 -- 172.21.15.6:0/563185663 learned_addr learned my addr 172.21.15.6:0/563185663 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e1a4b700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7dc0cdb50 0x7ff7dc1d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e0a49700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0b4a70 0x7ff7dc06aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e124a700 1 -- 172.21.15.6:0/563185663 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0b4a70 msgr2=0x7ff7dc06aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e124a700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0b4a70 0x7ff7dc06aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e124a700 1 -- 172.21.15.6:0/563185663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7dc0cdb50 msgr2=0x7ff7dc1d5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e124a700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7dc0cdb50 0x7ff7dc1d5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e124a700 1 -- 172.21.15.6:0/563185663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4009a70 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e124a700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7dc0aefe0 0x7ff7dc0afaf0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0090b0 tx=0x7ff7cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7d1ffb700 1 -- 172.21.15.6:0/563185663 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc004550 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7d1ffb700 1 -- 172.21.15.6:0/563185663 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc01f070 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7e34ae700 1 -- 172.21.15.6:0/563185663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc0094a0 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.741+0000 7ff7d1ffb700 1 -- 172.21.15.6:0/563185663 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc0038a0 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.742+0000 7ff7e34ae700 1 -- 172.21.15.6:0/563185663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc0aead0 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.743+0000 7ff7e34ae700 1 -- 172.21.15.6:0/563185663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7dc0b5550 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.743+0000 7ff7d1ffb700 1 -- 172.21.15.6:0/563185663 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff7cc00f050 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.746 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.743+0000 7ff7d1ffb700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7c00682f0 0x7ff7c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.744+0000 7ff7d1ffb700 1 -- 172.21.15.6:0/563185663 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff7cc021070 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.744+0000 7ff7e0a49700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7c00682f0 0x7ff7c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:25.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.744+0000 7ff7e0a49700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7c00682f0 0x7ff7c006a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0b02e0 tx=0x7ff7d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:25.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.748+0000 7ff7d1ffb700 1 -- 172.21.15.6:0/563185663 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff7cc053c10 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.987+0000 7ff7e34ae700 1 -- 172.21.15.6:0/563185663 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff7dc0aec60 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.988+0000 7ff7d1ffb700 1 -- 172.21.15.6:0/563185663 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ff7cc057230 con 0x7ff7dc0aefe0 2023-11-28T13:06:25.989 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:25.989 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:25.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 -- 172.21.15.6:0/563185663 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7c00682f0 msgr2=0x7ff7c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7c00682f0 0x7ff7c006a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc0b02e0 tx=0x7ff7d8009040 comp rx=0 tx=0).stop 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 -- 172.21.15.6:0/563185663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7dc0aefe0 msgr2=0x7ff7dc0afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7dc0aefe0 0x7ff7dc0afaf0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0090b0 tx=0x7ff7cc004750 comp rx=0 tx=0).stop 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 -- 172.21.15.6:0/563185663 shutdown_connections 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff7dc0cdb50 0x7ff7dc1d5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7dc0aefe0 0x7ff7dc0afaf0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff7c00682f0 0x7ff7c006a7a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 --2- 172.21.15.6:0/563185663 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff7dc0b4a70 0x7ff7dc06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 -- 172.21.15.6:0/563185663 >> 172.21.15.6:0/563185663 conn(0x7ff7dc0b3610 msgr2=0x7ff7dc0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 -- 172.21.15.6:0/563185663 shutdown_connections 2023-11-28T13:06:25.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:25.990+0000 7ff7e34ae700 1 -- 172.21.15.6:0/563185663 wait complete. 2023-11-28T13:06:26.003 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:27.306 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:06:27.309 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:06:31.004 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:31.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.141+0000 7f1cab5b2700 1 Processor -- start 2023-11-28T13:06:31.143 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.141+0000 7f1cab5b2700 1 -- start start 2023-11-28T13:06:31.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.142+0000 7f1cab5b2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1ca40ccfe0 0x7f1ca4070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.142+0000 7f1cab5b2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca4070a60 0x7f1ca40cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.142+0000 7f1cab5b2700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca40ce9b0 con 0x7f1ca40b6510 2023-11-28T13:06:31.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.142+0000 7f1cab5b2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca4070f00 con 0x7f1ca40ccfe0 2023-11-28T13:06:31.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.142+0000 7f1cab5b2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40b0d30 con 0x7f1ca4070a60 2023-11-28T13:06:31.144 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.142+0000 7f1ca8b4d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1ca40ccfe0 0x7f1ca4070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.142+0000 7f1ca9b4f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca4070a60 0x7f1ca40cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1ca8b4d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f1ca40ccfe0 0x7f1ca4070520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:38750/0 (socket says 172.21.15.6:38750) 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1ca9b4f700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca4070a60 0x7f1ca40cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:52970/0 (socket says 172.21.15.6:52970) 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1ca8b4d700 1 -- 172.21.15.6:0/2273475518 learned_addr learned my addr 172.21.15.6:0/2273475518 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1ca934e700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f1ca40b6510 0x7f1ca40cb220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:40612/0 (socket says 172.21.15.6:40612) 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1ca9b4f700 1 -- 172.21.15.6:0/2273475518 >> v2:172.21.15.61:3300/0 conn(0x7f1ca40ccfe0 msgr2=0x7f1ca4070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1ca9b4f700 1 --2- 172.21.15.6:0/2273475518 >> v2:172.21.15.61:3300/0 conn(0x7f1ca40ccfe0 0x7f1ca4070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1ca9b4f700 1 -- 172.21.15.6:0/2273475518 >> v1:172.21.15.61:6789/0 conn(0x7f1ca40b6510 legacy=0x7f1ca40cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1ca9b4f700 1 -- 172.21.15.6:0/2273475518 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40b0fd0 con 0x7f1ca4070a60 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1ca8b4d700 1 --2- 172.21.15.6:0/2273475518 >> v2:172.21.15.61:3300/0 conn(0x7f1ca40ccfe0 0x7f1ca4070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:06:31.145 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1ca9b4f700 1 --2- 172.21.15.6:0/2273475518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca4070a60 0x7f1ca40cee40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009d90 tx=0x7f1c9c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1129fcd47284262f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:31.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1c9bfff700 1 -- 172.21.15.6:0/2273475518 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c00e070 con 0x7f1ca4070a60 2023-11-28T13:06:31.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1c9bfff700 1 -- 172.21.15.6:0/2273475518 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c00bb20 con 0x7f1ca4070a60 2023-11-28T13:06:31.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.143+0000 7f1c9bfff700 1 -- 172.21.15.6:0/2273475518 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c012430 con 0x7f1ca4070a60 2023-11-28T13:06:31.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.144+0000 7f1cab5b2700 1 -- 172.21.15.6:0/2273475518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca4070a60 msgr2=0x7f1ca40cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.144+0000 7f1cab5b2700 1 --2- 172.21.15.6:0/2273475518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca4070a60 0x7f1ca40cee40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009d90 tx=0x7f1c9c00b330 comp rx=0 tx=0).stop 2023-11-28T13:06:31.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.144+0000 7f1cab5b2700 1 -- 172.21.15.6:0/2273475518 shutdown_connections 2023-11-28T13:06:31.149 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.144+0000 7f1cab5b2700 1 --2- 172.21.15.6:0/2273475518 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca4070a60 0x7f1ca40cee40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.144+0000 7f1cab5b2700 1 --2- 172.21.15.6:0/2273475518 >> v2:172.21.15.61:3300/0 conn(0x7f1ca40ccfe0 0x7f1ca4070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.144+0000 7f1cab5b2700 1 -- 172.21.15.6:0/2273475518 >> 172.21.15.6:0/2273475518 conn(0x7f1ca40d0bd0 msgr2=0x7f1ca40d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.144+0000 7f1cab5b2700 1 -- 172.21.15.6:0/2273475518 shutdown_connections 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.144+0000 7f1cab5b2700 1 -- 172.21.15.6:0/2273475518 wait complete. 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.145+0000 7f1cab5b2700 1 Processor -- start 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.145+0000 7f1cab5b2700 1 -- start start 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.146+0000 7f1cab5b2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ca4070a60 0x7f1ca40c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.146+0000 7f1cab5b2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca40b6510 0x7f1ca40c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.146+0000 7f1cab5b2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca40ccfe0 0x7f1ca406f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.146+0000 7f1cab5b2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40ca6a0 con 0x7f1ca4070a60 2023-11-28T13:06:31.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.146+0000 7f1cab5b2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40cab30 con 0x7f1ca40b6510 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.146+0000 7f1cab5b2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40c9a90 con 0x7f1ca40ccfe0 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.146+0000 7f1ca934e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ca4070a60 0x7f1ca40c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.146+0000 7f1ca934e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ca4070a60 0x7f1ca40c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35036/0 (socket says 172.21.15.6:35036) 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.146+0000 7f1ca934e700 1 -- 172.21.15.6:0/3696652644 learned_addr learned my addr 172.21.15.6:0/3696652644 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1ca9b4f700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca40ccfe0 0x7f1ca406f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1ca8b4d700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca40b6510 0x7f1ca40c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1ca934e700 1 -- 172.21.15.6:0/3696652644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca40ccfe0 msgr2=0x7f1ca406f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1ca934e700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca40ccfe0 0x7f1ca406f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1ca934e700 1 -- 172.21.15.6:0/3696652644 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca40b6510 msgr2=0x7f1ca40c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1ca934e700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca40b6510 0x7f1ca40c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1ca934e700 1 -- 172.21.15.6:0/3696652644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca0009550 con 0x7f1ca4070a60 2023-11-28T13:06:31.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1ca8b4d700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca40b6510 0x7f1ca40c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:31.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1ca934e700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ca4070a60 0x7f1ca40c93f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c490 tx=0x7f1c9400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:31.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1c99ffb700 1 -- 172.21.15.6:0/3696652644 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94005cb0 con 0x7f1ca4070a60 2023-11-28T13:06:31.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1c99ffb700 1 -- 172.21.15.6:0/3696652644 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c94005e10 con 0x7f1ca4070a60 2023-11-28T13:06:31.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1cab5b2700 1 -- 172.21.15.6:0/3696652644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c009a70 con 0x7f1ca4070a60 2023-11-28T13:06:31.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1c99ffb700 1 -- 172.21.15.6:0/3696652644 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940072d0 con 0x7f1ca4070a60 2023-11-28T13:06:31.152 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.147+0000 7f1cab5b2700 1 -- 172.21.15.6:0/3696652644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca40cf380 con 0x7f1ca4070a60 2023-11-28T13:06:31.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.148+0000 7f1cab5b2700 1 -- 172.21.15.6:0/3696652644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca40b12b0 con 0x7f1ca4070a60 2023-11-28T13:06:31.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.149+0000 7f1c99ffb700 1 -- 172.21.15.6:0/3696652644 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1c94019030 con 0x7f1ca4070a60 2023-11-28T13:06:31.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.149+0000 7f1c99ffb700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1c880682f0 0x7f1c8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:31.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.149+0000 7f1c99ffb700 1 -- 172.21.15.6:0/3696652644 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f1c94085030 con 0x7f1ca4070a60 2023-11-28T13:06:31.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.149+0000 7f1ca8b4d700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1c880682f0 0x7f1c8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:31.153 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.150+0000 7f1ca8b4d700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1c880682f0 0x7f1c8806a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4071380 tx=0x7f1ca0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:31.154 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.153+0000 7f1c99ffb700 1 -- 172.21.15.6:0/3696652644 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1c9404b5e0 con 0x7f1ca4070a60 2023-11-28T13:06:31.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.389+0000 7f1cab5b2700 1 -- 172.21.15.6:0/3696652644 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1ca40cf510 con 0x7f1ca4070a60 2023-11-28T13:06:31.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.389+0000 7f1c99ffb700 1 -- 172.21.15.6:0/3696652644 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f1c94017080 con 0x7f1ca4070a60 2023-11-28T13:06:31.391 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:31.391 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 -- 172.21.15.6:0/3696652644 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1c880682f0 msgr2=0x7f1c8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1c880682f0 0x7f1c8806a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4071380 tx=0x7f1ca0008040 comp rx=0 tx=0).stop 2023-11-28T13:06:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 -- 172.21.15.6:0/3696652644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ca4070a60 msgr2=0x7f1ca40c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ca4070a60 0x7f1ca40c93f0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c490 tx=0x7f1c9400c850 comp rx=0 tx=0).stop 2023-11-28T13:06:31.393 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 -- 172.21.15.6:0/3696652644 shutdown_connections 2023-11-28T13:06:31.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1c880682f0 0x7f1c8806a7a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1ca40ccfe0 0x7f1ca406f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1ca4070a60 0x7f1ca40c93f0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 --2- 172.21.15.6:0/3696652644 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca40b6510 0x7f1ca40c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:31.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 -- 172.21.15.6:0/3696652644 >> 172.21.15.6:0/3696652644 conn(0x7f1ca40d0bd0 msgr2=0x7f1ca40b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:31.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 -- 172.21.15.6:0/3696652644 shutdown_connections 2023-11-28T13:06:31.394 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:31.392+0000 7f1cab5b2700 1 -- 172.21.15.6:0/3696652644 wait complete. 2023-11-28T13:06:31.405 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:36.405 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.542+0000 7f95313ce700 1 Processor -- start 2023-11-28T13:06:36.543 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.542+0000 7f95313ce700 1 -- start start 2023-11-28T13:06:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f95313ce700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f952c15c8a0 0x7f952c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f95313ce700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f952c0acd70 0x7f952c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f95313ce700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c0d0280 con 0x7f952c0b2290 2023-11-28T13:06:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f95313ce700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952c0d0560 con 0x7f952c15c8a0 2023-11-28T13:06:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f95313ce700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0d0840 con 0x7f952c0acd70 2023-11-28T13:06:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f952affd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f952c15c8a0 0x7f952c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.546 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f952affd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f952c15c8a0 0x7f952c0b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35044/0 (socket says 172.21.15.6:35044) 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f952affd700 1 -- 172.21.15.6:0/1423541660 learned_addr learned my addr 172.21.15.6:0/1423541660 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f952a7fc700 1 --1- 172.21.15.6:0/1423541660 >> v1:172.21.15.6:6789/0 conn(0x7f952c0b2290 0x7f952c0ac660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50598/0 (socket says 172.21.15.6:50598) 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f952b7fe700 1 --2- 172.21.15.6:0/1423541660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f952c0acd70 0x7f952c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f952affd700 1 -- 172.21.15.6:0/1423541660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f952c0acd70 msgr2=0x7f952c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f952affd700 1 --2- 172.21.15.6:0/1423541660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f952c0acd70 0x7f952c0cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f952affd700 1 -- 172.21.15.6:0/1423541660 >> v1:172.21.15.6:6789/0 conn(0x7f952c0b2290 legacy=0x7f952c0ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f952affd700 1 -- 172.21.15.6:0/1423541660 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c0b1550 con 0x7f952c15c8a0 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.543+0000 7f952affd700 1 --2- 172.21.15.6:0/1423541660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c15c8a0 0x7f952c0b1d50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f95200098f0 tx=0x7f9520009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdd0741d09d4e9a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f9529ffb700 1 -- 172.21.15.6:0/1423541660 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000b8a0 con 0x7f952c15c8a0 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f9529ffb700 1 -- 172.21.15.6:0/1423541660 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952000ba00 con 0x7f952c15c8a0 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f9529ffb700 1 -- 172.21.15.6:0/1423541660 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952000bd90 con 0x7f952c15c8a0 2023-11-28T13:06:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f95313ce700 1 -- 172.21.15.6:0/1423541660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c15c8a0 msgr2=0x7f952c0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f95313ce700 1 --2- 172.21.15.6:0/1423541660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c15c8a0 0x7f952c0b1d50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f95200098f0 tx=0x7f9520009c00 comp rx=0 tx=0).stop 2023-11-28T13:06:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f95313ce700 1 -- 172.21.15.6:0/1423541660 shutdown_connections 2023-11-28T13:06:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f95313ce700 1 --2- 172.21.15.6:0/1423541660 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f952c0acd70 0x7f952c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f95313ce700 1 --2- 172.21.15.6:0/1423541660 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c15c8a0 0x7f952c0b1d50 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f95313ce700 1 -- 172.21.15.6:0/1423541660 >> 172.21.15.6:0/1423541660 conn(0x7f952c0b3610 msgr2=0x7f952c0cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f95313ce700 1 -- 172.21.15.6:0/1423541660 shutdown_connections 2023-11-28T13:06:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.544+0000 7f95313ce700 1 -- 172.21.15.6:0/1423541660 wait complete. 2023-11-28T13:06:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.546+0000 7f95313ce700 1 Processor -- start 2023-11-28T13:06:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.546+0000 7f95313ce700 1 -- start start 2023-11-28T13:06:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f95313ce700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c0acd70 0x7f952c158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f95313ce700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f952c0b2290 0x7f952c0af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f95313ce700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f952c15c8a0 0x7f952c0afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f95313ce700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c1f2100 con 0x7f952c0acd70 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f95313ce700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0c4970 con 0x7f952c0b2290 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f95313ce700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0c7ff0 con 0x7f952c15c8a0 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f952affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c0acd70 0x7f952c158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f952affd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c0acd70 0x7f952c158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35050/0 (socket says 172.21.15.6:35050) 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f952affd700 1 -- 172.21.15.6:0/2900593066 learned_addr learned my addr 172.21.15.6:0/2900593066 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f952a7fc700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f952c0b2290 0x7f952c0af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f952b7fe700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f952c15c8a0 0x7f952c0afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f952affd700 1 -- 172.21.15.6:0/2900593066 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f952c15c8a0 msgr2=0x7f952c0afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f952affd700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f952c15c8a0 0x7f952c0afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f952affd700 1 -- 172.21.15.6:0/2900593066 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f952c0b2290 msgr2=0x7f952c0af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f952affd700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f952c0b2290 0x7f952c0af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.547+0000 7f952affd700 1 -- 172.21.15.6:0/2900593066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9520009550 con 0x7f952c0acd70 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.548+0000 7f952affd700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c0acd70 0x7f952c158620 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9520014040 tx=0x7f9520004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.548+0000 7f95137fe700 1 -- 172.21.15.6:0/2900593066 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520004510 con 0x7f952c0acd70 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.548+0000 7f95137fe700 1 -- 172.21.15.6:0/2900593066 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952001e070 con 0x7f952c0acd70 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.548+0000 7f95313ce700 1 -- 172.21.15.6:0/2900593066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c15cc70 con 0x7f952c0acd70 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.548+0000 7f95137fe700 1 -- 172.21.15.6:0/2900593066 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95200038e0 con 0x7f952c0acd70 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.548+0000 7f95313ce700 1 -- 172.21.15.6:0/2900593066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c0b1830 con 0x7f952c0acd70 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.549+0000 7f95313ce700 1 -- 172.21.15.6:0/2900593066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c06a6c0 con 0x7f952c0acd70 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.549+0000 7f95137fe700 1 -- 172.21.15.6:0/2900593066 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f952000f050 con 0x7f952c0acd70 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.550+0000 7f95137fe700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95180682f0 0x7f951806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.550+0000 7f95137fe700 1 -- 172.21.15.6:0/2900593066 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9520020070 con 0x7f952c0acd70 2023-11-28T13:06:36.553 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.550+0000 7f952a7fc700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95180682f0 0x7f951806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:36.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.550+0000 7f952a7fc700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95180682f0 0x7f951806a7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f952c0c5b10 tx=0x7f951c007ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:36.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.554+0000 7f95137fe700 1 -- 172.21.15.6:0/2900593066 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f95200538e0 con 0x7f952c0acd70 2023-11-28T13:06:36.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.793+0000 7f95313ce700 1 -- 172.21.15.6:0/2900593066 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f952c0675a0 con 0x7f952c0acd70 2023-11-28T13:06:36.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.793+0000 7f95137fe700 1 -- 172.21.15.6:0/2900593066 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f9520056f00 con 0x7f952c0acd70 2023-11-28T13:06:36.794 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:36.795 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:36.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 -- 172.21.15.6:0/2900593066 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95180682f0 msgr2=0x7f951806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95180682f0 0x7f951806a7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f952c0c5b10 tx=0x7f951c007ed0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 -- 172.21.15.6:0/2900593066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c0acd70 msgr2=0x7f952c158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c0acd70 0x7f952c158620 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9520014040 tx=0x7f9520004750 comp rx=0 tx=0).stop 2023-11-28T13:06:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 -- 172.21.15.6:0/2900593066 shutdown_connections 2023-11-28T13:06:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95180682f0 0x7f951806a7a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f952c15c8a0 0x7f952c0afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f952c0acd70 0x7f952c158620 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 --2- 172.21.15.6:0/2900593066 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f952c0b2290 0x7f952c0af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 -- 172.21.15.6:0/2900593066 >> 172.21.15.6:0/2900593066 conn(0x7f952c0b3610 msgr2=0x7f952c15aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 -- 172.21.15.6:0/2900593066 shutdown_connections 2023-11-28T13:06:36.797 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:36.795+0000 7f95313ce700 1 -- 172.21.15.6:0/2900593066 wait complete. 2023-11-28T13:06:36.808 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:41.809 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:41.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.946+0000 7f4344cff700 1 Processor -- start 2023-11-28T13:06:41.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.946+0000 7f4344cff700 1 -- start start 2023-11-28T13:06:41.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.947+0000 7f4344cff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 0x7f43400ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.947+0000 7f4344cff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f43400acba0 0x7f43400cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.947+0000 7f4344cff700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43400b2460 con 0x7f434015c8a0 2023-11-28T13:06:41.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.947+0000 7f4344cff700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43400acfe0 con 0x7f43400acba0 2023-11-28T13:06:41.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.947+0000 7f4344cff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43400d0280 con 0x7f434006f120 2023-11-28T13:06:41.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.947+0000 7f433dd9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 0x7f43400ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.947+0000 7f433ed9d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f43400acba0 0x7f43400cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.947+0000 7f433dd9b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 0x7f43400ac660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:52994/0 (socket says 172.21.15.6:52994) 2023-11-28T13:06:41.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.947+0000 7f433dd9b700 1 -- 172.21.15.6:0/2475471917 learned_addr learned my addr 172.21.15.6:0/2475471917 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:41.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.948+0000 7f433dd9b700 1 -- 172.21.15.6:0/2475471917 >> v2:172.21.15.61:3300/0 conn(0x7f43400acba0 msgr2=0x7f43400cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:41.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.948+0000 7f433dd9b700 1 --2- 172.21.15.6:0/2475471917 >> v2:172.21.15.61:3300/0 conn(0x7f43400acba0 0x7f43400cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:41.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.948+0000 7f433dd9b700 1 -- 172.21.15.6:0/2475471917 >> v1:172.21.15.61:6789/0 conn(0x7f434015c8a0 legacy=0x7f43400b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.948+0000 7f433dd9b700 1 -- 172.21.15.6:0/2475471917 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400d0560 con 0x7f434006f120 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.948+0000 7f433dd9b700 1 --2- 172.21.15.6:0/2475471917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 0x7f43400ac660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43340098f0 tx=0x7f4334009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd69c9d52303b10 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.948+0000 7f433d59a700 1 -- 172.21.15.6:0/2475471917 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4334011830 con 0x7f434006f120 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.948+0000 7f433d59a700 1 -- 172.21.15.6:0/2475471917 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4334011990 con 0x7f434006f120 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.948+0000 7f433d59a700 1 -- 172.21.15.6:0/2475471917 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4334011d20 con 0x7f434006f120 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.949+0000 7f4344cff700 1 -- 172.21.15.6:0/2475471917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 msgr2=0x7f43400ac660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.949+0000 7f4344cff700 1 --2- 172.21.15.6:0/2475471917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 0x7f43400ac660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43340098f0 tx=0x7f4334009c00 comp rx=0 tx=0).stop 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.949+0000 7f4344cff700 1 -- 172.21.15.6:0/2475471917 shutdown_connections 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.949+0000 7f4344cff700 1 --2- 172.21.15.6:0/2475471917 >> v2:172.21.15.61:3300/0 conn(0x7f43400acba0 0x7f43400cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.949+0000 7f4344cff700 1 --2- 172.21.15.6:0/2475471917 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 0x7f43400ac660 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.949+0000 7f4344cff700 1 -- 172.21.15.6:0/2475471917 >> 172.21.15.6:0/2475471917 conn(0x7f43400b3610 msgr2=0x7f43400cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.949+0000 7f4344cff700 1 -- 172.21.15.6:0/2475471917 shutdown_connections 2023-11-28T13:06:41.951 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.949+0000 7f4344cff700 1 -- 172.21.15.6:0/2475471917 wait complete. 2023-11-28T13:06:41.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.950+0000 7f4344cff700 1 Processor -- start 2023-11-28T13:06:41.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.950+0000 7f4344cff700 1 -- start start 2023-11-28T13:06:41.953 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f4344cff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 0x7f43400add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f4344cff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43400acba0 0x7f434015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f4344cff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f434015c8a0 0x7f4340146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f4344cff700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43400c7070 con 0x7f434015c8a0 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f4344cff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434015b640 con 0x7f43400acba0 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f4344cff700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434015b920 con 0x7f434006f120 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f433ed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f434015c8a0 0x7f4340146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f433ed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f434015c8a0 0x7f4340146d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35068/0 (socket says 172.21.15.6:35068) 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f433ed9d700 1 -- 172.21.15.6:0/676228088 learned_addr learned my addr 172.21.15.6:0/676228088 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f433e59c700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 0x7f43400add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f433dd9b700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43400acba0 0x7f434015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f433ed9d700 1 -- 172.21.15.6:0/676228088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 msgr2=0x7f43400add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f433ed9d700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 0x7f43400add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:41.954 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f433ed9d700 1 -- 172.21.15.6:0/676228088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43400acba0 msgr2=0x7f434015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:41.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f433ed9d700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43400acba0 0x7f434015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:41.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.951+0000 7f433ed9d700 1 -- 172.21.15.6:0/676228088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4334009550 con 0x7f434015c8a0 2023-11-28T13:06:41.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.952+0000 7f433ed9d700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f434015c8a0 0x7f4340146d00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f433800efc0 tx=0x7f433800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:41.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.952+0000 7f432effd700 1 -- 172.21.15.6:0/676228088 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338009ea0 con 0x7f434015c8a0 2023-11-28T13:06:41.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.952+0000 7f432effd700 1 -- 172.21.15.6:0/676228088 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433800f040 con 0x7f434015c8a0 2023-11-28T13:06:41.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.952+0000 7f4344cff700 1 -- 172.21.15.6:0/676228088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43400b1550 con 0x7f434015c8a0 2023-11-28T13:06:41.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.952+0000 7f432effd700 1 -- 172.21.15.6:0/676228088 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800ced0 con 0x7f434015c8a0 2023-11-28T13:06:41.955 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.952+0000 7f4344cff700 1 -- 172.21.15.6:0/676228088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43400d0840 con 0x7f434015c8a0 2023-11-28T13:06:41.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.953+0000 7f4344cff700 1 -- 172.21.15.6:0/676228088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43400b1830 con 0x7f434015c8a0 2023-11-28T13:06:41.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.953+0000 7f432effd700 1 -- 172.21.15.6:0/676228088 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4338014480 con 0x7f434015c8a0 2023-11-28T13:06:41.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.954+0000 7f432effd700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f43200682f0 0x7f432006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:41.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.954+0000 7f432effd700 1 -- 172.21.15.6:0/676228088 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f4338086790 con 0x7f434015c8a0 2023-11-28T13:06:41.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.954+0000 7f433e59c700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f43200682f0 0x7f432006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:41.956 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.954+0000 7f433e59c700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f43200682f0 0x7f432006a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4330007fd0 tx=0x7f4330009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:41.959 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:41.958+0000 7f432effd700 1 -- 172.21.15.6:0/676228088 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4338052e10 con 0x7f434015c8a0 2023-11-28T13:06:42.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.196+0000 7f4344cff700 1 -- 172.21.15.6:0/676228088 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4340066c60 con 0x7f434015c8a0 2023-11-28T13:06:42.198 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.197+0000 7f432effd700 1 -- 172.21.15.6:0/676228088 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f4338056430 con 0x7f434015c8a0 2023-11-28T13:06:42.198 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:42.198 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:42.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.198+0000 7f4344cff700 1 -- 172.21.15.6:0/676228088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f43200682f0 msgr2=0x7f432006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:42.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.198+0000 7f4344cff700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f43200682f0 0x7f432006a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4330007fd0 tx=0x7f4330009040 comp rx=0 tx=0).stop 2023-11-28T13:06:42.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.199+0000 7f4344cff700 1 -- 172.21.15.6:0/676228088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f434015c8a0 msgr2=0x7f4340146d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:42.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.199+0000 7f4344cff700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f434015c8a0 0x7f4340146d00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f433800efc0 tx=0x7f433800c3a0 comp rx=0 tx=0).stop 2023-11-28T13:06:42.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.199+0000 7f4344cff700 1 -- 172.21.15.6:0/676228088 shutdown_connections 2023-11-28T13:06:42.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.199+0000 7f4344cff700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f434015c8a0 0x7f4340146d00 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:42.200 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.199+0000 7f4344cff700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f43200682f0 0x7f432006a7a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:42.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.199+0000 7f4344cff700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f434006f120 0x7f43400add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:42.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.199+0000 7f4344cff700 1 --2- 172.21.15.6:0/676228088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f43400acba0 0x7f434015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:42.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.199+0000 7f4344cff700 1 -- 172.21.15.6:0/676228088 >> 172.21.15.6:0/676228088 conn(0x7f43400b3610 msgr2=0x7f43400cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:42.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.199+0000 7f4344cff700 1 -- 172.21.15.6:0/676228088 shutdown_connections 2023-11-28T13:06:42.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:42.199+0000 7f4344cff700 1 -- 172.21.15.6:0/676228088 wait complete. 2023-11-28T13:06:42.212 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:47.214 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:47.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.351+0000 7f68ee069700 1 Processor -- start 2023-11-28T13:06:47.352 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.351+0000 7f68ee069700 1 -- start start 2023-11-28T13:06:47.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68ee069700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e8070520 0x7f68e80708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68ee069700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f68e80cb220 0x7f68e80b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68ee069700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e815dbd0 con 0x7f68e80cb690 2023-11-28T13:06:47.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68ee069700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e8070e30 con 0x7f68e80cb220 2023-11-28T13:06:47.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68ee069700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e815d610 con 0x7f68e8070520 2023-11-28T13:06:47.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68e77fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e8070520 0x7f68e80708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68e6ffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f68e80cb220 0x7f68e80b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68e77fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e8070520 0x7f68e80708f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:53008/0 (socket says 172.21.15.6:53008) 2023-11-28T13:06:47.354 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68e77fe700 1 -- 172.21.15.6:0/3763559159 learned_addr learned my addr 172.21.15.6:0/3763559159 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:47.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68e6ffd700 1 -- 172.21.15.6:0/3763559159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e8070520 msgr2=0x7f68e80708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:47.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68e6ffd700 1 --2- 172.21.15.6:0/3763559159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e8070520 0x7f68e80708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68e6ffd700 1 -- 172.21.15.6:0/3763559159 >> v1:172.21.15.61:6789/0 conn(0x7f68e80cb690 legacy=0x7f68e815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68e6ffd700 1 -- 172.21.15.6:0/3763559159 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e815d8f0 con 0x7f68e80cb220 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.352+0000 7f68e77fe700 1 --2- 172.21.15.6:0/3763559159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e8070520 0x7f68e80708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.353+0000 7f68e6ffd700 1 --2- 172.21.15.6:0/3763559159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e80cb220 0x7f68e80b0a90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009840 tx=0x7f68dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=384ab5380054c73a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.353+0000 7f68e67fc700 1 -- 172.21.15.6:0/3763559159 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc011830 con 0x7f68e80cb220 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.353+0000 7f68e67fc700 1 -- 172.21.15.6:0/3763559159 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc011990 con 0x7f68e80cb220 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.353+0000 7f68e67fc700 1 -- 172.21.15.6:0/3763559159 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc011d20 con 0x7f68e80cb220 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.354+0000 7f68ee069700 1 -- 172.21.15.6:0/3763559159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e80cb220 msgr2=0x7f68e80b0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.354+0000 7f68ee069700 1 --2- 172.21.15.6:0/3763559159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e80cb220 0x7f68e80b0a90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009840 tx=0x7f68dc009b50 comp rx=0 tx=0).stop 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.354+0000 7f68ee069700 1 -- 172.21.15.6:0/3763559159 shutdown_connections 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.354+0000 7f68ee069700 1 --2- 172.21.15.6:0/3763559159 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e8070520 0x7f68e80708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.354+0000 7f68ee069700 1 --2- 172.21.15.6:0/3763559159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e80cb220 0x7f68e80b0a90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.354+0000 7f68ee069700 1 -- 172.21.15.6:0/3763559159 >> 172.21.15.6:0/3763559159 conn(0x7f68e80d0bd0 msgr2=0x7f68e80d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.354+0000 7f68ee069700 1 -- 172.21.15.6:0/3763559159 shutdown_connections 2023-11-28T13:06:47.356 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.354+0000 7f68ee069700 1 -- 172.21.15.6:0/3763559159 wait complete. 2023-11-28T13:06:47.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.355+0000 7f68ee069700 1 Processor -- start 2023-11-28T13:06:47.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.355+0000 7f68ee069700 1 -- start start 2023-11-28T13:06:47.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68ee069700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68e8070520 0x7f68e80c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68ee069700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e80cb220 0x7f68e806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68ee069700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e80cb690 0x7f68e806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68ee069700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8147240 con 0x7f68e8070520 2023-11-28T13:06:47.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68ee069700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8147520 con 0x7f68e80cb220 2023-11-28T13:06:47.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68ee069700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8147800 con 0x7f68e80cb690 2023-11-28T13:06:47.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68e77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68e8070520 0x7f68e80c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68e77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68e8070520 0x7f68e80c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35078/0 (socket says 172.21.15.6:35078) 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68e77fe700 1 -- 172.21.15.6:0/3402666537 learned_addr learned my addr 172.21.15.6:0/3402666537 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68e6ffd700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e80cb220 0x7f68e806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68e7fff700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e80cb690 0x7f68e806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68e77fe700 1 -- 172.21.15.6:0/3402666537 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e80cb690 msgr2=0x7f68e806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68e77fe700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e80cb690 0x7f68e806f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68e77fe700 1 -- 172.21.15.6:0/3402666537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e80cb220 msgr2=0x7f68e806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68e77fe700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e80cb220 0x7f68e806eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.356+0000 7f68e77fe700 1 -- 172.21.15.6:0/3402666537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d8009550 con 0x7f68e8070520 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.357+0000 7f68e77fe700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68e8070520 0x7f68e80c7ca0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f68d800ebe0 tx=0x7f68d800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.357+0000 7f68cffff700 1 -- 172.21.15.6:0/3402666537 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d800c850 con 0x7f68e8070520 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.357+0000 7f68cffff700 1 -- 172.21.15.6:0/3402666537 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d800c9b0 con 0x7f68e8070520 2023-11-28T13:06:47.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.357+0000 7f68cffff700 1 -- 172.21.15.6:0/3402666537 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d800cd40 con 0x7f68e8070520 2023-11-28T13:06:47.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.357+0000 7f68ee069700 1 -- 172.21.15.6:0/3402666537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68dc0094a0 con 0x7f68e8070520 2023-11-28T13:06:47.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.357+0000 7f68ee069700 1 -- 172.21.15.6:0/3402666537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e80b0570 con 0x7f68e8070520 2023-11-28T13:06:47.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.358+0000 7f68ee069700 1 -- 172.21.15.6:0/3402666537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e80b0290 con 0x7f68e8070520 2023-11-28T13:06:47.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.358+0000 7f68cffff700 1 -- 172.21.15.6:0/3402666537 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f68d8019030 con 0x7f68e8070520 2023-11-28T13:06:47.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.359+0000 7f68cffff700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f68d40682f0 0x7f68d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:47.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.359+0000 7f68cffff700 1 -- 172.21.15.6:0/3402666537 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f68d8085e00 con 0x7f68e8070520 2023-11-28T13:06:47.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.359+0000 7f68e6ffd700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f68d40682f0 0x7f68d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:47.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.359+0000 7f68e6ffd700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f68d40682f0 0x7f68d406a7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f68dc000c00 tx=0x7f68dc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.363+0000 7f68cffff700 1 -- 172.21.15.6:0/3402666537 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f68d80524e0 con 0x7f68e8070520 2023-11-28T13:06:47.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.602+0000 7f68ee069700 1 -- 172.21.15.6:0/3402666537 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f68e80b2360 con 0x7f68e8070520 2023-11-28T13:06:47.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.602+0000 7f68cffff700 1 -- 172.21.15.6:0/3402666537 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f68d80170c0 con 0x7f68e8070520 2023-11-28T13:06:47.604 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:47.604 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 -- 172.21.15.6:0/3402666537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f68d40682f0 msgr2=0x7f68d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f68d40682f0 0x7f68d406a7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f68dc000c00 tx=0x7f68dc008030 comp rx=0 tx=0).stop 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 -- 172.21.15.6:0/3402666537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68e8070520 msgr2=0x7f68e80c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68e8070520 0x7f68e80c7ca0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f68d800ebe0 tx=0x7f68d800eef0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 -- 172.21.15.6:0/3402666537 shutdown_connections 2023-11-28T13:06:47.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f68d40682f0 0x7f68d406a7a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f68e80cb690 0x7f68e806f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f68e8070520 0x7f68e80c7ca0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 --2- 172.21.15.6:0/3402666537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68e80cb220 0x7f68e806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:47.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 -- 172.21.15.6:0/3402666537 >> 172.21.15.6:0/3402666537 conn(0x7f68e80d0bd0 msgr2=0x7f68e806de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:47.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 -- 172.21.15.6:0/3402666537 shutdown_connections 2023-11-28T13:06:47.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:47.604+0000 7f68ee069700 1 -- 172.21.15.6:0/3402666537 wait complete. 2023-11-28T13:06:47.618 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:52.619 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:52.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.756+0000 7fe8697d7700 1 Processor -- start 2023-11-28T13:06:52.757 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.756+0000 7fe8697d7700 1 -- start start 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe8697d7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe8640b8640 0x7fe8640cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe8697d7700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8640cd9f0 con 0x7fe86406c810 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe8697d7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86415cb30 con 0x7fe8640ccee0 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe8697d7700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8640b8a10 con 0x7fe8640b8640 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe862ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe8640b8640 0x7fe8640cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe8637fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe86406c810 0x7fe8640d0a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50642/0 (socket says 172.21.15.6:50642) 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe8637fe700 1 -- 172.21.15.6:0/3489521708 learned_addr learned my addr 172.21.15.6:0/3489521708 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe861ffb700 1 -- 172.21.15.6:0/3489521708 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227904256 0 0) 0x7fe8640cd9f0 con 0x7fe86406c810 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe861ffb700 1 -- 172.21.15.6:0/3489521708 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe84c0035b0 con 0x7fe86406c810 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe862ffd700 1 -- 172.21.15.6:0/3489521708 >> v1:172.21.15.61:6789/0 conn(0x7fe8640ccee0 legacy=0x7fe86406c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe862ffd700 1 -- 172.21.15.6:0/3489521708 >> v1:172.21.15.6:6789/0 conn(0x7fe86406c810 legacy=0x7fe8640d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.757+0000 7fe862ffd700 1 -- 172.21.15.6:0/3489521708 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86406abf0 con 0x7fe8640b8640 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe862ffd700 1 --2- 172.21.15.6:0/3489521708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 0x7fe8640cf010 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe858009840 tx=0x7fe858009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901c761f38ae6153 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe861ffb700 1 -- 172.21.15.6:0/3489521708 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858011830 con 0x7fe8640b8640 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe861ffb700 1 -- 172.21.15.6:0/3489521708 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe858011990 con 0x7fe8640b8640 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe861ffb700 1 -- 172.21.15.6:0/3489521708 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858011d20 con 0x7fe8640b8640 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe8697d7700 1 -- 172.21.15.6:0/3489521708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 msgr2=0x7fe8640cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe8697d7700 1 --2- 172.21.15.6:0/3489521708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 0x7fe8640cf010 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe858009840 tx=0x7fe858009b50 comp rx=0 tx=0).stop 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe8697d7700 1 -- 172.21.15.6:0/3489521708 shutdown_connections 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe8697d7700 1 --2- 172.21.15.6:0/3489521708 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 0x7fe8640cf010 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe8697d7700 1 -- 172.21.15.6:0/3489521708 >> 172.21.15.6:0/3489521708 conn(0x7fe8640d0460 msgr2=0x7fe8640d2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe8697d7700 1 -- 172.21.15.6:0/3489521708 shutdown_connections 2023-11-28T13:06:52.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.758+0000 7fe8697d7700 1 -- 172.21.15.6:0/3489521708 wait complete. 2023-11-28T13:06:52.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.760+0000 7fe8697d7700 1 Processor -- start 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.760+0000 7fe8697d7700 1 -- start start 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.760+0000 7fe8697d7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe86406c810 0x7fe8640c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.760+0000 7fe8697d7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 0x7fe8640b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.760+0000 7fe8697d7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8640ccee0 0x7fe8641f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8697d7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640d24b0 con 0x7fe8640b8640 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8697d7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640d2020 con 0x7fe8640ccee0 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8697d7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640d1430 con 0x7fe86406c810 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8627fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 0x7fe8640b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8627fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 0x7fe8640b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35094/0 (socket says 172.21.15.6:35094) 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8627fc700 1 -- 172.21.15.6:0/1141567040 learned_addr learned my addr 172.21.15.6:0/1141567040 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8637fe700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8640ccee0 0x7fe8641f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe862ffd700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe86406c810 0x7fe8640c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8627fc700 1 -- 172.21.15.6:0/1141567040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe86406c810 msgr2=0x7fe8640c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8627fc700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe86406c810 0x7fe8640c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8627fc700 1 -- 172.21.15.6:0/1141567040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8640ccee0 msgr2=0x7fe8641f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8627fc700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8640ccee0 0x7fe8641f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8627fc700 1 -- 172.21.15.6:0/1141567040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580094a0 con 0x7fe8640b8640 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8627fc700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 0x7fe8640b4c60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe85400c450 tx=0x7fe85400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe84b7fe700 1 -- 172.21.15.6:0/1141567040 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8540052d0 con 0x7fe8640b8640 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe84b7fe700 1 -- 172.21.15.6:0/1141567040 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe854005430 con 0x7fe8640b8640 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8697d7700 1 -- 172.21.15.6:0/1141567040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86406ae40 con 0x7fe8640b8640 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe84b7fe700 1 -- 172.21.15.6:0/1141567040 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8540038c0 con 0x7fe8640b8640 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.761+0000 7fe8697d7700 1 -- 172.21.15.6:0/1141567040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8640c9c00 con 0x7fe8640b8640 2023-11-28T13:06:52.765 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.762+0000 7fe8697d7700 1 -- 172.21.15.6:0/1141567040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86406b120 con 0x7fe8640b8640 2023-11-28T13:06:52.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.763+0000 7fe84b7fe700 1 -- 172.21.15.6:0/1141567040 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe854019030 con 0x7fe8640b8640 2023-11-28T13:06:52.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.763+0000 7fe84b7fe700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8500682f0 0x7fe85006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:52.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.763+0000 7fe84b7fe700 1 -- 172.21.15.6:0/1141567040 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fe854084f80 con 0x7fe8640b8640 2023-11-28T13:06:52.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.763+0000 7fe862ffd700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8500682f0 0x7fe85006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:52.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.764+0000 7fe862ffd700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8500682f0 0x7fe85006a7a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe8580090b0 tx=0x7fe858008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:52.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:52.768+0000 7fe84b7fe700 1 -- 172.21.15.6:0/1141567040 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe854051660 con 0x7fe8640b8640 2023-11-28T13:06:53.006 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.005+0000 7fe8697d7700 1 -- 172.21.15.6:0/1141567040 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe86415cb10 con 0x7fe8640b8640 2023-11-28T13:06:53.006 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.005+0000 7fe84b7fe700 1 -- 172.21.15.6:0/1141567040 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fe854054c80 con 0x7fe8640b8640 2023-11-28T13:06:53.006 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:53.007 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:53.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 -- 172.21.15.6:0/1141567040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8500682f0 msgr2=0x7fe85006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:53.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8500682f0 0x7fe85006a7a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe8580090b0 tx=0x7fe858008030 comp rx=0 tx=0).stop 2023-11-28T13:06:53.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 -- 172.21.15.6:0/1141567040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 msgr2=0x7fe8640b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:53.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 0x7fe8640b4c60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe85400c450 tx=0x7fe85400c810 comp rx=0 tx=0).stop 2023-11-28T13:06:53.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 -- 172.21.15.6:0/1141567040 shutdown_connections 2023-11-28T13:06:53.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8640ccee0 0x7fe8641f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:53.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fe8500682f0 0x7fe85006a7a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:53.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fe86406c810 0x7fe8640c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:53.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 --2- 172.21.15.6:0/1141567040 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe8640b8640 0x7fe8640b4c60 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:53.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 -- 172.21.15.6:0/1141567040 >> 172.21.15.6:0/1141567040 conn(0x7fe8640d0460 msgr2=0x7fe86415d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:53.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 -- 172.21.15.6:0/1141567040 shutdown_connections 2023-11-28T13:06:53.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:53.007+0000 7fe8697d7700 1 -- 172.21.15.6:0/1141567040 wait complete. 2023-11-28T13:06:53.020 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:06:57.376 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:06:57.379 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:06:58.021 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:06:58.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.160+0000 7fdffeb03700 1 Processor -- start 2023-11-28T13:06:58.160 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.160+0000 7fdffeb03700 1 -- start start 2023-11-28T13:06:58.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdffeb03700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fdff80b4a70 0x7fdff80cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdffeb03700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80b5880 con 0x7fdff80aefe0 2023-11-28T13:06:58.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdffeb03700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80d01e0 con 0x7fdff80cdb50 2023-11-28T13:06:58.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdffeb03700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff80d04c0 con 0x7fdff80b4a70 2023-11-28T13:06:58.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdffd0a0700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fdff80aefe0 0x7fdff80cb3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50656/0 (socket says 172.21.15.6:50656) 2023-11-28T13:06:58.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdffd0a0700 1 -- 172.21.15.6:0/1538662830 learned_addr learned my addr 172.21.15.6:0/1538662830 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:06:58.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdffc89f700 1 --2- 172.21.15.6:0/1538662830 >> v2:172.21.15.61:3300/0 conn(0x7fdff80b4a70 0x7fdff80cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.164 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941882990 0 0) 0x7fdff80b5880 con 0x7fdff80aefe0 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe80035b0 con 0x7fdff80aefe0 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365125222 0 0) 0x7fdfe80035b0 con 0x7fdff80aefe0 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 >> v2:172.21.15.61:3300/0 conn(0x7fdff80b4a70 msgr2=0x7fdff80cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdff77fe700 1 --2- 172.21.15.6:0/1538662830 >> v2:172.21.15.61:3300/0 conn(0x7fdff80b4a70 0x7fdff80cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 >> v1:172.21.15.61:6789/0 conn(0x7fdff80cdb50 legacy=0x7fdff80b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff80d07a0 con 0x7fdff80aefe0 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fdff00019f0 con 0x7fdff80aefe0 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.161+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 >> v1:172.21.15.6:6789/0 conn(0x7fdff80aefe0 legacy=0x7fdff80cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff77fe700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdfe8003a00 0x7fdfe8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff77fe700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfe8004850 0x7fdfe8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff77fe700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8005200 0x7fdfe800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.165 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80d04c0 con 0x7fdfe8004850 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe800c080 con 0x7fdfe8003a00 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe800c240 con 0x7fdfe8005200 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff7fff700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfe8004850 0x7fdfe8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdffc89f700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdfe8003a00 0x7fdfe8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdffd0a0700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8005200 0x7fdfe800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff7fff700 1 -- 172.21.15.6:0/1538662830 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8005200 msgr2=0x7fdfe800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff7fff700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8005200 0x7fdfe800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff7fff700 1 -- 172.21.15.6:0/1538662830 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdfe8003a00 msgr2=0x7fdfe8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff7fff700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdfe8003a00 0x7fdfe8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff7fff700 1 -- 172.21.15.6:0/1538662830 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe80035b0 con 0x7fdfe8004850 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdffd0a0700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8005200 0x7fdfe800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdffd0a0700 1 -- 172.21.15.6:0/1538662830 reap_dead start 2023-11-28T13:06:58.166 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff7fff700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfe8004850 0x7fdfe8004cc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fdff80c3100 tx=0x7fdfe8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5386db5fa73ea461 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:58.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40052d0 con 0x7fdfe8004850 2023-11-28T13:06:58.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe4005430 con 0x7fdfe8004850 2023-11-28T13:06:58.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.162+0000 7fdff77fe700 1 -- 172.21.15.6:0/1538662830 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40038c0 con 0x7fdfe8004850 2023-11-28T13:06:58.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.163+0000 7fdffeb03700 1 -- 172.21.15.6:0/1538662830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfe8004850 msgr2=0x7fdfe8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.163+0000 7fdffeb03700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfe8004850 0x7fdfe8004cc0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fdff80c3100 tx=0x7fdfe8005880 comp rx=0 tx=0).stop 2023-11-28T13:06:58.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.163+0000 7fdffeb03700 1 -- 172.21.15.6:0/1538662830 shutdown_connections 2023-11-28T13:06:58.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.163+0000 7fdffeb03700 1 --2- 172.21.15.6:0/1538662830 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdfe8004850 0x7fdfe8004cc0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.163+0000 7fdffeb03700 1 -- 172.21.15.6:0/1538662830 >> 172.21.15.6:0/1538662830 conn(0x7fdff80b3610 msgr2=0x7fdff80cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:58.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.163+0000 7fdffeb03700 1 -- 172.21.15.6:0/1538662830 shutdown_connections 2023-11-28T13:06:58.167 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.163+0000 7fdffeb03700 1 -- 172.21.15.6:0/1538662830 wait complete. 2023-11-28T13:06:58.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.164+0000 7fdffeb03700 1 Processor -- start 2023-11-28T13:06:58.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.164+0000 7fdffeb03700 1 -- start start 2023-11-28T13:06:58.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.165+0000 7fdffeb03700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8004850 0x7fdff809b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.165+0000 7fdffeb03700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff80c9a30 0x7fdff81d8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.170 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.165+0000 7fdffeb03700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdff81d45e0 0x7fdff81d1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.165+0000 7fdffeb03700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff81d22e0 con 0x7fdff80c9a30 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.165+0000 7fdffeb03700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff81d24c0 con 0x7fdff81d45e0 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.165+0000 7fdffeb03700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff81d27a0 con 0x7fdfe8004850 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.165+0000 7fdff7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff80c9a30 0x7fdff81d8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.165+0000 7fdff7fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff80c9a30 0x7fdff81d8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35114/0 (socket says 172.21.15.6:35114) 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.165+0000 7fdff7fff700 1 -- 172.21.15.6:0/2362260143 learned_addr learned my addr 172.21.15.6:0/2362260143 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdffc89f700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8004850 0x7fdff809b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdffd0a0700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdff81d45e0 0x7fdff81d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdff7fff700 1 -- 172.21.15.6:0/2362260143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8004850 msgr2=0x7fdff809b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdff7fff700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8004850 0x7fdff809b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdff7fff700 1 -- 172.21.15.6:0/2362260143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdff81d45e0 msgr2=0x7fdff81d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdff7fff700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdff81d45e0 0x7fdff81d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.171 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdff7fff700 1 -- 172.21.15.6:0/2362260143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff80aec60 con 0x7fdff80c9a30 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdffc89f700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8004850 0x7fdff809b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdff7fff700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff80c9a30 0x7fdff81d8380 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005040 tx=0x7fdfe4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdff57fa700 1 -- 172.21.15.6:0/2362260143 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4001290 con 0x7fdff80c9a30 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdff57fa700 1 -- 172.21.15.6:0/2362260143 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe401d480 con 0x7fdff80c9a30 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdff57fa700 1 -- 172.21.15.6:0/2362260143 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe401d830 con 0x7fdff80c9a30 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdffeb03700 1 -- 172.21.15.6:0/2362260143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff80aead0 con 0x7fdff80c9a30 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.166+0000 7fdffeb03700 1 -- 172.21.15.6:0/2362260143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff80cc100 con 0x7fdff80c9a30 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.167+0000 7fdffeb03700 1 -- 172.21.15.6:0/2362260143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff8066c60 con 0x7fdff80c9a30 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.167+0000 7fdff57fa700 1 -- 172.21.15.6:0/2362260143 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdfe402f070 con 0x7fdff80c9a30 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.168+0000 7fdff57fa700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdfdc068320 0x7fdfdc06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.168+0000 7fdff57fa700 1 -- 172.21.15.6:0/2362260143 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fdfe4086ba0 con 0x7fdff80c9a30 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.168+0000 7fdffc89f700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdfdc068320 0x7fdfdc06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:06:58.172 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.168+0000 7fdffc89f700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdfdc068320 0x7fdfdc06a7d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009bd0 tx=0x7fdfec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:06:58.173 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.172+0000 7fdff57fa700 1 -- 172.21.15.6:0/2362260143 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdfe4053190 con 0x7fdff80c9a30 2023-11-28T13:06:58.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.409+0000 7fdffeb03700 1 -- 172.21.15.6:0/2362260143 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdff80675a0 con 0x7fdff80c9a30 2023-11-28T13:06:58.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.409+0000 7fdff57fa700 1 -- 172.21.15.6:0/2362260143 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fdfe402d0c0 con 0x7fdff80c9a30 2023-11-28T13:06:58.410 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:06:58.410 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 -- 172.21.15.6:0/2362260143 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdfdc068320 msgr2=0x7fdfdc06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdfdc068320 0x7fdfdc06a7d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009bd0 tx=0x7fdfec008040 comp rx=0 tx=0).stop 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 -- 172.21.15.6:0/2362260143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff80c9a30 msgr2=0x7fdff81d8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff80c9a30 0x7fdff81d8380 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005040 tx=0x7fdfe4001120 comp rx=0 tx=0).stop 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 -- 172.21.15.6:0/2362260143 shutdown_connections 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdff81d45e0 0x7fdff81d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fdfdc068320 0x7fdfdc06a7d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fdfe8004850 0x7fdff809b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 --2- 172.21.15.6:0/2362260143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdff80c9a30 0x7fdff81d8380 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 -- 172.21.15.6:0/2362260143 >> 172.21.15.6:0/2362260143 conn(0x7fdff80b3610 msgr2=0x7fdff80af830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:06:58.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 -- 172.21.15.6:0/2362260143 shutdown_connections 2023-11-28T13:06:58.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:06:58.411+0000 7fdffeb03700 1 -- 172.21.15.6:0/2362260143 wait complete. 2023-11-28T13:06:58.423 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:07:03.424 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:07:03.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.561+0000 7f0aedb34700 1 Processor -- start 2023-11-28T13:07:03.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.561+0000 7f0aedb34700 1 -- start start 2023-11-28T13:07:03.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0aedb34700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0ae806f120 0x7f0ae80ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0aedb34700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 0x7f0ae80cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0aedb34700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae80b2460 con 0x7f0ae815c8a0 2023-11-28T13:07:03.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0aedb34700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae80acfe0 con 0x7f0ae806f120 2023-11-28T13:07:03.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0aedb34700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80d0280 con 0x7f0ae80acba0 2023-11-28T13:07:03.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0ae6ffd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0ae806f120 0x7f0ae80ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0ae7fff700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 0x7f0ae80cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0ae77fe700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f0ae815c8a0 0x7f0ae80b1d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:40702/0 (socket says 172.21.15.6:40702) 2023-11-28T13:07:03.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0ae77fe700 1 -- 172.21.15.6:0/1959242058 learned_addr learned my addr 172.21.15.6:0/1959242058 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:03.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0ae6ffd700 1 --2- 172.21.15.6:0/1959242058 >> v2:172.21.15.61:3300/0 conn(0x7f0ae806f120 0x7f0ae80ac660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:38840/0 (socket says 172.21.15.6:38840) 2023-11-28T13:07:03.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.562+0000 7f0ae7fff700 1 --2- 172.21.15.6:0/1959242058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 0x7f0ae80cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3301/0 says I am v2:172.21.15.6:53060/0 (socket says 172.21.15.6:53060) 2023-11-28T13:07:03.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae67fc700 1 -- 172.21.15.6:0/1959242058 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275199614 0 0) 0x7f0ae80b2460 con 0x7f0ae815c8a0 2023-11-28T13:07:03.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae67fc700 1 -- 172.21.15.6:0/1959242058 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ad40035b0 con 0x7f0ae815c8a0 2023-11-28T13:07:03.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae6ffd700 1 -- 172.21.15.6:0/1959242058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 msgr2=0x7f0ae80cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae6ffd700 1 --2- 172.21.15.6:0/1959242058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 0x7f0ae80cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae6ffd700 1 -- 172.21.15.6:0/1959242058 >> v1:172.21.15.61:6789/0 conn(0x7f0ae815c8a0 legacy=0x7f0ae80b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae6ffd700 1 -- 172.21.15.6:0/1959242058 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae80d0560 con 0x7f0ae806f120 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae7fff700 1 --2- 172.21.15.6:0/1959242058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 0x7f0ae80cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae6ffd700 1 --2- 172.21.15.6:0/1959242058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae806f120 0x7f0ae80ac660 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009840 tx=0x7f0ad0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b71be76fb581ff64 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae67fc700 1 -- 172.21.15.6:0/1959242058 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0011830 con 0x7f0ae806f120 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae67fc700 1 -- 172.21.15.6:0/1959242058 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad0011990 con 0x7f0ae806f120 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.563+0000 7f0ae67fc700 1 -- 172.21.15.6:0/1959242058 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0011d20 con 0x7f0ae806f120 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.564+0000 7f0aedb34700 1 -- 172.21.15.6:0/1959242058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae806f120 msgr2=0x7f0ae80ac660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.564+0000 7f0aedb34700 1 --2- 172.21.15.6:0/1959242058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae806f120 0x7f0ae80ac660 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009840 tx=0x7f0ad0009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.564+0000 7f0aedb34700 1 -- 172.21.15.6:0/1959242058 shutdown_connections 2023-11-28T13:07:03.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.564+0000 7f0aedb34700 1 --2- 172.21.15.6:0/1959242058 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 0x7f0ae80cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.564+0000 7f0aedb34700 1 --2- 172.21.15.6:0/1959242058 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae806f120 0x7f0ae80ac660 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.564+0000 7f0aedb34700 1 -- 172.21.15.6:0/1959242058 >> 172.21.15.6:0/1959242058 conn(0x7f0ae80b3610 msgr2=0x7f0ae80cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:03.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.564+0000 7f0aedb34700 1 -- 172.21.15.6:0/1959242058 shutdown_connections 2023-11-28T13:07:03.568 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.564+0000 7f0aedb34700 1 -- 172.21.15.6:0/1959242058 wait complete. 2023-11-28T13:07:03.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.565+0000 7f0aedb34700 1 Processor -- start 2023-11-28T13:07:03.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.565+0000 7f0aedb34700 1 -- start start 2023-11-28T13:07:03.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0aedb34700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ae806f120 0x7f0ae81f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0aedb34700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 0x7f0ae80c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0aedb34700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae815c8a0 0x7f0ae8149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0aedb34700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80c7010 con 0x7f0ae806f120 2023-11-28T13:07:03.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0aedb34700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80c72f0 con 0x7f0ae815c8a0 2023-11-28T13:07:03.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0aedb34700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae80c75d0 con 0x7f0ae80acba0 2023-11-28T13:07:03.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0ae77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ae806f120 0x7f0ae81f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0ae77fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ae806f120 0x7f0ae81f2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35126/0 (socket says 172.21.15.6:35126) 2023-11-28T13:07:03.571 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0ae77fe700 1 -- 172.21.15.6:0/2003650259 learned_addr learned my addr 172.21.15.6:0/2003650259 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0ae6ffd700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 0x7f0ae80c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0ae7fff700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae815c8a0 0x7f0ae8149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0ae77fe700 1 -- 172.21.15.6:0/2003650259 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 msgr2=0x7f0ae80c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0ae77fe700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 0x7f0ae80c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0ae77fe700 1 -- 172.21.15.6:0/2003650259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae815c8a0 msgr2=0x7f0ae8149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0ae77fe700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae815c8a0 0x7f0ae8149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.566+0000 7f0ae77fe700 1 -- 172.21.15.6:0/2003650259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009a70 con 0x7f0ae806f120 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.567+0000 7f0ae77fe700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ae806f120 0x7f0ae81f2b20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8001eb0 tx=0x7f0ad800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.567+0000 7f0adffff700 1 -- 172.21.15.6:0/2003650259 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800cea0 con 0x7f0ae806f120 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.567+0000 7f0adffff700 1 -- 172.21.15.6:0/2003650259 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad8005d70 con 0x7f0ae806f120 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.567+0000 7f0aedb34700 1 -- 172.21.15.6:0/2003650259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad00094a0 con 0x7f0ae806f120 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.567+0000 7f0adffff700 1 -- 172.21.15.6:0/2003650259 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80072f0 con 0x7f0ae806f120 2023-11-28T13:07:03.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.567+0000 7f0aedb34700 1 -- 172.21.15.6:0/2003650259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae80b1550 con 0x7f0ae806f120 2023-11-28T13:07:03.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.568+0000 7f0aedb34700 1 -- 172.21.15.6:0/2003650259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae80d0840 con 0x7f0ae806f120 2023-11-28T13:07:03.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.568+0000 7f0adffff700 1 -- 172.21.15.6:0/2003650259 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0ad8019030 con 0x7f0ae806f120 2023-11-28T13:07:03.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.569+0000 7f0adffff700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ac80682f0 0x7f0ac806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:03.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.569+0000 7f0adffff700 1 -- 172.21.15.6:0/2003650259 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f0ad80850b0 con 0x7f0ae806f120 2023-11-28T13:07:03.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.569+0000 7f0ae6ffd700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ac80682f0 0x7f0ac806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:03.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.569+0000 7f0ae6ffd700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ac80682f0 0x7f0ac806a7a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00090b0 tx=0x7f0ad0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:03.574 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.573+0000 7f0adffff700 1 -- 172.21.15.6:0/2003650259 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0ad8051790 con 0x7f0ae806f120 2023-11-28T13:07:03.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.811+0000 7f0aedb34700 1 -- 172.21.15.6:0/2003650259 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0ae80b1830 con 0x7f0ae806f120 2023-11-28T13:07:03.813 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.811+0000 7f0adffff700 1 -- 172.21.15.6:0/2003650259 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f0ad80170c0 con 0x7f0ae806f120 2023-11-28T13:07:03.813 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:03.813 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:07:03.815 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 -- 172.21.15.6:0/2003650259 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ac80682f0 msgr2=0x7f0ac806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ac80682f0 0x7f0ac806a7a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00090b0 tx=0x7f0ad0008030 comp rx=0 tx=0).stop 2023-11-28T13:07:03.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 -- 172.21.15.6:0/2003650259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ae806f120 msgr2=0x7f0ae81f2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:03.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ae806f120 0x7f0ae81f2b20 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8001eb0 tx=0x7f0ad800c7b0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 -- 172.21.15.6:0/2003650259 shutdown_connections 2023-11-28T13:07:03.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0ae815c8a0 0x7f0ae8149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0ae806f120 0x7f0ae81f2b20 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.816 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0ac80682f0 0x7f0ac806a7a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 --2- 172.21.15.6:0/2003650259 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0ae80acba0 0x7f0ae80c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:03.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 -- 172.21.15.6:0/2003650259 >> 172.21.15.6:0/2003650259 conn(0x7f0ae80b3610 msgr2=0x7f0ae80b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:03.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 -- 172.21.15.6:0/2003650259 shutdown_connections 2023-11-28T13:07:03.817 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:03.813+0000 7f0aedb34700 1 -- 172.21.15.6:0/2003650259 wait complete. 2023-11-28T13:07:03.827 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:07:08.828 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:07:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.963+0000 7fab2bac7700 1 Processor -- start 2023-11-28T13:07:08.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.964+0000 7fab2bac7700 1 -- start start 2023-11-28T13:07:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.964+0000 7fab2bac7700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fab240ccfe0 0x7fab24070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.964+0000 7fab2bac7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fab24070a60 0x7fab240cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.964+0000 7fab2bac7700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240ce9b0 con 0x7fab240b6510 2023-11-28T13:07:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.964+0000 7fab2bac7700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab24070ee0 con 0x7fab24070a60 2023-11-28T13:07:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.964+0000 7fab2bac7700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab240b0d30 con 0x7fab240ccfe0 2023-11-28T13:07:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.964+0000 7fab2a064700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fab24070a60 0x7fab240cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.964+0000 7fab29863700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fab240b6510 0x7fab240cb220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50686/0 (socket says 172.21.15.6:50686) 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.964+0000 7fab29863700 1 -- 172.21.15.6:0/2160880457 learned_addr learned my addr 172.21.15.6:0/2160880457 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.965+0000 7fab29062700 1 --2- 172.21.15.6:0/2160880457 >> v2:172.21.15.61:3300/0 conn(0x7fab240ccfe0 0x7fab24070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.965+0000 7fab2a064700 1 -- 172.21.15.6:0/2160880457 >> v2:172.21.15.61:3300/0 conn(0x7fab240ccfe0 msgr2=0x7fab24070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.965+0000 7fab2a064700 1 --2- 172.21.15.6:0/2160880457 >> v2:172.21.15.61:3300/0 conn(0x7fab240ccfe0 0x7fab24070520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.965+0000 7fab2a064700 1 -- 172.21.15.6:0/2160880457 >> v1:172.21.15.6:6789/0 conn(0x7fab240b6510 legacy=0x7fab240cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.965+0000 7fab2a064700 1 -- 172.21.15.6:0/2160880457 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab240b0fd0 con 0x7fab24070a60 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.965+0000 7fab28861700 1 -- 172.21.15.6:0/2160880457 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519812726 0 0) 0x7fab240ce9b0 con 0x7fab240b6510 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.965+0000 7fab2a064700 1 --2- 172.21.15.6:0/2160880457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 0x7fab240cee40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fab240c7be0 tx=0x7fab1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e153342ab5c28d0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.965+0000 7fab28861700 1 -- 172.21.15.6:0/2160880457 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00e070 con 0x7fab24070a60 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.965+0000 7fab28861700 1 -- 172.21.15.6:0/2160880457 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c00bbd0 con 0x7fab24070a60 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.965+0000 7fab28861700 1 -- 172.21.15.6:0/2160880457 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c012430 con 0x7fab24070a60 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.966+0000 7fab2bac7700 1 -- 172.21.15.6:0/2160880457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 msgr2=0x7fab240cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:08.968 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.966+0000 7fab2bac7700 1 --2- 172.21.15.6:0/2160880457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 0x7fab240cee40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fab240c7be0 tx=0x7fab1c00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:08.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.966+0000 7fab2bac7700 1 -- 172.21.15.6:0/2160880457 shutdown_connections 2023-11-28T13:07:08.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.966+0000 7fab2bac7700 1 --2- 172.21.15.6:0/2160880457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 0x7fab240cee40 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.966+0000 7fab2bac7700 1 --2- 172.21.15.6:0/2160880457 >> v2:172.21.15.61:3300/0 conn(0x7fab240ccfe0 0x7fab24070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.966+0000 7fab2bac7700 1 -- 172.21.15.6:0/2160880457 >> 172.21.15.6:0/2160880457 conn(0x7fab240d0bd0 msgr2=0x7fab240d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:08.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.966+0000 7fab2bac7700 1 -- 172.21.15.6:0/2160880457 shutdown_connections 2023-11-28T13:07:08.969 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.966+0000 7fab2bac7700 1 -- 172.21.15.6:0/2160880457 wait complete. 2023-11-28T13:07:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.967+0000 7fab2bac7700 1 Processor -- start 2023-11-28T13:07:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.967+0000 7fab2bac7700 1 -- start start 2023-11-28T13:07:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab2bac7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 0x7fab241f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab2bac7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab240b6510 0x7fab24154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab2bac7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab240ccfe0 0x7fab2415bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab2bac7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab241f5fa0 con 0x7fab24070a60 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab2bac7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab24154970 con 0x7fab240b6510 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab2bac7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2415a9b0 con 0x7fab240ccfe0 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab29863700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 0x7fab241f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab29863700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 0x7fab241f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35140/0 (socket says 172.21.15.6:35140) 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab29863700 1 -- 172.21.15.6:0/2136948964 learned_addr learned my addr 172.21.15.6:0/2136948964 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab2a064700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab240ccfe0 0x7fab2415bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab29062700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab240b6510 0x7fab24154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab29863700 1 -- 172.21.15.6:0/2136948964 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab240ccfe0 msgr2=0x7fab2415bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab29863700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab240ccfe0 0x7fab2415bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab29863700 1 -- 172.21.15.6:0/2136948964 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab240b6510 msgr2=0x7fab24154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab29863700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab240b6510 0x7fab24154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:08.972 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.968+0000 7fab29863700 1 -- 172.21.15.6:0/2136948964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c009a70 con 0x7fab24070a60 2023-11-28T13:07:08.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.969+0000 7fab29863700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 0x7fab241f5a60 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fab200011e0 tx=0x7fab2000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:08.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.969+0000 7fab167fc700 1 -- 172.21.15.6:0/2136948964 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20004c80 con 0x7fab24070a60 2023-11-28T13:07:08.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.969+0000 7fab2bac7700 1 -- 172.21.15.6:0/2136948964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab240cf380 con 0x7fab24070a60 2023-11-28T13:07:08.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.969+0000 7fab2bac7700 1 -- 172.21.15.6:0/2136948964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab240b12b0 con 0x7fab24070a60 2023-11-28T13:07:08.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.969+0000 7fab167fc700 1 -- 172.21.15.6:0/2136948964 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab20004de0 con 0x7fab24070a60 2023-11-28T13:07:08.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.969+0000 7fab167fc700 1 -- 172.21.15.6:0/2136948964 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab200072d0 con 0x7fab24070a60 2023-11-28T13:07:08.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.970+0000 7fab2bac7700 1 -- 172.21.15.6:0/2136948964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0c0052f0 con 0x7fab24070a60 2023-11-28T13:07:08.973 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.970+0000 7fab167fc700 1 -- 172.21.15.6:0/2136948964 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fab2001a020 con 0x7fab24070a60 2023-11-28T13:07:08.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.970+0000 7fab167fc700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab080682f0 0x7fab0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:08.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.971+0000 7fab167fc700 1 -- 172.21.15.6:0/2136948964 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fab200859a0 con 0x7fab24070a60 2023-11-28T13:07:08.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.971+0000 7fab29062700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab080682f0 0x7fab0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:08.974 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.971+0000 7fab29062700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab080682f0 0x7fab0806a7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fab2415a1f0 tx=0x7fab18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:08.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:08.975+0000 7fab167fc700 1 -- 172.21.15.6:0/2136948964 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fab20052080 con 0x7fab24070a60 2023-11-28T13:07:09.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.214+0000 7fab2bac7700 1 -- 172.21.15.6:0/2136948964 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fab0c005b20 con 0x7fab24070a60 2023-11-28T13:07:09.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.215+0000 7fab167fc700 1 -- 172.21.15.6:0/2136948964 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fab2001a2b0 con 0x7fab24070a60 2023-11-28T13:07:09.216 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:09.216 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:07:09.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 -- 172.21.15.6:0/2136948964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab080682f0 msgr2=0x7fab0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:09.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab080682f0 0x7fab0806a7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fab2415a1f0 tx=0x7fab18008040 comp rx=0 tx=0).stop 2023-11-28T13:07:09.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 -- 172.21.15.6:0/2136948964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 msgr2=0x7fab241f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:09.218 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 0x7fab241f5a60 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fab200011e0 tx=0x7fab2000cae0 comp rx=0 tx=0).stop 2023-11-28T13:07:09.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 -- 172.21.15.6:0/2136948964 shutdown_connections 2023-11-28T13:07:09.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fab080682f0 0x7fab0806a7a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:09.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fab240ccfe0 0x7fab2415bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:09.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fab24070a60 0x7fab241f5a60 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:09.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 --2- 172.21.15.6:0/2136948964 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fab240b6510 0x7fab24154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:09.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 -- 172.21.15.6:0/2136948964 >> 172.21.15.6:0/2136948964 conn(0x7fab240d0bd0 msgr2=0x7fab240b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:09.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 -- 172.21.15.6:0/2136948964 shutdown_connections 2023-11-28T13:07:09.219 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:09.217+0000 7fab2bac7700 1 -- 172.21.15.6:0/2136948964 wait complete. 2023-11-28T13:07:09.230 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:07:14.231 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:07:14.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.367+0000 7f3ff8539700 1 Processor -- start 2023-11-28T13:07:14.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.367+0000 7f3ff8539700 1 -- start start 2023-11-28T13:07:14.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.368+0000 7f3ff8539700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f3ff015c8a0 0x7f3ff00b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.368+0000 7f3ff8539700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3ff00acd70 0x7f3ff00cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.368+0000 7f3ff8539700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff00d0280 con 0x7f3ff00b2290 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.368+0000 7f3ff8539700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff00d0560 con 0x7f3ff00acd70 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.368+0000 7f3ff8539700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff00d0840 con 0x7f3ff015c8a0 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff6ad6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3ff00acd70 0x7f3ff00cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff5ad4700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f3ff00b2290 0x7f3ff00ac660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50700/0 (socket says 172.21.15.6:50700) 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff5ad4700 1 -- 172.21.15.6:0/3541569438 learned_addr learned my addr 172.21.15.6:0/3541569438 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff62d5700 1 --2- 172.21.15.6:0/3541569438 >> v2:172.21.15.61:3300/0 conn(0x7f3ff015c8a0 0x7f3ff00b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff6ad6700 1 -- 172.21.15.6:0/3541569438 >> v2:172.21.15.61:3300/0 conn(0x7f3ff015c8a0 msgr2=0x7f3ff00b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff6ad6700 1 --2- 172.21.15.6:0/3541569438 >> v2:172.21.15.61:3300/0 conn(0x7f3ff015c8a0 0x7f3ff00b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff6ad6700 1 -- 172.21.15.6:0/3541569438 >> v1:172.21.15.6:6789/0 conn(0x7f3ff00b2290 legacy=0x7f3ff00ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff6ad6700 1 -- 172.21.15.6:0/3541569438 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00b1550 con 0x7f3ff00acd70 2023-11-28T13:07:14.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff6ad6700 1 --2- 172.21.15.6:0/3541569438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00acd70 0x7f3ff00cd610 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0072990 tx=0x7f3fe800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998880d5bb6528e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:14.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff52d3700 1 -- 172.21.15.6:0/3541569438 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800e070 con 0x7f3ff00acd70 2023-11-28T13:07:14.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff52d3700 1 -- 172.21.15.6:0/3541569438 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe800bbd0 con 0x7f3ff00acd70 2023-11-28T13:07:14.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.369+0000 7f3ff52d3700 1 -- 172.21.15.6:0/3541569438 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8012430 con 0x7f3ff00acd70 2023-11-28T13:07:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.370+0000 7f3ff8539700 1 -- 172.21.15.6:0/3541569438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00acd70 msgr2=0x7f3ff00cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.370+0000 7f3ff8539700 1 --2- 172.21.15.6:0/3541569438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00acd70 0x7f3ff00cd610 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0072990 tx=0x7f3fe800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.370+0000 7f3ff8539700 1 -- 172.21.15.6:0/3541569438 shutdown_connections 2023-11-28T13:07:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.370+0000 7f3ff8539700 1 --2- 172.21.15.6:0/3541569438 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00acd70 0x7f3ff00cd610 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.370+0000 7f3ff8539700 1 --2- 172.21.15.6:0/3541569438 >> v2:172.21.15.61:3300/0 conn(0x7f3ff015c8a0 0x7f3ff00b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.370+0000 7f3ff8539700 1 -- 172.21.15.6:0/3541569438 >> 172.21.15.6:0/3541569438 conn(0x7f3ff00b3610 msgr2=0x7f3ff00cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.370+0000 7f3ff8539700 1 -- 172.21.15.6:0/3541569438 shutdown_connections 2023-11-28T13:07:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.370+0000 7f3ff8539700 1 -- 172.21.15.6:0/3541569438 wait complete. 2023-11-28T13:07:14.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.372+0000 7f3ff8539700 1 Processor -- start 2023-11-28T13:07:14.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.372+0000 7f3ff8539700 1 -- start start 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff8539700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ff00acd70 0x7f3ff01464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff8539700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00b2290 0x7f3ff00cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff8539700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ff015c8a0 0x7f3ff0148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff8539700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff01f67c0 con 0x7f3ff00b2290 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff8539700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00c9940 con 0x7f3ff015c8a0 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff8539700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff01f5a00 con 0x7f3ff00acd70 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff5ad4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00b2290 0x7f3ff00cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff5ad4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00b2290 0x7f3ff00cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35154/0 (socket says 172.21.15.6:35154) 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff5ad4700 1 -- 172.21.15.6:0/4170398044 learned_addr learned my addr 172.21.15.6:0/4170398044 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff6ad6700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ff015c8a0 0x7f3ff0148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff62d5700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ff00acd70 0x7f3ff01464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff5ad4700 1 -- 172.21.15.6:0/4170398044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ff00acd70 msgr2=0x7f3ff01464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff5ad4700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ff00acd70 0x7f3ff01464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff5ad4700 1 -- 172.21.15.6:0/4170398044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ff015c8a0 msgr2=0x7f3ff0148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff5ad4700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ff015c8a0 0x7f3ff0148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff5ad4700 1 -- 172.21.15.6:0/4170398044 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8009a70 con 0x7f3ff00b2290 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff6ad6700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ff015c8a0 0x7f3ff0148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff62d5700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ff00acd70 0x7f3ff01464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3ff5ad4700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00b2290 0x7f3ff00cde60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c500 tx=0x7f3fe400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3fe2ffd700 1 -- 172.21.15.6:0/4170398044 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4005d70 con 0x7f3ff00b2290 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.373+0000 7f3fe2ffd700 1 -- 172.21.15.6:0/4170398044 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe40070d0 con 0x7f3ff00b2290 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.374+0000 7f3fe2ffd700 1 -- 172.21.15.6:0/4170398044 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4007440 con 0x7f3ff00b2290 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.374+0000 7f3ff8539700 1 -- 172.21.15.6:0/4170398044 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff015cc70 con 0x7f3ff00b2290 2023-11-28T13:07:14.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.374+0000 7f3ff8539700 1 -- 172.21.15.6:0/4170398044 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff00b1830 con 0x7f3ff00b2290 2023-11-28T13:07:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.374+0000 7f3ff8539700 1 -- 172.21.15.6:0/4170398044 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff006a6c0 con 0x7f3ff00b2290 2023-11-28T13:07:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.375+0000 7f3fe2ffd700 1 -- 172.21.15.6:0/4170398044 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3fe4019030 con 0x7f3ff00b2290 2023-11-28T13:07:14.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.375+0000 7f3fe2ffd700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3fd40682f0 0x7f3fd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:14.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.375+0000 7f3fe2ffd700 1 -- 172.21.15.6:0/4170398044 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f3fe40850b0 con 0x7f3ff00b2290 2023-11-28T13:07:14.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.375+0000 7f3ff62d5700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3fd40682f0 0x7f3fd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:14.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.376+0000 7f3ff62d5700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3fd40682f0 0x7f3fd406a7a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009820 tx=0x7f3fec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:14.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.380+0000 7f3fe2ffd700 1 -- 172.21.15.6:0/4170398044 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3fe404b5e0 con 0x7f3ff00b2290 2023-11-28T13:07:14.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.617+0000 7f3ff8539700 1 -- 172.21.15.6:0/4170398044 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3ff00675a0 con 0x7f3ff00b2290 2023-11-28T13:07:14.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.617+0000 7f3fe2ffd700 1 -- 172.21.15.6:0/4170398044 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f3fe40170c0 con 0x7f3ff00b2290 2023-11-28T13:07:14.619 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:14.619 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:07:14.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 -- 172.21.15.6:0/4170398044 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3fd40682f0 msgr2=0x7f3fd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3fd40682f0 0x7f3fd406a7a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009820 tx=0x7f3fec008040 comp rx=0 tx=0).stop 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 -- 172.21.15.6:0/4170398044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00b2290 msgr2=0x7f3ff00cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00b2290 0x7f3ff00cde60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c500 tx=0x7f3fe400c8c0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 -- 172.21.15.6:0/4170398044 shutdown_connections 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3ff015c8a0 0x7f3ff0148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3fd40682f0 0x7f3fd406a7a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3ff00acd70 0x7f3ff01464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 --2- 172.21.15.6:0/4170398044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ff00b2290 0x7f3ff00cde60 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 -- 172.21.15.6:0/4170398044 >> 172.21.15.6:0/4170398044 conn(0x7f3ff00b3610 msgr2=0x7f3ff015aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 -- 172.21.15.6:0/4170398044 shutdown_connections 2023-11-28T13:07:14.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:14.619+0000 7f3ff8539700 1 -- 172.21.15.6:0/4170398044 wait complete. 2023-11-28T13:07:14.632 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:07:19.634 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:07:19.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.770+0000 7f11aac06700 1 Processor -- start 2023-11-28T13:07:19.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.771+0000 7f11aac06700 1 -- start start 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.771+0000 7f11aac06700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f11a415c8a0 0x7f11a40b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.771+0000 7f11aac06700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a40acd70 0x7f11a40cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.771+0000 7f11aac06700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a40d0280 con 0x7f11a40b2290 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.771+0000 7f11aac06700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a40d0560 con 0x7f11a415c8a0 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.771+0000 7f11aac06700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40d0840 con 0x7f11a40acd70 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.771+0000 7f11a3fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f11a40b2290 0x7f11a40ac660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50712/0 (socket says 172.21.15.6:50712) 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.771+0000 7f11a3fff700 1 -- 172.21.15.6:0/3474876155 learned_addr learned my addr 172.21.15.6:0/3474876155 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a89a2700 1 --2- 172.21.15.6:0/3474876155 >> v2:172.21.15.61:3300/0 conn(0x7f11a415c8a0 0x7f11a40b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a91a3700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a40acd70 0x7f11a40cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494011677 0 0) 0x7f11a40d0280 con 0x7f11a40b2290 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11940035b0 con 0x7f11a40b2290 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2952338978 0 0) 0x7f11940035b0 con 0x7f11a40b2290 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a40acd70 msgr2=0x7f11a40cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a40acd70 0x7f11a40cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.780 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 >> v2:172.21.15.61:3300/0 conn(0x7f11a415c8a0 msgr2=0x7f11a40b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 --2- 172.21.15.6:0/3474876155 >> v2:172.21.15.61:3300/0 conn(0x7f11a415c8a0 0x7f11a40b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a40b1550 con 0x7f11a40b2290 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f1190001670 con 0x7f11a40b2290 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 >> v1:172.21.15.6:6789/0 conn(0x7f11a40b2290 legacy=0x7f11a40ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a40acd70 0x7f1194003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1194003f80 0x7f1194004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1194004d90 0x7f1194005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40d0560 con 0x7f1194004d90 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40d0840 con 0x7f1194003f80 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119400ba10 con 0x7f11a40acd70 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a91a3700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1194004d90 0x7f1194005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.781 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a89a2700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a40acd70 0x7f1194003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.772+0000 7f11a3fff700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1194003f80 0x7f1194004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11a91a3700 1 -- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a40acd70 msgr2=0x7f1194003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11a91a3700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a40acd70 0x7f1194003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11a91a3700 1 -- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1194003f80 msgr2=0x7f1194004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11a91a3700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1194003f80 0x7f1194004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11a91a3700 1 -- 172.21.15.6:0/3474876155 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11940035b0 con 0x7f1194004d90 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11a91a3700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1194004d90 0x7f1194005200 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f119c009f90 tx=0x7f1194005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1f692813e73f6e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c00ebd0 con 0x7f1194004d90 2023-11-28T13:07:19.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c00f040 con 0x7f1194004d90 2023-11-28T13:07:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11a37fe700 1 -- 172.21.15.6:0/3474876155 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c014480 con 0x7f1194004d90 2023-11-28T13:07:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11aac06700 1 -- 172.21.15.6:0/3474876155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1194004d90 msgr2=0x7f1194005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11aac06700 1 --2- 172.21.15.6:0/3474876155 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1194004d90 0x7f1194005200 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f119c009f90 tx=0x7f1194005880 comp rx=0 tx=0).stop 2023-11-28T13:07:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11a91a3700 1 -- 172.21.15.6:0/3474876155 reap_dead start 2023-11-28T13:07:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11aac06700 1 -- 172.21.15.6:0/3474876155 shutdown_connections 2023-11-28T13:07:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.773+0000 7f11aac06700 1 -- 172.21.15.6:0/3474876155 >> 172.21.15.6:0/3474876155 conn(0x7f11a40b3610 msgr2=0x7f11a40cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:19.783 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.774+0000 7f11aac06700 1 -- 172.21.15.6:0/3474876155 shutdown_connections 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.774+0000 7f11aac06700 1 -- 172.21.15.6:0/3474876155 wait complete. 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.775+0000 7f11aac06700 1 Processor -- start 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.775+0000 7f11aac06700 1 -- start start 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11aac06700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11a41f22c0 0x7f11a41f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11aac06700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a41f2bd0 0x7f11a41e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11aac06700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11a41e5910 0x7f11a415a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11aac06700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a415a1d0 con 0x7f11a41e5910 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11aac06700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a41e6160 con 0x7f11a41f22c0 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11aac06700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a41e6440 con 0x7f11a41f2bd0 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a91a3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11a41e5910 0x7f11a415a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a91a3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11a41e5910 0x7f11a415a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35174/0 (socket says 172.21.15.6:35174) 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a91a3700 1 -- 172.21.15.6:0/376674173 learned_addr learned my addr 172.21.15.6:0/376674173 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a89a2700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11a41f22c0 0x7f11a41f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a3fff700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a41f2bd0 0x7f11a41e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a91a3700 1 -- 172.21.15.6:0/376674173 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a41f2bd0 msgr2=0x7f11a41e53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a91a3700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a41f2bd0 0x7f11a41e53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a91a3700 1 -- 172.21.15.6:0/376674173 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11a41f22c0 msgr2=0x7f11a41f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a91a3700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11a41f22c0 0x7f11a41f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a91a3700 1 -- 172.21.15.6:0/376674173 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a415cc70 con 0x7f11a41e5910 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.776+0000 7f11a91a3700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11a41e5910 0x7f11a415a620 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f11940046d0 tx=0x7f119c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.777+0000 7f11a17fa700 1 -- 172.21.15.6:0/376674173 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c007000 con 0x7f11a41e5910 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.777+0000 7f11a17fa700 1 -- 172.21.15.6:0/376674173 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c022070 con 0x7f11a41e5910 2023-11-28T13:07:19.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.777+0000 7f11aac06700 1 -- 172.21.15.6:0/376674173 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a40b1830 con 0x7f11a41e5910 2023-11-28T13:07:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.777+0000 7f11a17fa700 1 -- 172.21.15.6:0/376674173 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c003a20 con 0x7f11a41e5910 2023-11-28T13:07:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.777+0000 7f11aac06700 1 -- 172.21.15.6:0/376674173 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a406a6c0 con 0x7f11a41e5910 2023-11-28T13:07:19.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.778+0000 7f11aac06700 1 -- 172.21.15.6:0/376674173 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a415a340 con 0x7f11a41e5910 2023-11-28T13:07:19.787 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.778+0000 7f11a17fa700 1 -- 172.21.15.6:0/376674173 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f119c003b80 con 0x7f11a41e5910 2023-11-28T13:07:19.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.779+0000 7f11a17fa700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f11880682f0 0x7f118806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:19.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.779+0000 7f11a17fa700 1 -- 172.21.15.6:0/376674173 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f119c024070 con 0x7f11a41e5910 2023-11-28T13:07:19.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.779+0000 7f11a89a2700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f11880682f0 0x7f118806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:19.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.779+0000 7f11a89a2700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f11880682f0 0x7f118806a7a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1198009ce0 tx=0x7f1198009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:19.788 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:19.783+0000 7f11a17fa700 1 -- 172.21.15.6:0/376674173 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f11a415a340 con 0x7f11a41e5910 2023-11-28T13:07:20.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.026+0000 7f11aac06700 1 -- 172.21.15.6:0/376674173 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f11a40675a0 con 0x7f11a41e5910 2023-11-28T13:07:20.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.026+0000 7f11a17fa700 1 -- 172.21.15.6:0/376674173 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f119c0540d0 con 0x7f11a41e5910 2023-11-28T13:07:20.028 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:20.028 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:07:20.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 -- 172.21.15.6:0/376674173 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f11880682f0 msgr2=0x7f118806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f11880682f0 0x7f118806a7a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1198009ce0 tx=0x7f1198009210 comp rx=0 tx=0).stop 2023-11-28T13:07:20.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 -- 172.21.15.6:0/376674173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11a41e5910 msgr2=0x7f11a415a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:20.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11a41e5910 0x7f11a415a620 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f11940046d0 tx=0x7f119c009750 comp rx=0 tx=0).stop 2023-11-28T13:07:20.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 -- 172.21.15.6:0/376674173 shutdown_connections 2023-11-28T13:07:20.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f11a41e5910 0x7f11a415a620 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.030 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11a41f22c0 0x7f11a41f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f11880682f0 0x7f118806a7a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 --2- 172.21.15.6:0/376674173 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f11a41f2bd0 0x7f11a41e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:20.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 -- 172.21.15.6:0/376674173 >> 172.21.15.6:0/376674173 conn(0x7f11a40b3610 msgr2=0x7f11a40b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:20.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 -- 172.21.15.6:0/376674173 shutdown_connections 2023-11-28T13:07:20.031 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:20.028+0000 7f11aac06700 1 -- 172.21.15.6:0/376674173 wait complete. 2023-11-28T13:07:20.042 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:07:25.043 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:07:25.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.180+0000 7f19fa274700 1 Processor -- start 2023-11-28T13:07:25.181 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.180+0000 7f19fa274700 1 -- start start 2023-11-28T13:07:25.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19fa274700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f19f40b4a70 0x7f19f40cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19fa274700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f40aefe0 0x7f19f40cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19fa274700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f40b5880 con 0x7f19f40cdb50 2023-11-28T13:07:25.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19fa274700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40d01e0 con 0x7f19f40b4a70 2023-11-28T13:07:25.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19fa274700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40d04c0 con 0x7f19f40aefe0 2023-11-28T13:07:25.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f3fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f19f40b4a70 0x7f19f40cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f37fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f19f40cdb50 0x7f19f40b4e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50732/0 (socket says 172.21.15.6:50732) 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f37fe700 1 -- 172.21.15.6:0/3014104384 learned_addr learned my addr 172.21.15.6:0/3014104384 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f8811700 1 --2- 172.21.15.6:0/3014104384 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f40aefe0 0x7f19f40cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f2ffd700 1 -- 172.21.15.6:0/3014104384 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337954225 0 0) 0x7f19f40b5880 con 0x7f19f40cdb50 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f2ffd700 1 -- 172.21.15.6:0/3014104384 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e00035b0 con 0x7f19f40cdb50 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f3fff700 1 -- 172.21.15.6:0/3014104384 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f40aefe0 msgr2=0x7f19f40cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f3fff700 1 --2- 172.21.15.6:0/3014104384 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f40aefe0 0x7f19f40cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f3fff700 1 -- 172.21.15.6:0/3014104384 >> v1:172.21.15.6:6789/0 conn(0x7f19f40cdb50 legacy=0x7f19f40b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f3fff700 1 -- 172.21.15.6:0/3014104384 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40d07a0 con 0x7f19f40b4a70 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.181+0000 7f19f3fff700 1 --2- 172.21.15.6:0/3014104384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40b4a70 0x7f19f40cd610 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009840 tx=0x7f19e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8a531a8883faae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19f2ffd700 1 -- 172.21.15.6:0/3014104384 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e400f8a0 con 0x7f19f40b4a70 2023-11-28T13:07:25.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19f2ffd700 1 -- 172.21.15.6:0/3014104384 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e400fa00 con 0x7f19f40b4a70 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19f2ffd700 1 -- 172.21.15.6:0/3014104384 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e400fd90 con 0x7f19f40b4a70 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19fa274700 1 -- 172.21.15.6:0/3014104384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40b4a70 msgr2=0x7f19f40cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19fa274700 1 --2- 172.21.15.6:0/3014104384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40b4a70 0x7f19f40cd610 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009840 tx=0x7f19e4009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19fa274700 1 -- 172.21.15.6:0/3014104384 shutdown_connections 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19fa274700 1 --2- 172.21.15.6:0/3014104384 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f40aefe0 0x7f19f40cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19fa274700 1 --2- 172.21.15.6:0/3014104384 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40b4a70 0x7f19f40cd610 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19fa274700 1 -- 172.21.15.6:0/3014104384 >> 172.21.15.6:0/3014104384 conn(0x7f19f40b3610 msgr2=0x7f19f40cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19fa274700 1 -- 172.21.15.6:0/3014104384 shutdown_connections 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.182+0000 7f19fa274700 1 -- 172.21.15.6:0/3014104384 wait complete. 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.183+0000 7f19fa274700 1 Processor -- start 2023-11-28T13:07:25.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.184+0000 7f19fa274700 1 -- start start 2023-11-28T13:07:25.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.184+0000 7f19fa274700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40aefe0 0x7f19f40c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.184+0000 7f19fa274700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f40b4a70 0x7f19f409baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.184+0000 7f19fa274700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f40cdb50 0x7f19f409c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.184+0000 7f19fa274700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f41f1cf0 con 0x7f19f40aefe0 2023-11-28T13:07:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.184+0000 7f19fa274700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40c2e40 con 0x7f19f40b4a70 2023-11-28T13:07:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.184+0000 7f19fa274700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f409a820 con 0x7f19f40cdb50 2023-11-28T13:07:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40aefe0 0x7f19f40c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40aefe0 0x7f19f40c2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35184/0 (socket says 172.21.15.6:35184) 2023-11-28T13:07:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f3fff700 1 -- 172.21.15.6:0/2027395302 learned_addr learned my addr 172.21.15.6:0/2027395302 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f37fe700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f40b4a70 0x7f19f409baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.190 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f8811700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f40cdb50 0x7f19f409c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f3fff700 1 -- 172.21.15.6:0/2027395302 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f40cdb50 msgr2=0x7f19f409c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f3fff700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f40cdb50 0x7f19f409c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f3fff700 1 -- 172.21.15.6:0/2027395302 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f40b4a70 msgr2=0x7f19f409baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f3fff700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f40b4a70 0x7f19f409baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f3fff700 1 -- 172.21.15.6:0/2027395302 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e40094a0 con 0x7f19f40aefe0 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f37fe700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f40b4a70 0x7f19f409baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f3fff700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40aefe0 0x7f19f40c2900 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f19e40090b0 tx=0x7f19e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f0ff9700 1 -- 172.21.15.6:0/2027395302 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4003ab0 con 0x7f19f40aefe0 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f0ff9700 1 -- 172.21.15.6:0/2027395302 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e4003c30 con 0x7f19f40aefe0 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19fa274700 1 -- 172.21.15.6:0/2027395302 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f40aead0 con 0x7f19f40aefe0 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19f0ff9700 1 -- 172.21.15.6:0/2027395302 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4010470 con 0x7f19f40aefe0 2023-11-28T13:07:25.191 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.185+0000 7f19fa274700 1 -- 172.21.15.6:0/2027395302 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f40b5550 con 0x7f19f40aefe0 2023-11-28T13:07:25.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.186+0000 7f19fa274700 1 -- 172.21.15.6:0/2027395302 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f40aec60 con 0x7f19f40aefe0 2023-11-28T13:07:25.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.187+0000 7f19f0ff9700 1 -- 172.21.15.6:0/2027395302 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19e4029020 con 0x7f19f40aefe0 2023-11-28T13:07:25.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.187+0000 7f19f0ff9700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d4068320 0x7f19d406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:25.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.187+0000 7f19f0ff9700 1 -- 172.21.15.6:0/2027395302 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f19e40871a0 con 0x7f19f40aefe0 2023-11-28T13:07:25.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.187+0000 7f19f37fe700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d4068320 0x7f19d406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:25.192 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.188+0000 7f19f37fe700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d4068320 0x7f19d406a7d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f19f40cd870 tx=0x7f19dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:25.193 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.192+0000 7f19f0ff9700 1 -- 172.21.15.6:0/2027395302 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19e4053930 con 0x7f19f40aefe0 2023-11-28T13:07:25.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.433+0000 7f19fa274700 1 -- 172.21.15.6:0/2027395302 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f19f4066c60 con 0x7f19f40aefe0 2023-11-28T13:07:25.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.433+0000 7f19f0ff9700 1 -- 172.21.15.6:0/2027395302 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f19e4056f50 con 0x7f19f40aefe0 2023-11-28T13:07:25.434 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:25.434 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:07:25.436 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 -- 172.21.15.6:0/2027395302 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d4068320 msgr2=0x7f19d406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d4068320 0x7f19d406a7d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f19f40cd870 tx=0x7f19dc009040 comp rx=0 tx=0).stop 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 -- 172.21.15.6:0/2027395302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40aefe0 msgr2=0x7f19f40c2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40aefe0 0x7f19f40c2900 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f19e40090b0 tx=0x7f19e4004750 comp rx=0 tx=0).stop 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 -- 172.21.15.6:0/2027395302 shutdown_connections 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19d4068320 0x7f19d406a7d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f19f40cdb50 0x7f19f409c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f19f40aefe0 0x7f19f40c2900 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 --2- 172.21.15.6:0/2027395302 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f19f40b4a70 0x7f19f409baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 -- 172.21.15.6:0/2027395302 >> 172.21.15.6:0/2027395302 conn(0x7f19f40b3610 msgr2=0x7f19f40b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 -- 172.21.15.6:0/2027395302 shutdown_connections 2023-11-28T13:07:25.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:25.435+0000 7f19fa274700 1 -- 172.21.15.6:0/2027395302 wait complete. 2023-11-28T13:07:25.449 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:07:27.438 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:27.441 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:30.449 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:07:30.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.586+0000 7f1bb3cad700 1 Processor -- start 2023-11-28T13:07:30.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.586+0000 7f1bb3cad700 1 -- start start 2023-11-28T13:07:30.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb3cad700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1bac15c8a0 0x7f1bac0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb3cad700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0acd70 0x7f1bac0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb3cad700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bac0d0280 con 0x7f1bac0b2290 2023-11-28T13:07:30.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb3cad700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bac0d0560 con 0x7f1bac15c8a0 2023-11-28T13:07:30.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb3cad700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac0d0840 con 0x7f1bac0acd70 2023-11-28T13:07:30.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb1a49700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1bac15c8a0 0x7f1bac0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb1248700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1bac0b2290 0x7f1bac0ac660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50744/0 (socket says 172.21.15.6:50744) 2023-11-28T13:07:30.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb1248700 1 -- 172.21.15.6:0/1139491356 learned_addr learned my addr 172.21.15.6:0/1139491356 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:30.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb224a700 1 --2- 172.21.15.6:0/1139491356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0acd70 0x7f1bac0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb0a47700 1 -- 172.21.15.6:0/1139491356 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008112232 0 0) 0x7f1bac0d0280 con 0x7f1bac0b2290 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb0a47700 1 -- 172.21.15.6:0/1139491356 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b980035b0 con 0x7f1bac0b2290 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb1a49700 1 -- 172.21.15.6:0/1139491356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0acd70 msgr2=0x7f1bac0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb1a49700 1 --2- 172.21.15.6:0/1139491356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0acd70 0x7f1bac0cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb1a49700 1 -- 172.21.15.6:0/1139491356 >> v1:172.21.15.6:6789/0 conn(0x7f1bac0b2290 legacy=0x7f1bac0ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.587+0000 7f1bb1a49700 1 -- 172.21.15.6:0/1139491356 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac0b1550 con 0x7f1bac15c8a0 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb1a49700 1 --2- 172.21.15.6:0/1139491356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 0x7f1bac0b1d50 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009840 tx=0x7f1b9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c90ceb77bb9e31 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb0a47700 1 -- 172.21.15.6:0/1139491356 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c011830 con 0x7f1bac15c8a0 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb0a47700 1 -- 172.21.15.6:0/1139491356 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c011990 con 0x7f1bac15c8a0 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb0a47700 1 -- 172.21.15.6:0/1139491356 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c011d20 con 0x7f1bac15c8a0 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb3cad700 1 -- 172.21.15.6:0/1139491356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 msgr2=0x7f1bac0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb3cad700 1 --2- 172.21.15.6:0/1139491356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 0x7f1bac0b1d50 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009840 tx=0x7f1b9c009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb3cad700 1 -- 172.21.15.6:0/1139491356 shutdown_connections 2023-11-28T13:07:30.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb3cad700 1 --2- 172.21.15.6:0/1139491356 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0acd70 0x7f1bac0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb3cad700 1 --2- 172.21.15.6:0/1139491356 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 0x7f1bac0b1d50 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb3cad700 1 -- 172.21.15.6:0/1139491356 >> 172.21.15.6:0/1139491356 conn(0x7f1bac0b3610 msgr2=0x7f1bac0cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb3cad700 1 -- 172.21.15.6:0/1139491356 shutdown_connections 2023-11-28T13:07:30.592 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.588+0000 7f1bb3cad700 1 -- 172.21.15.6:0/1139491356 wait complete. 2023-11-28T13:07:30.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.590+0000 7f1bb3cad700 1 Processor -- start 2023-11-28T13:07:30.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.590+0000 7f1bb3cad700 1 -- start start 2023-11-28T13:07:30.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.590+0000 7f1bb3cad700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1bac0acd70 0x7f1bac0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.590+0000 7f1bb3cad700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0b2290 0x7f1bac0cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.590+0000 7f1bb3cad700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 0x7f1bac15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.590+0000 7f1bb3cad700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac0c9d50 con 0x7f1bac15c8a0 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.590+0000 7f1bb3cad700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac155610 con 0x7f1bac0acd70 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.590+0000 7f1bb3cad700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac0c9fa0 con 0x7f1bac0b2290 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.590+0000 7f1bb224a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 0x7f1bac15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb224a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 0x7f1bac15ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35202/0 (socket says 172.21.15.6:35202) 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb224a700 1 -- 172.21.15.6:0/3602671540 learned_addr learned my addr 172.21.15.6:0/3602671540 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb1a49700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1bac0acd70 0x7f1bac0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb1248700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0b2290 0x7f1bac0cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb224a700 1 -- 172.21.15.6:0/3602671540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0b2290 msgr2=0x7f1bac0cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb224a700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0b2290 0x7f1bac0cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb224a700 1 -- 172.21.15.6:0/3602671540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1bac0acd70 msgr2=0x7f1bac0cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb224a700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1bac0acd70 0x7f1bac0cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb224a700 1 -- 172.21.15.6:0/3602671540 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c0094a0 con 0x7f1bac15c8a0 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb1248700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0b2290 0x7f1bac0cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb224a700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 0x7f1bac15ad50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400efc0 tx=0x7f1ba400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1ba27fc700 1 -- 172.21.15.6:0/3602671540 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4009ca0 con 0x7f1bac15c8a0 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1ba27fc700 1 -- 172.21.15.6:0/3602671540 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba400f040 con 0x7f1bac15c8a0 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb3cad700 1 -- 172.21.15.6:0/3602671540 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bac15cc70 con 0x7f1bac15c8a0 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1ba27fc700 1 -- 172.21.15.6:0/3602671540 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4014480 con 0x7f1bac15c8a0 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.591+0000 7f1bb3cad700 1 -- 172.21.15.6:0/3602671540 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac0b1830 con 0x7f1bac15c8a0 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.592+0000 7f1bb3cad700 1 -- 172.21.15.6:0/3602671540 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bac06a6c0 con 0x7f1bac15c8a0 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.593+0000 7f1ba27fc700 1 -- 172.21.15.6:0/3602671540 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1ba40145e0 con 0x7f1bac15c8a0 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.593+0000 7f1ba27fc700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b900682f0 0x7f1b9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:30.596 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.593+0000 7f1ba27fc700 1 -- 172.21.15.6:0/3602671540 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f1ba4086cd0 con 0x7f1bac15c8a0 2023-11-28T13:07:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.593+0000 7f1bb1a49700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b900682f0 0x7f1b9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:30.597 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.594+0000 7f1bb1a49700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b900682f0 0x7f1b9006a7a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000c00 tx=0x7f1b9c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:30.598 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.598+0000 7f1ba27fc700 1 -- 172.21.15.6:0/3602671540 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1ba40533b0 con 0x7f1bac15c8a0 2023-11-28T13:07:30.840 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.839+0000 7f1bb3cad700 1 -- 172.21.15.6:0/3602671540 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1bac0675a0 con 0x7f1bac15c8a0 2023-11-28T13:07:30.841 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.839+0000 7f1ba27fc700 1 -- 172.21.15.6:0/3602671540 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f1ba40180c0 con 0x7f1bac15c8a0 2023-11-28T13:07:30.841 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:30.841 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:07:30.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.841+0000 7f1bb3cad700 1 -- 172.21.15.6:0/3602671540 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b900682f0 msgr2=0x7f1b9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.841+0000 7f1bb3cad700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b900682f0 0x7f1b9006a7a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c000c00 tx=0x7f1b9c01c040 comp rx=0 tx=0).stop 2023-11-28T13:07:30.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.841+0000 7f1bb3cad700 1 -- 172.21.15.6:0/3602671540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 msgr2=0x7f1bac15ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:30.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.841+0000 7f1bb3cad700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 0x7f1bac15ad50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400efc0 tx=0x7f1ba400c3a0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.841+0000 7f1bb3cad700 1 -- 172.21.15.6:0/3602671540 shutdown_connections 2023-11-28T13:07:30.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.841+0000 7f1bb3cad700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1bac15c8a0 0x7f1bac15ad50 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.841+0000 7f1bb3cad700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1bac0acd70 0x7f1bac0cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.841+0000 7f1bb3cad700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f1b900682f0 0x7f1b9006a7a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.841+0000 7f1bb3cad700 1 --2- 172.21.15.6:0/3602671540 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1bac0b2290 0x7f1bac0cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:30.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.841+0000 7f1bb3cad700 1 -- 172.21.15.6:0/3602671540 >> 172.21.15.6:0/3602671540 conn(0x7f1bac0b3610 msgr2=0x7f1bac0ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:30.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.842+0000 7f1bb3cad700 1 -- 172.21.15.6:0/3602671540 shutdown_connections 2023-11-28T13:07:30.844 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:30.842+0000 7f1bb3cad700 1 -- 172.21.15.6:0/3602671540 wait complete. 2023-11-28T13:07:30.855 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=200 retry_count=0)... 2023-11-28T13:07:35.856 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T13:07:35.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.992+0000 7f0911ee9700 1 Processor -- start 2023-11-28T13:07:35.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.992+0000 7f0911ee9700 1 -- start start 2023-11-28T13:07:35.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.993+0000 7f0911ee9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f090c0cd280 0x7f090c0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:35.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.993+0000 7f0911ee9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f090c0c4a80 0x7f090c0c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:35.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.993+0000 7f0911ee9700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c0cf3a0 con 0x7f090c0b26d0 2023-11-28T13:07:35.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.993+0000 7f0911ee9700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c0cf680 con 0x7f090c0cd280 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.993+0000 7f0911ee9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c0cf960 con 0x7f090c0c4a80 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.993+0000 7f090affd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f090c0cd280 0x7f090c0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.993+0000 7f090b7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f090c0b26d0 0x7f090c0b2aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50758/0 (socket says 172.21.15.6:50758) 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.993+0000 7f090b7fe700 1 -- 172.21.15.6:0/3605604220 learned_addr learned my addr 172.21.15.6:0/3605604220 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090a7fc700 1 -- 172.21.15.6:0/3605604220 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119210347 0 0) 0x7f090c0cf3a0 con 0x7f090c0b26d0 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090bfff700 1 --2- 172.21.15.6:0/3605604220 >> v2:172.21.15.61:3300/0 conn(0x7f090c0c4a80 0x7f090c0c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090a7fc700 1 -- 172.21.15.6:0/3605604220 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08f80035b0 con 0x7f090c0b26d0 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090affd700 1 -- 172.21.15.6:0/3605604220 >> v2:172.21.15.61:3300/0 conn(0x7f090c0c4a80 msgr2=0x7f090c0c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090affd700 1 --2- 172.21.15.6:0/3605604220 >> v2:172.21.15.61:3300/0 conn(0x7f090c0c4a80 0x7f090c0c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090affd700 1 -- 172.21.15.6:0/3605604220 >> v1:172.21.15.6:6789/0 conn(0x7f090c0b26d0 legacy=0x7f090c0b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090affd700 1 -- 172.21.15.6:0/3605604220 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c0cfc40 con 0x7f090c0cd280 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090affd700 1 --2- 172.21.15.6:0/3605604220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 0x7f090c0cea30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f090c0b31b0 tx=0x7f08f4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c53d59796942d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:35.996 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090a7fc700 1 -- 172.21.15.6:0/3605604220 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4005e30 con 0x7f090c0cd280 2023-11-28T13:07:35.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090a7fc700 1 -- 172.21.15.6:0/3605604220 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f4006030 con 0x7f090c0cd280 2023-11-28T13:07:35.997 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.994+0000 7f090a7fc700 1 -- 172.21.15.6:0/3605604220 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f40063a0 con 0x7f090c0cd280 2023-11-28T13:07:35.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.995+0000 7f0911ee9700 1 -- 172.21.15.6:0/3605604220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 msgr2=0x7f090c0cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:35.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.995+0000 7f0911ee9700 1 --2- 172.21.15.6:0/3605604220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 0x7f090c0cea30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f090c0b31b0 tx=0x7f08f4009b00 comp rx=0 tx=0).stop 2023-11-28T13:07:35.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.995+0000 7f0911ee9700 1 -- 172.21.15.6:0/3605604220 shutdown_connections 2023-11-28T13:07:35.999 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.995+0000 7f0911ee9700 1 --2- 172.21.15.6:0/3605604220 >> v2:172.21.15.61:3300/0 conn(0x7f090c0c4a80 0x7f090c0c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.995+0000 7f0911ee9700 1 --2- 172.21.15.6:0/3605604220 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 0x7f090c0cea30 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.995+0000 7f0911ee9700 1 -- 172.21.15.6:0/3605604220 >> 172.21.15.6:0/3605604220 conn(0x7f090c0cd6d0 msgr2=0x7f090c0cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.995+0000 7f0911ee9700 1 -- 172.21.15.6:0/3605604220 shutdown_connections 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.995+0000 7f0911ee9700 1 -- 172.21.15.6:0/3605604220 wait complete. 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.996+0000 7f0911ee9700 1 Processor -- start 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.996+0000 7f0911ee9700 1 -- start start 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f0911ee9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f090c0b26d0 0x7f090c150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f0911ee9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f090c0c4a80 0x7f090c0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f0911ee9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 0x7f090c0b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f0911ee9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0b4250 con 0x7f090c0cd280 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f0911ee9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0b68e0 con 0x7f090c0c4a80 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f0911ee9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0b6600 con 0x7f090c0b26d0 2023-11-28T13:07:36.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f090bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 0x7f090c0b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f090bfff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 0x7f090c0b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35216/0 (socket says 172.21.15.6:35216) 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f090bfff700 1 -- 172.21.15.6:0/1776483374 learned_addr learned my addr 172.21.15.6:0/1776483374 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f090b7fe700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f090c0b26d0 0x7f090c150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f090affd700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f090c0c4a80 0x7f090c0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f090bfff700 1 -- 172.21.15.6:0/1776483374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f090c0b26d0 msgr2=0x7f090c150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f090bfff700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f090c0b26d0 0x7f090c150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f090bfff700 1 -- 172.21.15.6:0/1776483374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f090c0c4a80 msgr2=0x7f090c0b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f090bfff700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f090c0c4a80 0x7f090c0b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.997+0000 7f090bfff700 1 -- 172.21.15.6:0/1776483374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f4009430 con 0x7f090c0cd280 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.998+0000 7f090bfff700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 0x7f090c0b46e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0904007e40 tx=0x7f090400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.998+0000 7f0903fff700 1 -- 172.21.15.6:0/1776483374 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090400ed30 con 0x7f090c0cd280 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.998+0000 7f0903fff700 1 -- 172.21.15.6:0/1776483374 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904014420 con 0x7f090c0cd280 2023-11-28T13:07:36.001 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.998+0000 7f0911ee9700 1 -- 172.21.15.6:0/1776483374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c15dd70 con 0x7f090c0cd280 2023-11-28T13:07:36.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.998+0000 7f0903fff700 1 -- 172.21.15.6:0/1776483374 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09040147d0 con 0x7f090c0cd280 2023-11-28T13:07:36.002 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.998+0000 7f0911ee9700 1 -- 172.21.15.6:0/1776483374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c0cff20 con 0x7f090c0cd280 2023-11-28T13:07:36.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.999+0000 7f0911ee9700 1 -- 172.21.15.6:0/1776483374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c15df00 con 0x7f090c0cd280 2023-11-28T13:07:36.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:35.999+0000 7f0903fff700 1 -- 172.21.15.6:0/1776483374 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0904012050 con 0x7f090c0cd280 2023-11-28T13:07:36.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.000+0000 7f0903fff700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08ec0682f0 0x7f08ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.000+0000 7f0903fff700 1 -- 172.21.15.6:0/1776483374 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f0904086390 con 0x7f090c0cd280 2023-11-28T13:07:36.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.000+0000 7f090b7fe700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08ec0682f0 0x7f08ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.003 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.000+0000 7f090b7fe700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08ec0682f0 0x7f08ec06a7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f08fc007fd0 tx=0x7f08fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:36.005 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.004+0000 7f0903fff700 1 -- 172.21.15.6:0/1776483374 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0904052a70 con 0x7f090c0cd280 2023-11-28T13:07:36.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.243+0000 7f0911ee9700 1 -- 172.21.15.6:0/1776483374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f090c068570 con 0x7f090c0cd280 2023-11-28T13:07:36.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.244+0000 7f0903fff700 1 -- 172.21.15.6:0/1776483374 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f0904056090 con 0x7f090c0cd280 2023-11-28T13:07:36.245 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:36.245 INFO:teuthology.orchestra.run.smithi006.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T13:07:36.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 -- 172.21.15.6:0/1776483374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08ec0682f0 msgr2=0x7f08ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08ec0682f0 0x7f08ec06a7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f08fc007fd0 tx=0x7f08fc009040 comp rx=0 tx=0).stop 2023-11-28T13:07:36.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 -- 172.21.15.6:0/1776483374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 msgr2=0x7f090c0b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 0x7f090c0b46e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0904007e40 tx=0x7f090400e3c0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 -- 172.21.15.6:0/1776483374 shutdown_connections 2023-11-28T13:07:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f090c0cd280 0x7f090c0b46e0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f08ec0682f0 0x7f08ec06a7a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f090c0b26d0 0x7f090c150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 --2- 172.21.15.6:0/1776483374 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f090c0c4a80 0x7f090c0b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 -- 172.21.15.6:0/1776483374 >> 172.21.15.6:0/1776483374 conn(0x7f090c0cd6d0 msgr2=0x7f090c0c4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 -- 172.21.15.6:0/1776483374 shutdown_connections 2023-11-28T13:07:36.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.246+0000 7f0911ee9700 1 -- 172.21.15.6:0/1776483374 wait complete. 2023-11-28T13:07:36.259 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T13:07:36.259 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:07:36.313 DEBUG:teuthology.orchestra.run.smithi061:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T13:07:36.358 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T13:07:36.361 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:07:36.361 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:36.379 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:36.544 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.543+0000 7fa183c6a700 1 Processor -- start 2023-11-28T13:07:36.545 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.543+0000 7fa183c6a700 1 -- start start 2023-11-28T13:07:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa183c6a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa17c15c990 0x7fa17c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.547 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa183c6a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa17c161090 0x7fa17c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa183c6a700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa17c0caeb0 0x7fa17c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa183c6a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c0ce340 con 0x7fa17c15c990 2023-11-28T13:07:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa183c6a700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c0cebe0 con 0x7fa17c0caeb0 2023-11-28T13:07:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa183c6a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0cb3f0 con 0x7fa17c161090 2023-11-28T13:07:36.548 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa181a06700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa17c15c990 0x7fa17c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa181a06700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa17c15c990 0x7fa17c0cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35220/0 (socket says 172.21.15.6:35220) 2023-11-28T13:07:36.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa181a06700 1 -- 172.21.15.6:0/1837156096 learned_addr learned my addr 172.21.15.6:0/1837156096 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:36.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa181205700 1 --2- 172.21.15.6:0/1837156096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa17c161090 0x7fa17c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa182207700 1 --2- 172.21.15.6:0/1837156096 >> v2:172.21.15.61:3300/0 conn(0x7fa17c0caeb0 0x7fa17c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa181a06700 1 -- 172.21.15.6:0/1837156096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa17c161090 msgr2=0x7fa17c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:36.549 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa181a06700 1 --2- 172.21.15.6:0/1837156096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa17c161090 0x7fa17c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa181a06700 1 -- 172.21.15.6:0/1837156096 >> v2:172.21.15.61:3300/0 conn(0x7fa17c0caeb0 msgr2=0x7fa17c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:36.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa181a06700 1 --2- 172.21.15.6:0/1837156096 >> v2:172.21.15.61:3300/0 conn(0x7fa17c0caeb0 0x7fa17c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.544+0000 7fa181a06700 1 -- 172.21.15.6:0/1837156096 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0ce590 con 0x7fa17c15c990 2023-11-28T13:07:36.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa181a06700 1 --2- 172.21.15.6:0/1837156096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c15c990 0x7fa17c0cf070 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0ca080 tx=0x7fa16c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c738385b5efb27 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:36.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa180a04700 1 -- 172.21.15.6:0/1837156096 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c00b870 con 0x7fa17c15c990 2023-11-28T13:07:36.550 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa180a04700 1 -- 172.21.15.6:0/1837156096 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c00b9d0 con 0x7fa17c15c990 2023-11-28T13:07:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa180a04700 1 -- 172.21.15.6:0/1837156096 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c00bd60 con 0x7fa17c15c990 2023-11-28T13:07:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa183c6a700 1 -- 172.21.15.6:0/1837156096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c15c990 msgr2=0x7fa17c0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa183c6a700 1 --2- 172.21.15.6:0/1837156096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c15c990 0x7fa17c0cf070 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0ca080 tx=0x7fa16c009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa183c6a700 1 -- 172.21.15.6:0/1837156096 shutdown_connections 2023-11-28T13:07:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa183c6a700 1 --2- 172.21.15.6:0/1837156096 >> v2:172.21.15.61:3300/0 conn(0x7fa17c0caeb0 0x7fa17c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa183c6a700 1 --2- 172.21.15.6:0/1837156096 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c15c990 0x7fa17c0cf070 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.551 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa183c6a700 1 --2- 172.21.15.6:0/1837156096 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa17c161090 0x7fa17c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.545+0000 7fa183c6a700 1 -- 172.21.15.6:0/1837156096 >> 172.21.15.6:0/1837156096 conn(0x7fa17c0b1650 msgr2=0x7fa17c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.546+0000 7fa183c6a700 1 -- 172.21.15.6:0/1837156096 shutdown_connections 2023-11-28T13:07:36.552 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.546+0000 7fa183c6a700 1 -- 172.21.15.6:0/1837156096 wait complete. 2023-11-28T13:07:36.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.547+0000 7fa183c6a700 1 Processor -- start 2023-11-28T13:07:36.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.547+0000 7fa183c6a700 1 -- start start 2023-11-28T13:07:36.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.547+0000 7fa183c6a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa17c0caeb0 0x7fa17c0ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.547+0000 7fa183c6a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa17c15c990 0x7fa17c0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.547+0000 7fa183c6a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c161090 0x7fa17c0c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa183c6a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0c43a0 con 0x7fa17c161090 2023-11-28T13:07:36.555 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa183c6a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c14b1d0 con 0x7fa17c0caeb0 2023-11-28T13:07:36.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa183c6a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0c9a30 con 0x7fa17c15c990 2023-11-28T13:07:36.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa182207700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c161090 0x7fa17c0c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa182207700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c161090 0x7fa17c0c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35230/0 (socket says 172.21.15.6:35230) 2023-11-28T13:07:36.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa182207700 1 -- 172.21.15.6:0/1135999211 learned_addr learned my addr 172.21.15.6:0/1135999211 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:36.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa181205700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa17c15c990 0x7fa17c0c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.556 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa181a06700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa17c0caeb0 0x7fa17c0ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa182207700 1 -- 172.21.15.6:0/1135999211 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa17c15c990 msgr2=0x7fa17c0c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa182207700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa17c15c990 0x7fa17c0c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa182207700 1 -- 172.21.15.6:0/1135999211 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa17c0caeb0 msgr2=0x7fa17c0ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa182207700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa17c0caeb0 0x7fa17c0ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa182207700 1 -- 172.21.15.6:0/1135999211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c009550 con 0x7fa17c161090 2023-11-28T13:07:36.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa182207700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c161090 0x7fa17c0c5660 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa174007e60 tx=0x7fa17400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:36.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa1727fc700 1 -- 172.21.15.6:0/1135999211 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17400ed30 con 0x7fa17c161090 2023-11-28T13:07:36.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa1727fc700 1 -- 172.21.15.6:0/1135999211 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174014420 con 0x7fa17c161090 2023-11-28T13:07:36.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa183c6a700 1 -- 172.21.15.6:0/1135999211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c0cf5b0 con 0x7fa17c161090 2023-11-28T13:07:36.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa1727fc700 1 -- 172.21.15.6:0/1135999211 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740147d0 con 0x7fa17c161090 2023-11-28T13:07:36.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.548+0000 7fa183c6a700 1 -- 172.21.15.6:0/1135999211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c0ce870 con 0x7fa17c161090 2023-11-28T13:07:36.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.549+0000 7fa183c6a700 1 -- 172.21.15.6:0/1135999211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c0cf740 con 0x7fa17c161090 2023-11-28T13:07:36.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.550+0000 7fa1727fc700 1 -- 172.21.15.6:0/1135999211 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa174012050 con 0x7fa17c161090 2023-11-28T13:07:36.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.550+0000 7fa1727fc700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1600682f0 0x7fa16006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.550+0000 7fa1727fc700 1 -- 172.21.15.6:0/1135999211 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa1740862f0 con 0x7fa17c161090 2023-11-28T13:07:36.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.550+0000 7fa181a06700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1600682f0 0x7fa16006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.551+0000 7fa181a06700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1600682f0 0x7fa16006a7a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa16c000c00 tx=0x7fa16c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:36.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.555+0000 7fa1727fc700 1 -- 172.21.15.6:0/1135999211 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa1740529d0 con 0x7fa17c161090 2023-11-28T13:07:36.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.750+0000 7fa183c6a700 1 -- 172.21.15.6:0/1135999211 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa17c0675a0 con 0x7fa17c161090 2023-11-28T13:07:36.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.752+0000 7fa1727fc700 1 -- 172.21.15.6:0/1135999211 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18127 (secure 0 0 0) 0x7fa174055ff0 con 0x7fa17c161090 2023-11-28T13:07:36.753 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:36.753 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":68,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:04:11.062741+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:03:14.665009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T13:03:15.497374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:36.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 -- 172.21.15.6:0/1135999211 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1600682f0 msgr2=0x7fa16006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1600682f0 0x7fa16006a7a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa16c000c00 tx=0x7fa16c008030 comp rx=0 tx=0).stop 2023-11-28T13:07:36.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 -- 172.21.15.6:0/1135999211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c161090 msgr2=0x7fa17c0c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c161090 0x7fa17c0c5660 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa174007e60 tx=0x7fa17400e3c0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 -- 172.21.15.6:0/1135999211 shutdown_connections 2023-11-28T13:07:36.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa17c161090 0x7fa17c0c5660 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa17c0caeb0 0x7fa17c0ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa1600682f0 0x7fa16006a7a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 --2- 172.21.15.6:0/1135999211 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa17c15c990 0x7fa17c0c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 -- 172.21.15.6:0/1135999211 >> 172.21.15.6:0/1135999211 conn(0x7fa17c0b1650 msgr2=0x7fa17c0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:36.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 -- 172.21.15.6:0/1135999211 shutdown_connections 2023-11-28T13:07:36.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.754+0000 7fa183c6a700 1 -- 172.21.15.6:0/1135999211 wait complete. 2023-11-28T13:07:36.767 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:03:14.665009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:03:15.497374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:07:36.768 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:07:36.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.894+0000 7f030577a700 1 Processor -- start 2023-11-28T13:07:36.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.894+0000 7f030577a700 1 -- start start 2023-11-28T13:07:36.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.894+0000 7f030577a700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f030015c640 0x7f03000cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.894+0000 7f030577a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03000d2820 0x7f03000b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.894+0000 7f030577a700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03000d1470 con 0x7f030006bf30 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.894+0000 7f030577a700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030015ca10 con 0x7f030015c640 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.894+0000 7f030577a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000cd4e0 con 0x7f03000d2820 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02feffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f030015c640 0x7f03000cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02feffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f030015c640 0x7f03000cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35234/0 (socket says 172.21.15.6:35234) 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02feffd700 1 -- 172.21.15.6:0/2670856196 learned_addr learned my addr 172.21.15.6:0/2670856196 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02fe7fc700 1 --2- 172.21.15.6:0/2670856196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03000d2820 0x7f03000b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02feffd700 1 -- 172.21.15.6:0/2670856196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03000d2820 msgr2=0x7f03000b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02feffd700 1 --2- 172.21.15.6:0/2670856196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03000d2820 0x7f03000b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02feffd700 1 -- 172.21.15.6:0/2670856196 >> v1:172.21.15.61:6789/0 conn(0x7f030006bf30 legacy=0x7f030006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02feffd700 1 -- 172.21.15.6:0/2670856196 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03000cd7c0 con 0x7f030015c640 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02feffd700 1 --2- 172.21.15.6:0/2670856196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f030015c640 0x7f03000cf040 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f02f00098f0 tx=0x7f02f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55dbdd2453e0b84 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02fdffb700 1 -- 172.21.15.6:0/2670856196 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011830 con 0x7f030015c640 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02fdffb700 1 -- 172.21.15.6:0/2670856196 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f0011990 con 0x7f030015c640 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.895+0000 7f02fdffb700 1 -- 172.21.15.6:0/2670856196 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011d20 con 0x7f030015c640 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.896+0000 7f030577a700 1 -- 172.21.15.6:0/2670856196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f030015c640 msgr2=0x7f03000cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.896+0000 7f030577a700 1 --2- 172.21.15.6:0/2670856196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f030015c640 0x7f03000cf040 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f02f00098f0 tx=0x7f02f0009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.896+0000 7f030577a700 1 -- 172.21.15.6:0/2670856196 shutdown_connections 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.896+0000 7f030577a700 1 --2- 172.21.15.6:0/2670856196 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f030015c640 0x7f03000cf040 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.896+0000 7f030577a700 1 --2- 172.21.15.6:0/2670856196 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f03000d2820 0x7f03000b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.896+0000 7f030577a700 1 -- 172.21.15.6:0/2670856196 >> 172.21.15.6:0/2670856196 conn(0x7f03000d2df0 msgr2=0x7f03000d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.896+0000 7f030577a700 1 -- 172.21.15.6:0/2670856196 shutdown_connections 2023-11-28T13:07:36.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.896+0000 7f030577a700 1 -- 172.21.15.6:0/2670856196 wait complete. 2023-11-28T13:07:36.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.897+0000 7f030577a700 1 Processor -- start 2023-11-28T13:07:36.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.897+0000 7f030577a700 1 -- start start 2023-11-28T13:07:36.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f030577a700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f030006bf30 0x7f030015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f030577a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03000d2820 0x7f03000b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f030577a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f030015c640 0x7f03000b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f030577a700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03000c81c0 con 0x7f03000d2820 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f030577a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03000cc8e0 con 0x7f030015c640 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f030577a700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000c7890 con 0x7f030006bf30 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f02fe7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03000d2820 0x7f03000b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f02fe7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03000d2820 0x7f03000b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35242/0 (socket says 172.21.15.6:35242) 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f02fe7fc700 1 -- 172.21.15.6:0/388911411 learned_addr learned my addr 172.21.15.6:0/388911411 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f02ff7fe700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f030015c640 0x7f03000b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f02feffd700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f030006bf30 0x7f030015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f02fe7fc700 1 -- 172.21.15.6:0/388911411 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f030006bf30 msgr2=0x7f030015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f02fe7fc700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f030006bf30 0x7f030015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f02fe7fc700 1 -- 172.21.15.6:0/388911411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f030015c640 msgr2=0x7f03000b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:36.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f02fe7fc700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f030015c640 0x7f03000b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.898+0000 7f02fe7fc700 1 -- 172.21.15.6:0/388911411 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0009550 con 0x7f03000d2820 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.899+0000 7f02fe7fc700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03000d2820 0x7f03000b5c20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f02e800e9b0 tx=0x7f02e800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.899+0000 7f02f77fe700 1 -- 172.21.15.6:0/388911411 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8009710 con 0x7f03000d2820 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.899+0000 7f02f77fe700 1 -- 172.21.15.6:0/388911411 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e800ee50 con 0x7f03000d2820 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.899+0000 7f030577a700 1 -- 172.21.15.6:0/388911411 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03000cf580 con 0x7f03000d2820 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.899+0000 7f02f77fe700 1 -- 172.21.15.6:0/388911411 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800ca50 con 0x7f03000d2820 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.899+0000 7f030577a700 1 -- 172.21.15.6:0/388911411 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03000d1710 con 0x7f03000d2820 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.900+0000 7f030577a700 1 -- 172.21.15.6:0/388911411 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03000cdaa0 con 0x7f03000d2820 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.900+0000 7f02f77fe700 1 -- 172.21.15.6:0/388911411 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f02e8019030 con 0x7f03000d2820 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.901+0000 7f02f77fe700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f02e00682f0 0x7f02e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.901+0000 7f02f77fe700 1 -- 172.21.15.6:0/388911411 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f02e8085cb0 con 0x7f03000d2820 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.901+0000 7f02feffd700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f02e00682f0 0x7f02e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:36.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.901+0000 7f02feffd700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f02e00682f0 0x7f02e006a7a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f02f0014040 tx=0x7f02f0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:36.906 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:36.905+0000 7f02f77fe700 1 -- 172.21.15.6:0/388911411 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f02e8052390 con 0x7f03000d2820 2023-11-28T13:07:37.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.099+0000 7f030577a700 1 -- 172.21.15.6:0/388911411 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f03000cd120 con 0x7f03000d2820 2023-11-28T13:07:37.100 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.099+0000 7f02f77fe700 1 -- 172.21.15.6:0/388911411 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7f02e80559b0 con 0x7f03000d2820 2023-11-28T13:07:37.101 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:07:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 -- 172.21.15.6:0/388911411 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f02e00682f0 msgr2=0x7f02e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f02e00682f0 0x7f02e006a7a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f02f0014040 tx=0x7f02f0008030 comp rx=0 tx=0).stop 2023-11-28T13:07:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 -- 172.21.15.6:0/388911411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03000d2820 msgr2=0x7f03000b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03000d2820 0x7f03000b5c20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f02e800e9b0 tx=0x7f02e800ecc0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 -- 172.21.15.6:0/388911411 shutdown_connections 2023-11-28T13:07:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f030015c640 0x7f03000b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f02e00682f0 0x7f02e006a7a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.103 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f030006bf30 0x7f030015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 --2- 172.21.15.6:0/388911411 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03000d2820 0x7f03000b5c20 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 -- 172.21.15.6:0/388911411 >> 172.21.15.6:0/388911411 conn(0x7f03000d2df0 msgr2=0x7f03000d36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.101+0000 7f030577a700 1 -- 172.21.15.6:0/388911411 shutdown_connections 2023-11-28T13:07:37.104 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.102+0000 7f030577a700 1 -- 172.21.15.6:0/388911411 wait complete. 2023-11-28T13:07:37.115 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:07:37.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.242+0000 7f5e5df29700 1 Processor -- start 2023-11-28T13:07:37.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.242+0000 7f5e5df29700 1 -- start start 2023-11-28T13:07:37.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e5df29700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5e58165670 0x7f5e58167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e5df29700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e580ce550 con 0x7f5e580cf5f0 2023-11-28T13:07:37.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e5df29700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e580ce830 con 0x7f5e580b1630 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e5df29700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e580ceb10 con 0x7f5e58165670 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e56ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f5e580cf5f0 0x7f5e580caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50802/0 (socket says 172.21.15.6:50802) 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e56ffd700 1 -- 172.21.15.6:0/242161187 learned_addr learned my addr 172.21.15.6:0/242161187 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e57fff700 1 --2- 172.21.15.6:0/242161187 >> v2:172.21.15.61:3300/0 conn(0x7f5e58165670 0x7f5e58167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591215495 0 0) 0x7f5e580ce550 con 0x7f5e580cf5f0 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e440035b0 con 0x7f5e580cf5f0 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815616430 0 0) 0x7f5e580ce830 con 0x7f5e580b1630 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e580ce550 con 0x7f5e580b1630 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 534665890 0 0) 0x7f5e440035b0 con 0x7f5e580cf5f0 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 >> v2:172.21.15.61:3300/0 conn(0x7f5e58165670 msgr2=0x7f5e58167a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e567fc700 1 --2- 172.21.15.6:0/242161187 >> v2:172.21.15.61:3300/0 conn(0x7f5e58165670 0x7f5e58167a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 >> v1:172.21.15.61:6789/0 conn(0x7f5e580b1630 legacy=0x7f5e580b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580cedf0 con 0x7f5e580cf5f0 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.243+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f5e48001670 con 0x7f5e580cf5f0 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 >> v1:172.21.15.6:6789/0 conn(0x7f5e580cf5f0 legacy=0x7f5e580caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e567fc700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e44003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e567fc700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e44004850 0x7f5e44004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e567fc700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e44005200 0x7f5e4400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580ceb10 con 0x7f5e44003be0 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4400c0e0 con 0x7f5e44004850 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4400c270 con 0x7f5e44005200 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e577fe700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e44003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e56ffd700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e44004850 0x7f5e44004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e57fff700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e44005200 0x7f5e4400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e577fe700 1 -- 172.21.15.6:0/242161187 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e44005200 msgr2=0x7f5e4400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e577fe700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e44005200 0x7f5e4400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e577fe700 1 -- 172.21.15.6:0/242161187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e44004850 msgr2=0x7f5e44004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e577fe700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e44004850 0x7f5e44004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e577fe700 1 -- 172.21.15.6:0/242161187 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e440035b0 con 0x7f5e44003be0 2023-11-28T13:07:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e56ffd700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e44004850 0x7f5e44004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e57fff700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e44005200 0x7f5e4400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e57fff700 1 -- 172.21.15.6:0/242161187 reap_dead start 2023-11-28T13:07:37.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e577fe700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e44003fd0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5e40007fd0 tx=0x7f5e44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833a363e3897ba12 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40001430 con 0x7f5e44003be0 2023-11-28T13:07:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e400052d0 con 0x7f5e44003be0 2023-11-28T13:07:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.244+0000 7f5e567fc700 1 -- 172.21.15.6:0/242161187 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40001d40 con 0x7f5e44003be0 2023-11-28T13:07:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.245+0000 7f5e5df29700 1 -- 172.21.15.6:0/242161187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 msgr2=0x7f5e44003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.245+0000 7f5e5df29700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e44003fd0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5e40007fd0 tx=0x7f5e44005880 comp rx=0 tx=0).stop 2023-11-28T13:07:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.245+0000 7f5e5df29700 1 -- 172.21.15.6:0/242161187 shutdown_connections 2023-11-28T13:07:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.245+0000 7f5e5df29700 1 --2- 172.21.15.6:0/242161187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e44003fd0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.245+0000 7f5e5df29700 1 -- 172.21.15.6:0/242161187 >> 172.21.15.6:0/242161187 conn(0x7f5e580cc630 msgr2=0x7f5e580cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.245+0000 7f5e5df29700 1 -- 172.21.15.6:0/242161187 shutdown_connections 2023-11-28T13:07:37.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.245+0000 7f5e5df29700 1 -- 172.21.15.6:0/242161187 wait complete. 2023-11-28T13:07:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.246+0000 7f5e5df29700 1 Processor -- start 2023-11-28T13:07:37.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.246+0000 7f5e5df29700 1 -- start start 2023-11-28T13:07:37.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e5df29700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e580b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e5df29700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e5814f9d0 0x7f5e580b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e5df29700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e580b2f20 0x7f5e580b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e5df29700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580ac0b0 con 0x7f5e44003be0 2023-11-28T13:07:37.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e5df29700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5814fe40 con 0x7f5e580b2f20 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e5df29700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e580c91f0 con 0x7f5e5814f9d0 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e577fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e580b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e577fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e580b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35260/0 (socket says 172.21.15.6:35260) 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e577fe700 1 -- 172.21.15.6:0/4183813725 learned_addr learned my addr 172.21.15.6:0/4183813725 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e56ffd700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e5814f9d0 0x7f5e580b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e577fe700 1 -- 172.21.15.6:0/4183813725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e5814f9d0 msgr2=0x7f5e580b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e577fe700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e5814f9d0 0x7f5e580b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e577fe700 1 -- 172.21.15.6:0/4183813725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e580b2f20 msgr2=0x7f5e580b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e577fe700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e580b2f20 0x7f5e580b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e577fe700 1 -- 172.21.15.6:0/4183813725 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580ce140 con 0x7f5e44003be0 2023-11-28T13:07:37.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.247+0000 7f5e577fe700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e580b5460 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5e440047a0 tx=0x7f5e400012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.248+0000 7f5e4ffff700 1 -- 172.21.15.6:0/4183813725 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40001d40 con 0x7f5e44003be0 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.248+0000 7f5e4ffff700 1 -- 172.21.15.6:0/4183813725 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e400013d0 con 0x7f5e44003be0 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.248+0000 7f5e5df29700 1 -- 172.21.15.6:0/4183813725 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e580cf0d0 con 0x7f5e44003be0 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.248+0000 7f5e4ffff700 1 -- 172.21.15.6:0/4183813725 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40007040 con 0x7f5e44003be0 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.248+0000 7f5e5df29700 1 -- 172.21.15.6:0/4183813725 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e580ce2d0 con 0x7f5e44003be0 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.249+0000 7f5e5df29700 1 -- 172.21.15.6:0/4183813725 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e580ab8c0 con 0x7f5e44003be0 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.249+0000 7f5e4ffff700 1 -- 172.21.15.6:0/4183813725 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5e4002d070 con 0x7f5e44003be0 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.249+0000 7f5e4ffff700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5e380682f0 0x7f5e3806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.250+0000 7f5e4ffff700 1 -- 172.21.15.6:0/4183813725 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5e400866f0 con 0x7f5e44003be0 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.250+0000 7f5e56ffd700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5e380682f0 0x7f5e3806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.250+0000 7f5e56ffd700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5e380682f0 0x7f5e3806a7a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e48000c00 tx=0x7f5e48000f60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:37.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.254+0000 7f5e4ffff700 1 -- 172.21.15.6:0/4183813725 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5e580ab8c0 con 0x7f5e44003be0 2023-11-28T13:07:37.447 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.446+0000 7f5e5df29700 1 -- 172.21.15.6:0/4183813725 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e580ab8c0 con 0x7f5e44003be0 2023-11-28T13:07:37.448 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.447+0000 7f5e4ffff700 1 -- 172.21.15.6:0/4183813725 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e580ab8c0 con 0x7f5e44003be0 2023-11-28T13:07:37.448 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:07:37.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 -- 172.21.15.6:0/4183813725 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5e380682f0 msgr2=0x7f5e3806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5e380682f0 0x7f5e3806a7a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e48000c00 tx=0x7f5e48000f60 comp rx=0 tx=0).stop 2023-11-28T13:07:37.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 -- 172.21.15.6:0/4183813725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 msgr2=0x7f5e580b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.450 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e580b5460 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f5e440047a0 tx=0x7f5e400012d0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 -- 172.21.15.6:0/4183813725 shutdown_connections 2023-11-28T13:07:37.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e580b2f20 0x7f5e580b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e44003be0 0x7f5e580b5460 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5e380682f0 0x7f5e3806a7a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 --2- 172.21.15.6:0/4183813725 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e5814f9d0 0x7f5e580b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 -- 172.21.15.6:0/4183813725 >> 172.21.15.6:0/4183813725 conn(0x7f5e580cc630 msgr2=0x7f5e580cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:37.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 -- 172.21.15.6:0/4183813725 shutdown_connections 2023-11-28T13:07:37.451 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.449+0000 7f5e5df29700 1 -- 172.21.15.6:0/4183813725 wait complete. 2023-11-28T13:07:37.462 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:07:37.590 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.589+0000 7f18dee08700 1 Processor -- start 2023-11-28T13:07:37.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.589+0000 7f18dee08700 1 -- start start 2023-11-28T13:07:37.593 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.590+0000 7f18dee08700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d8159500 0x7f18d80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.590+0000 7f18dee08700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f18d815dbf0 0x7f18d80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.590+0000 7f18dee08700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d815e040 con 0x7f18d80b1630 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.590+0000 7f18dee08700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d80cede0 con 0x7f18d815dbf0 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.590+0000 7f18dee08700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80ceff0 con 0x7f18d8159500 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.590+0000 7f18d7fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f18d815dbf0 0x7f18d80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.590+0000 7f18dd3a5700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f18d80b1630 0x7f18d806bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50824/0 (socket says 172.21.15.6:50824) 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.590+0000 7f18dd3a5700 1 -- 172.21.15.6:0/4176412579 learned_addr learned my addr 172.21.15.6:0/4176412579 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.590+0000 7f18dcba4700 1 --2- 172.21.15.6:0/4176412579 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d8159500 0x7f18d80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.590+0000 7f18d7fff700 1 -- 172.21.15.6:0/4176412579 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d8159500 msgr2=0x7f18d80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18d7fff700 1 --2- 172.21.15.6:0/4176412579 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d8159500 0x7f18d80cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18d7fff700 1 -- 172.21.15.6:0/4176412579 >> v1:172.21.15.6:6789/0 conn(0x7f18d80b1630 legacy=0x7f18d806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.594 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18d77fe700 1 -- 172.21.15.6:0/4176412579 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967636945 0 0) 0x7f18d815e040 con 0x7f18d80b1630 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18d7fff700 1 -- 172.21.15.6:0/4176412579 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d80cf2d0 con 0x7f18d815dbf0 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18dcba4700 1 --2- 172.21.15.6:0/4176412579 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d8159500 0x7f18d80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18d7fff700 1 --2- 172.21.15.6:0/4176412579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 0x7f18d80ce8a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009840 tx=0x7f18c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3c96d3a0b5e920 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18d77fe700 1 -- 172.21.15.6:0/4176412579 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c4011830 con 0x7f18d815dbf0 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18d77fe700 1 -- 172.21.15.6:0/4176412579 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c4011990 con 0x7f18d815dbf0 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18d77fe700 1 -- 172.21.15.6:0/4176412579 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c4011d20 con 0x7f18d815dbf0 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18dee08700 1 -- 172.21.15.6:0/4176412579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 msgr2=0x7f18d80ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18dee08700 1 --2- 172.21.15.6:0/4176412579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 0x7f18d80ce8a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009840 tx=0x7f18c4009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18dee08700 1 -- 172.21.15.6:0/4176412579 shutdown_connections 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18dee08700 1 --2- 172.21.15.6:0/4176412579 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d8159500 0x7f18d80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18dee08700 1 --2- 172.21.15.6:0/4176412579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 0x7f18d80ce8a0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18dee08700 1 -- 172.21.15.6:0/4176412579 >> 172.21.15.6:0/4176412579 conn(0x7f18d80cc630 msgr2=0x7f18d80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18dee08700 1 -- 172.21.15.6:0/4176412579 shutdown_connections 2023-11-28T13:07:37.595 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.591+0000 7f18dee08700 1 -- 172.21.15.6:0/4176412579 wait complete. 2023-11-28T13:07:37.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.593+0000 7f18dee08700 1 Processor -- start 2023-11-28T13:07:37.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.593+0000 7f18dee08700 1 -- start start 2023-11-28T13:07:37.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18dee08700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d80b1630 0x7f18d80c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18dee08700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18d8159500 0x7f18d80c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18dee08700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 0x7f18d81f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18dee08700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d814a300 con 0x7f18d815dbf0 2023-11-28T13:07:37.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18dee08700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80c58e0 con 0x7f18d8159500 2023-11-28T13:07:37.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18dee08700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80c5bc0 con 0x7f18d80b1630 2023-11-28T13:07:37.599 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18dd3a5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 0x7f18d81f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18dd3a5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 0x7f18d81f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35278/0 (socket says 172.21.15.6:35278) 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18dd3a5700 1 -- 172.21.15.6:0/2903963482 learned_addr learned my addr 172.21.15.6:0/2903963482 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18dcba4700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d80b1630 0x7f18d80c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.594+0000 7f18d7fff700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18d8159500 0x7f18d80c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18dd3a5700 1 -- 172.21.15.6:0/2903963482 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d80b1630 msgr2=0x7f18d80c32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18dd3a5700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d80b1630 0x7f18d80c32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18dd3a5700 1 -- 172.21.15.6:0/2903963482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18d8159500 msgr2=0x7f18d80c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18dd3a5700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18d8159500 0x7f18d80c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18dd3a5700 1 -- 172.21.15.6:0/2903963482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c40094a0 con 0x7f18d815dbf0 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18d7fff700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18d8159500 0x7f18d80c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18dcba4700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d80b1630 0x7f18d80c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18dd3a5700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 0x7f18d81f27d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f18d000ee40 tx=0x7f18d000c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18d57fa700 1 -- 172.21.15.6:0/2903963482 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d00070d0 con 0x7f18d815dbf0 2023-11-28T13:07:37.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18d57fa700 1 -- 172.21.15.6:0/2903963482 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d0007250 con 0x7f18d815dbf0 2023-11-28T13:07:37.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18dee08700 1 -- 172.21.15.6:0/2903963482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d815d3f0 con 0x7f18d815dbf0 2023-11-28T13:07:37.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18d57fa700 1 -- 172.21.15.6:0/2903963482 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0007600 con 0x7f18d815dbf0 2023-11-28T13:07:37.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.595+0000 7f18dee08700 1 -- 172.21.15.6:0/2903963482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d815d110 con 0x7f18d815dbf0 2023-11-28T13:07:37.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.596+0000 7f18dee08700 1 -- 172.21.15.6:0/2903963482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d815d6d0 con 0x7f18d815dbf0 2023-11-28T13:07:37.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.597+0000 7f18d57fa700 1 -- 172.21.15.6:0/2903963482 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f18d0007760 con 0x7f18d815dbf0 2023-11-28T13:07:37.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.597+0000 7f18d57fa700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18bc0682f0 0x7f18bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.597+0000 7f18d57fa700 1 -- 172.21.15.6:0/2903963482 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f18d0088c20 con 0x7f18d815dbf0 2023-11-28T13:07:37.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.597+0000 7f18dcba4700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18bc0682f0 0x7f18bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.597+0000 7f18dcba4700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18bc0682f0 0x7f18bc06a7a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00c960 tx=0x7f18cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:37.602 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.601+0000 7f18d57fa700 1 -- 172.21.15.6:0/2903963482 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f18d0055300 con 0x7f18d815dbf0 2023-11-28T13:07:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.794+0000 7f18dee08700 1 -- 172.21.15.6:0/2903963482 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18d80b3a30 con 0x7f18d815dbf0 2023-11-28T13:07:37.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.794+0000 7f18d57fa700 1 -- 172.21.15.6:0/2903963482 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18d80b3a30 con 0x7f18d815dbf0 2023-11-28T13:07:37.796 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:07:37.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.796+0000 7f18dee08700 1 -- 172.21.15.6:0/2903963482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18bc0682f0 msgr2=0x7f18bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.796+0000 7f18dee08700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18bc0682f0 0x7f18bc06a7a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00c960 tx=0x7f18cc008040 comp rx=0 tx=0).stop 2023-11-28T13:07:37.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.796+0000 7f18dee08700 1 -- 172.21.15.6:0/2903963482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 msgr2=0x7f18d81f27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.796+0000 7f18dee08700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 0x7f18d81f27d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f18d000ee40 tx=0x7f18d000c550 comp rx=0 tx=0).stop 2023-11-28T13:07:37.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.796+0000 7f18dee08700 1 -- 172.21.15.6:0/2903963482 shutdown_connections 2023-11-28T13:07:37.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.796+0000 7f18dee08700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d815dbf0 0x7f18d81f27d0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.798 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.796+0000 7f18dee08700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f18bc0682f0 0x7f18bc06a7a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.796+0000 7f18dee08700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f18d80b1630 0x7f18d80c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.796+0000 7f18dee08700 1 --2- 172.21.15.6:0/2903963482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f18d8159500 0x7f18d80c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.797+0000 7f18dee08700 1 -- 172.21.15.6:0/2903963482 >> 172.21.15.6:0/2903963482 conn(0x7f18d80cc630 msgr2=0x7f18d80af530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:37.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.797+0000 7f18dee08700 1 -- 172.21.15.6:0/2903963482 shutdown_connections 2023-11-28T13:07:37.799 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.797+0000 7f18dee08700 1 -- 172.21.15.6:0/2903963482 wait complete. 2023-11-28T13:07:37.810 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:07:37.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.936+0000 7f0b2f7b2700 1 Processor -- start 2023-11-28T13:07:37.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.937+0000 7f0b2f7b2700 1 -- start start 2023-11-28T13:07:37.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.937+0000 7f0b2f7b2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0b280b62d0 0x7f0b280cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.939 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.937+0000 7f0b2f7b2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f0b280b2d90 0x7f0b280ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.937+0000 7f0b2f7b2700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b280cd2a0 con 0x7f0b2806f8b0 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.937+0000 7f0b2f7b2700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b280d0400 con 0x7f0b280b62d0 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.937+0000 7f0b2f7b2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b280d06e0 con 0x7f0b280b2d90 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.937+0000 7f0b2d54e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0b280b62d0 0x7f0b280cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b2cd4d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f0b2806f8b0 0x7f0b280b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50834/0 (socket says 172.21.15.6:50834) 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b2cd4d700 1 -- 172.21.15.6:0/752709972 learned_addr learned my addr 172.21.15.6:0/752709972 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b2dd4f700 1 --2- 172.21.15.6:0/752709972 >> v2:172.21.15.61:3300/0 conn(0x7f0b280b2d90 0x7f0b280ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b2d54e700 1 -- 172.21.15.6:0/752709972 >> v2:172.21.15.61:3300/0 conn(0x7f0b280b2d90 msgr2=0x7f0b280ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b1ffff700 1 -- 172.21.15.6:0/752709972 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2095056365 0 0) 0x7f0b280cd2a0 con 0x7f0b2806f8b0 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b2d54e700 1 --2- 172.21.15.6:0/752709972 >> v2:172.21.15.61:3300/0 conn(0x7f0b280b2d90 0x7f0b280ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b2d54e700 1 -- 172.21.15.6:0/752709972 >> v1:172.21.15.6:6789/0 conn(0x7f0b2806f8b0 legacy=0x7f0b280b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b2d54e700 1 -- 172.21.15.6:0/752709972 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b280cffb0 con 0x7f0b280b62d0 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b2d54e700 1 --2- 172.21.15.6:0/752709972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 0x7f0b280cb630 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0b2815dee0 tx=0x7f0b18009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b209ed548c75a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:37.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b1ffff700 1 -- 172.21.15.6:0/752709972 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18011830 con 0x7f0b280b62d0 2023-11-28T13:07:37.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b1ffff700 1 -- 172.21.15.6:0/752709972 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b18011990 con 0x7f0b280b62d0 2023-11-28T13:07:37.941 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.938+0000 7f0b1ffff700 1 -- 172.21.15.6:0/752709972 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18011d20 con 0x7f0b280b62d0 2023-11-28T13:07:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.939+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/752709972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 msgr2=0x7f0b280cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.939+0000 7f0b2f7b2700 1 --2- 172.21.15.6:0/752709972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 0x7f0b280cb630 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0b2815dee0 tx=0x7f0b18009b20 comp rx=0 tx=0).stop 2023-11-28T13:07:37.943 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.939+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/752709972 shutdown_connections 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.939+0000 7f0b2f7b2700 1 --2- 172.21.15.6:0/752709972 >> v2:172.21.15.61:3300/0 conn(0x7f0b280b2d90 0x7f0b280ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.939+0000 7f0b2f7b2700 1 --2- 172.21.15.6:0/752709972 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 0x7f0b280cb630 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.939+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/752709972 >> 172.21.15.6:0/752709972 conn(0x7f0b280cd680 msgr2=0x7f0b280cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.939+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/752709972 shutdown_connections 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.939+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/752709972 wait complete. 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.940+0000 7f0b2f7b2700 1 Processor -- start 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.940+0000 7f0b2f7b2700 1 -- start start 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2f7b2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b2806f8b0 0x7f0b2815b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2f7b2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b280b2d90 0x7f0b281560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2f7b2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 0x7f0b2815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2f7b2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2815bd80 con 0x7f0b280b62d0 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2f7b2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2815b3b0 con 0x7f0b280b2d90 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2f7b2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2806db70 con 0x7f0b2806f8b0 2023-11-28T13:07:37.944 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2dd4f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 0x7f0b2815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2dd4f700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 0x7f0b2815bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35292/0 (socket says 172.21.15.6:35292) 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2dd4f700 1 -- 172.21.15.6:0/3631813749 learned_addr learned my addr 172.21.15.6:0/3631813749 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2cd4d700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b280b2d90 0x7f0b281560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2d54e700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b2806f8b0 0x7f0b2815b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2dd4f700 1 -- 172.21.15.6:0/3631813749 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b2806f8b0 msgr2=0x7f0b2815b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2dd4f700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b2806f8b0 0x7f0b2815b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2dd4f700 1 -- 172.21.15.6:0/3631813749 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b280b2d90 msgr2=0x7f0b281560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2dd4f700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b280b2d90 0x7f0b281560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2dd4f700 1 -- 172.21.15.6:0/3631813749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b180094a0 con 0x7f0b280b62d0 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2cd4d700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b280b2d90 0x7f0b281560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.941+0000 7f0b2dd4f700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 0x7f0b2815bfd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007e40 tx=0x7f0b2000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:37.945 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.942+0000 7f0b1dffb700 1 -- 172.21.15.6:0/3631813749 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2000eea0 con 0x7f0b280b62d0 2023-11-28T13:07:37.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.942+0000 7f0b1dffb700 1 -- 172.21.15.6:0/3631813749 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b20014420 con 0x7f0b280b62d0 2023-11-28T13:07:37.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.942+0000 7f0b1dffb700 1 -- 172.21.15.6:0/3631813749 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b200147b0 con 0x7f0b280b62d0 2023-11-28T13:07:37.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.942+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/3631813749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b280b66a0 con 0x7f0b280b62d0 2023-11-28T13:07:37.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.942+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/3631813749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2806fd30 con 0x7f0b280b62d0 2023-11-28T13:07:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.943+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/3631813749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b280cce50 con 0x7f0b280b62d0 2023-11-28T13:07:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.943+0000 7f0b1dffb700 1 -- 172.21.15.6:0/3631813749 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0b20012050 con 0x7f0b280b62d0 2023-11-28T13:07:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.943+0000 7f0b1dffb700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0b0c0682f0 0x7f0b0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.944+0000 7f0b1dffb700 1 -- 172.21.15.6:0/3631813749 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f0b20086500 con 0x7f0b280b62d0 2023-11-28T13:07:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.944+0000 7f0b2d54e700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0b0c0682f0 0x7f0b0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:37.947 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.944+0000 7f0b2d54e700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0b0c0682f0 0x7f0b0c06a7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0b180090b0 tx=0x7f0b18008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:37.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:37.948+0000 7f0b1dffb700 1 -- 172.21.15.6:0/3631813749 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0b20052be0 con 0x7f0b280b62d0 2023-11-28T13:07:38.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.145+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/3631813749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b2815b520 con 0x7f0b280b62d0 2023-11-28T13:07:38.147 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.146+0000 7f0b1dffb700 1 -- 172.21.15.6:0/3631813749 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b20056200 con 0x7f0b280b62d0 2023-11-28T13:07:38.147 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/3631813749 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0b0c0682f0 msgr2=0x7f0b0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0b0c0682f0 0x7f0b0c06a7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0b180090b0 tx=0x7f0b18008030 comp rx=0 tx=0).stop 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/3631813749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 msgr2=0x7f0b2815bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 0x7f0b2815bfd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007e40 tx=0x7f0b2000e3c0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/3631813749 shutdown_connections 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0b280b62d0 0x7f0b2815bfd0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f0b0c0682f0 0x7f0b0c06a7a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f0b2806f8b0 0x7f0b2815b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 --2- 172.21.15.6:0/3631813749 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0b280b2d90 0x7f0b281560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/3631813749 >> 172.21.15.6:0/3631813749 conn(0x7f0b280cd680 msgr2=0x7f0b280b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:38.150 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/3631813749 shutdown_connections 2023-11-28T13:07:38.151 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.148+0000 7f0b2f7b2700 1 -- 172.21.15.6:0/3631813749 wait complete. 2023-11-28T13:07:38.161 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T13:07:38.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.286+0000 7f616e1e9700 1 Processor -- start 2023-11-28T13:07:38.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.286+0000 7f616e1e9700 1 -- start start 2023-11-28T13:07:38.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f616e1e9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 0x7f61680cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:38.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f616e1e9700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f61680cc310 0x7f61680cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:38.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f616e1e9700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61680ce340 con 0x7f6168161090 2023-11-28T13:07:38.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f616e1e9700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61680cebe0 con 0x7f61680cc310 2023-11-28T13:07:38.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f616e1e9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61680cccf0 con 0x7f616815c990 2023-11-28T13:07:38.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f6166ffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6168161090 0x7f61680ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50848/0 (socket says 172.21.15.6:50848) 2023-11-28T13:07:38.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f6166ffd700 1 -- 172.21.15.6:0/3885851384 learned_addr learned my addr 172.21.15.6:0/3885851384 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:38.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f61677fe700 1 --2- 172.21.15.6:0/3885851384 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 0x7f61680cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:38.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f6167fff700 1 --2- 172.21.15.6:0/3885851384 >> v2:172.21.15.61:3300/0 conn(0x7f61680cc310 0x7f61680cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:38.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f61667fc700 1 -- 172.21.15.6:0/3885851384 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122528109 0 0) 0x7f61680ce340 con 0x7f6168161090 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f61667fc700 1 -- 172.21.15.6:0/3885851384 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61500035b0 con 0x7f6168161090 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f6167fff700 1 -- 172.21.15.6:0/3885851384 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 msgr2=0x7f61680cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f61667fc700 1 -- 172.21.15.6:0/3885851384 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 781114695 0 0) 0x7f61500035b0 con 0x7f6168161090 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f6167fff700 1 --2- 172.21.15.6:0/3885851384 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 0x7f61680cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f6167fff700 1 -- 172.21.15.6:0/3885851384 >> v1:172.21.15.6:6789/0 conn(0x7f6168161090 legacy=0x7f61680ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f6167fff700 1 -- 172.21.15.6:0/3885851384 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61680cb3f0 con 0x7f61680cc310 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.287+0000 7f61677fe700 1 --2- 172.21.15.6:0/3885851384 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 0x7f61680cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f6167fff700 1 --2- 172.21.15.6:0/3885851384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61680cc310 0x7f61680cc7b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6160009d90 tx=0x7f616000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd6c391ba2bc4b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f61667fc700 1 -- 172.21.15.6:0/3885851384 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616000e070 con 0x7f61680cc310 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f61667fc700 1 -- 172.21.15.6:0/3885851384 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6160003a20 con 0x7f61680cc310 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f61667fc700 1 -- 172.21.15.6:0/3885851384 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c001670 con 0x7f61680cc310 2023-11-28T13:07:38.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f616e1e9700 1 -- 172.21.15.6:0/3885851384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61680cc310 msgr2=0x7f61680cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:38.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f616e1e9700 1 --2- 172.21.15.6:0/3885851384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61680cc310 0x7f61680cc7b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6160009d90 tx=0x7f616000b330 comp rx=0 tx=0).stop 2023-11-28T13:07:38.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f616e1e9700 1 -- 172.21.15.6:0/3885851384 shutdown_connections 2023-11-28T13:07:38.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f616e1e9700 1 --2- 172.21.15.6:0/3885851384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61680cc310 0x7f61680cc7b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f616e1e9700 1 --2- 172.21.15.6:0/3885851384 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 0x7f61680cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f616e1e9700 1 -- 172.21.15.6:0/3885851384 >> 172.21.15.6:0/3885851384 conn(0x7f61680b1650 msgr2=0x7f61680b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:38.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f616e1e9700 1 -- 172.21.15.6:0/3885851384 shutdown_connections 2023-11-28T13:07:38.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.288+0000 7f616e1e9700 1 -- 172.21.15.6:0/3885851384 wait complete. 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.290+0000 7f616e1e9700 1 Processor -- start 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.290+0000 7f616e1e9700 1 -- start start 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f616e1e9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61680cc310 0x7f61680c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f616e1e9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 0x7f616814be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f616e1e9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6168161090 0x7f61680b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f616e1e9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61680c4ac0 con 0x7f61680cc310 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f616e1e9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61680c4da0 con 0x7f6168161090 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f616e1e9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61680c5080 con 0x7f616815c990 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f61677fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61680cc310 0x7f61680c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f61677fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61680cc310 0x7f61680c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35300/0 (socket says 172.21.15.6:35300) 2023-11-28T13:07:38.295 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f61677fe700 1 -- 172.21.15.6:0/3827674582 learned_addr learned my addr 172.21.15.6:0/3827674582 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f6166ffd700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 0x7f616814be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f6167fff700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6168161090 0x7f61680b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f61677fe700 1 -- 172.21.15.6:0/3827674582 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 msgr2=0x7f616814be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f61677fe700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 0x7f616814be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f61677fe700 1 -- 172.21.15.6:0/3827674582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6168161090 msgr2=0x7f61680b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f61677fe700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6168161090 0x7f61680b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f61677fe700 1 -- 172.21.15.6:0/3827674582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580094a0 con 0x7f61680cc310 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f6166ffd700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 0x7f616814be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f6167fff700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6168161090 0x7f61680b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f61677fe700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61680cc310 0x7f61680c5e40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f615800ebe0 tx=0x7f6158009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f614ffff700 1 -- 172.21.15.6:0/3827674582 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800c910 con 0x7f61680cc310 2023-11-28T13:07:38.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f614ffff700 1 -- 172.21.15.6:0/3827674582 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615800ca70 con 0x7f61680cc310 2023-11-28T13:07:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f616e1e9700 1 -- 172.21.15.6:0/3827674582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6160009a70 con 0x7f61680cc310 2023-11-28T13:07:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f614ffff700 1 -- 172.21.15.6:0/3827674582 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800cde0 con 0x7f61680cc310 2023-11-28T13:07:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.291+0000 7f616e1e9700 1 -- 172.21.15.6:0/3827674582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61680ce870 con 0x7f61680cc310 2023-11-28T13:07:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.292+0000 7f616e1e9700 1 -- 172.21.15.6:0/3827674582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61680ce590 con 0x7f61680cc310 2023-11-28T13:07:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.293+0000 7f614ffff700 1 -- 172.21.15.6:0/3827674582 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6158019030 con 0x7f61680cc310 2023-11-28T13:07:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.293+0000 7f614ffff700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61540682f0 0x7f615406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.293+0000 7f614ffff700 1 -- 172.21.15.6:0/3827674582 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6158085ea0 con 0x7f61680cc310 2023-11-28T13:07:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.293+0000 7f6166ffd700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61540682f0 0x7f615406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:38.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.294+0000 7f6166ffd700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61540682f0 0x7f615406a7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f615c001860 tx=0x7f615c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:38.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.297+0000 7f614ffff700 1 -- 172.21.15.6:0/3827674582 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6158052580 con 0x7f61680cc310 2023-11-28T13:07:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.508+0000 7f616e1e9700 1 -- 172.21.15.6:0/3827674582 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f61680cb080 con 0x7f61680cc310 2023-11-28T13:07:38.510 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.509+0000 7f614ffff700 1 -- 172.21.15.6:0/3827674582 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6158017080 con 0x7f61680cc310 2023-11-28T13:07:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 -- 172.21.15.6:0/3827674582 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61540682f0 msgr2=0x7f615406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61540682f0 0x7f615406a7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f615c001860 tx=0x7f615c009040 comp rx=0 tx=0).stop 2023-11-28T13:07:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 -- 172.21.15.6:0/3827674582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61680cc310 msgr2=0x7f61680c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:38.512 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61680cc310 0x7f61680c5e40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f615800ebe0 tx=0x7f6158009ec0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 -- 172.21.15.6:0/3827674582 shutdown_connections 2023-11-28T13:07:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6168161090 0x7f61680b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f61680cc310 0x7f61680c5e40 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f61540682f0 0x7f615406a7a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 --2- 172.21.15.6:0/3827674582 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f616815c990 0x7f616814be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 -- 172.21.15.6:0/3827674582 >> 172.21.15.6:0/3827674582 conn(0x7f61680b1650 msgr2=0x7f61680c8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 -- 172.21.15.6:0/3827674582 shutdown_connections 2023-11-28T13:07:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:38.511+0000 7f616e1e9700 1 -- 172.21.15.6:0/3827674582 wait complete. 2023-11-28T13:07:38.513 INFO:teuthology.orchestra.run.smithi006.stderr:listed 0 entries 2023-11-28T13:07:38.524 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2023-11-28T13:07:38.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:38.524 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T13:07:38.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi006.stderr:ceph-fuse[129574]: fuse finished with error 0 and tester_r 0 2023-11-28T13:07:38.570 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:07:38.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:38.570 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T13:07:38.645 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:07:38.645 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:38.664 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:07:38.665 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:07:38.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:38.665 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T13:07:38.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:07:38.724 INFO:teuthology.orchestra.run.smithi061.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T13:07:38.724 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:07:38.724 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:38.742 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3446 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:07:38.742 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:07:38.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:38.742 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T13:07:38.798 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T13:07:38.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:07:38.799 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:07:38.800 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:38.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:07:38.820 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:07:38.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:38.820 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T13:07:38.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:07:38.879 INFO:teuthology.orchestra.run.smithi061.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T13:07:38.879 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:07:39.004 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.003+0000 7f6570b44700 1 Processor -- start 2023-11-28T13:07:39.004 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.003+0000 7f6570b44700 1 -- start start 2023-11-28T13:07:39.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f6570b44700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f656c15c560 0x7f656c06f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f6570b44700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f656c0ceec0 0x7f656c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f6570b44700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c0b30b0 con 0x7f656c0b2660 2023-11-28T13:07:39.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f6570b44700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656c06c020 con 0x7f656c15c560 2023-11-28T13:07:39.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f6570b44700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c0d09c0 con 0x7f656c0ceec0 2023-11-28T13:07:39.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f656a59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f656c15c560 0x7f656c06f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f656a59c700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f656c15c560 0x7f656c06f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35308/0 (socket says 172.21.15.6:35308) 2023-11-28T13:07:39.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f656a59c700 1 -- 172.21.15.6:0/3034684669 learned_addr learned my addr 172.21.15.6:0/3034684669 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:39.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f656ad9d700 1 --2- 172.21.15.6:0/3034684669 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f656c0ceec0 0x7f656c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.007 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f656a59c700 1 -- 172.21.15.6:0/3034684669 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f656c0ceec0 msgr2=0x7f656c06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f656a59c700 1 --2- 172.21.15.6:0/3034684669 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f656c0ceec0 0x7f656c06c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f656a59c700 1 -- 172.21.15.6:0/3034684669 >> v1:172.21.15.61:6789/0 conn(0x7f656c0b2660 legacy=0x7f656c0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.004+0000 7f656a59c700 1 -- 172.21.15.6:0/3034684669 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c06fdf0 con 0x7f656c15c560 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f656a59c700 1 --2- 172.21.15.6:0/3034684669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c15c560 0x7f656c06f8b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f65600098f0 tx=0x7f6560009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad6f25fe029defe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f656959a700 1 -- 172.21.15.6:0/3034684669 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560004230 con 0x7f656c15c560 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f656959a700 1 -- 172.21.15.6:0/3034684669 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6560004390 con 0x7f656c15c560 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f656959a700 1 -- 172.21.15.6:0/3034684669 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65600036a0 con 0x7f656c15c560 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f6570b44700 1 -- 172.21.15.6:0/3034684669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c15c560 msgr2=0x7f656c06f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f6570b44700 1 --2- 172.21.15.6:0/3034684669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c15c560 0x7f656c06f8b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f65600098f0 tx=0x7f6560009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f6570b44700 1 -- 172.21.15.6:0/3034684669 shutdown_connections 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f6570b44700 1 --2- 172.21.15.6:0/3034684669 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f656c0ceec0 0x7f656c06c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.008 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f6570b44700 1 --2- 172.21.15.6:0/3034684669 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c15c560 0x7f656c06f8b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f6570b44700 1 -- 172.21.15.6:0/3034684669 >> 172.21.15.6:0/3034684669 conn(0x7f656c0cd660 msgr2=0x7f656c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:39.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f6570b44700 1 -- 172.21.15.6:0/3034684669 shutdown_connections 2023-11-28T13:07:39.009 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.005+0000 7f6570b44700 1 -- 172.21.15.6:0/3034684669 wait complete. 2023-11-28T13:07:39.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.007+0000 7f6570b44700 1 Processor -- start 2023-11-28T13:07:39.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.007+0000 7f6570b44700 1 -- start start 2023-11-28T13:07:39.011 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.007+0000 7f6570b44700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f656c0b2660 0x7f656c151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.007+0000 7f6570b44700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c0ceec0 0x7f656c1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.007+0000 7f6570b44700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f656c15c560 0x7f656c155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.007+0000 7f6570b44700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c0c57d0 con 0x7f656c0ceec0 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6570b44700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1518e0 con 0x7f656c15c560 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6570b44700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1e7870 con 0x7f656c0b2660 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6569d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c0ceec0 0x7f656c1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6569d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c0ceec0 0x7f656c1513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35314/0 (socket says 172.21.15.6:35314) 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6569d9b700 1 -- 172.21.15.6:0/1979746836 learned_addr learned my addr 172.21.15.6:0/1979746836 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f656a59c700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f656c0b2660 0x7f656c151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f656ad9d700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f656c15c560 0x7f656c155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6569d9b700 1 -- 172.21.15.6:0/1979746836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f656c0b2660 msgr2=0x7f656c151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.012 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6569d9b700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f656c0b2660 0x7f656c151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6569d9b700 1 -- 172.21.15.6:0/1979746836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f656c15c560 msgr2=0x7f656c155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6569d9b700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f656c15c560 0x7f656c155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6569d9b700 1 -- 172.21.15.6:0/1979746836 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6560009550 con 0x7f656c0ceec0 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f656ad9d700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f656c15c560 0x7f656c155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6569d9b700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c0ceec0 0x7f656c1513a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f655c00c4d0 tx=0x7f655c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f655affd700 1 -- 172.21.15.6:0/1979746836 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c011790 con 0x7f656c0ceec0 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f655affd700 1 -- 172.21.15.6:0/1979746836 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c0118f0 con 0x7f656c0ceec0 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6570b44700 1 -- 172.21.15.6:0/1979746836 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c0b2dd0 con 0x7f656c0ceec0 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f655affd700 1 -- 172.21.15.6:0/1979746836 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c011ca0 con 0x7f656c0ceec0 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.008+0000 7f6570b44700 1 -- 172.21.15.6:0/1979746836 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c0b2af0 con 0x7f656c0ceec0 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.009+0000 7f6570b44700 1 -- 172.21.15.6:0/1979746836 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c06bcb0 con 0x7f656c0ceec0 2023-11-28T13:07:39.013 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.010+0000 7f655affd700 1 -- 172.21.15.6:0/1979746836 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f655c010050 con 0x7f656c0ceec0 2023-11-28T13:07:39.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.010+0000 7f655affd700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f654c0682f0 0x7f654c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.010+0000 7f655affd700 1 -- 172.21.15.6:0/1979746836 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f655c085b00 con 0x7f656c0ceec0 2023-11-28T13:07:39.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.010+0000 7f656a59c700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f654c0682f0 0x7f654c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.014 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.011+0000 7f656a59c700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f654c0682f0 0x7f654c06a7a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6560009130 tx=0x7f6560008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:39.015 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.014+0000 7f655affd700 1 -- 172.21.15.6:0/1979746836 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f655c0521e0 con 0x7f656c0ceec0 2023-11-28T13:07:39.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.254+0000 7f6570b44700 1 -- 172.21.15.6:0/1979746836 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f656c0c5910 con 0x7f656c0ceec0 2023-11-28T13:07:39.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.255+0000 7f655affd700 1 -- 172.21.15.6:0/1979746836 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4641 (secure 0 0 0) 0x7f656c0c5910 con 0x7f656c0ceec0 2023-11-28T13:07:39.256 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:39.256 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1198233336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1198233336},{"type":"v1","addr":"172.21.15.6:6836","nonce":1198233336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/235204249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":235204249},{"type":"v1","addr":"172.21.15.6:6837","nonce":235204249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2675481701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2675481701},{"type":"v1","addr":"172.21.15.61:6833","nonce":2675481701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:03:16.530114+0000","modified":"2023-11-28T13:03:29.044885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.61:6835/3076579711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":3076579711},{"type":"v1","addr":"172.21.15.61:6835","nonce":3076579711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:07:39.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 -- 172.21.15.6:0/1979746836 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f654c0682f0 msgr2=0x7f654c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f654c0682f0 0x7f654c06a7a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6560009130 tx=0x7f6560008030 comp rx=0 tx=0).stop 2023-11-28T13:07:39.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 -- 172.21.15.6:0/1979746836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c0ceec0 msgr2=0x7f656c1513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c0ceec0 0x7f656c1513a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f655c00c4d0 tx=0x7f655c00c890 comp rx=0 tx=0).stop 2023-11-28T13:07:39.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 -- 172.21.15.6:0/1979746836 shutdown_connections 2023-11-28T13:07:39.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f656c15c560 0x7f656c155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f654c0682f0 0x7f654c06a7a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f656c0b2660 0x7f656c151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 --2- 172.21.15.6:0/1979746836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f656c0ceec0 0x7f656c1513a0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 -- 172.21.15.6:0/1979746836 >> 172.21.15.6:0/1979746836 conn(0x7f656c0cd660 msgr2=0x7f656c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:39.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 -- 172.21.15.6:0/1979746836 shutdown_connections 2023-11-28T13:07:39.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.257+0000 7f6570b44700 1 -- 172.21.15.6:0/1979746836 wait complete. 2023-11-28T13:07:39.259 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 33 2023-11-28T13:07:39.270 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:39.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.393+0000 7f3a369cb700 1 Processor -- start 2023-11-28T13:07:39.395 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.393+0000 7f3a369cb700 1 -- start start 2023-11-28T13:07:39.398 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.394+0000 7f3a369cb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015c990 0x7f3a300cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.394+0000 7f3a369cb700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a300ce340 con 0x7f3a30161090 2023-11-28T13:07:39.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.394+0000 7f3a369cb700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a300cebe0 con 0x7f3a300cc310 2023-11-28T13:07:39.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.394+0000 7f3a369cb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a300cb3f0 con 0x7f3a3015c990 2023-11-28T13:07:39.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.394+0000 7f3a2f7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f3a30161090 0x7f3a300ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50874/0 (socket says 172.21.15.6:50874) 2023-11-28T13:07:39.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.394+0000 7f3a2f7fe700 1 -- 172.21.15.6:0/3870796868 learned_addr learned my addr 172.21.15.6:0/3870796868 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:39.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2ffff700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015c990 0x7f3a300cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831215545 0 0) 0x7f3a300ce340 con 0x7f3a30161090 2023-11-28T13:07:39.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a180035b0 con 0x7f3a30161090 2023-11-28T13:07:39.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 233477883 0 0) 0x7f3a180035b0 con 0x7f3a30161090 2023-11-28T13:07:39.399 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015c990 msgr2=0x7f3a300cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015c990 0x7f3a300cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 >> v1:172.21.15.61:6789/0 conn(0x7f3a300cc310 legacy=0x7f3a300cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a300ce590 con 0x7f3a30161090 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f3a24001670 con 0x7f3a30161090 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 >> v1:172.21.15.6:6789/0 conn(0x7f3a30161090 legacy=0x7f3a300ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a18003a30 0x7f3a18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015c990 0x7f3a18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a18004d90 0x7f3a18005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a300cb3f0 con 0x7f3a18003a30 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1800bb20 con 0x7f3a18004d90 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1800bce0 con 0x7f3a3015c990 2023-11-28T13:07:39.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2ffff700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a18003a30 0x7f3a18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a2f7fe700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015c990 0x7f3a18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.395+0000 7f3a34f68700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a18004d90 0x7f3a18005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a2ffff700 1 -- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015c990 msgr2=0x7f3a18004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a2ffff700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015c990 0x7f3a18004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a2ffff700 1 -- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a18004d90 msgr2=0x7f3a18005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a2ffff700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a18004d90 0x7f3a18005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a2ffff700 1 -- 172.21.15.6:0/3870796868 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a180035b0 con 0x7f3a18003a30 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a34f68700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a18004d90 0x7f3a18005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a2ffff700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a18003a30 0x7f3a18003e00 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009b60 tx=0x7f3a18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3b3d804f9c0a6c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000e6c0 con 0x7f3a18003a30 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2000e820 con 0x7f3a18003a30 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a2effd700 1 -- 172.21.15.6:0/3870796868 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000eb90 con 0x7f3a18003a30 2023-11-28T13:07:39.401 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a369cb700 1 -- 172.21.15.6:0/3870796868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a18003a30 msgr2=0x7f3a18003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a369cb700 1 --2- 172.21.15.6:0/3870796868 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a18003a30 0x7f3a18003e00 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009b60 tx=0x7f3a18005880 comp rx=0 tx=0).stop 2023-11-28T13:07:39.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a34f68700 1 -- 172.21.15.6:0/3870796868 reap_dead start 2023-11-28T13:07:39.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a369cb700 1 -- 172.21.15.6:0/3870796868 shutdown_connections 2023-11-28T13:07:39.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a369cb700 1 -- 172.21.15.6:0/3870796868 >> 172.21.15.6:0/3870796868 conn(0x7f3a300b1650 msgr2=0x7f3a300b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:39.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a369cb700 1 -- 172.21.15.6:0/3870796868 shutdown_connections 2023-11-28T13:07:39.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.396+0000 7f3a369cb700 1 -- 172.21.15.6:0/3870796868 wait complete. 2023-11-28T13:07:39.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.398+0000 7f3a369cb700 1 Processor -- start 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.398+0000 7f3a369cb700 1 -- start start 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a369cb700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a18003a30 0x7f3a30154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a369cb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a300c33c0 0x7f3a3015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a369cb700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015e8b0 0x7f3a3015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a369cb700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a301f23e0 con 0x7f3a300c33c0 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a369cb700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3015f270 con 0x7f3a18003a30 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a369cb700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a300c6160 con 0x7f3a3015e8b0 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a300c33c0 0x7f3a3015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2f7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a300c33c0 0x7f3a3015e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35334/0 (socket says 172.21.15.6:35334) 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2f7fe700 1 -- 172.21.15.6:0/87214206 learned_addr learned my addr 172.21.15.6:0/87214206 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:39.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2ffff700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a18003a30 0x7f3a30154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a34f68700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015e8b0 0x7f3a3015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2f7fe700 1 -- 172.21.15.6:0/87214206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015e8b0 msgr2=0x7f3a3015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2f7fe700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015e8b0 0x7f3a3015ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2f7fe700 1 -- 172.21.15.6:0/87214206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a18003a30 msgr2=0x7f3a30154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2f7fe700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a18003a30 0x7f3a30154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2f7fe700 1 -- 172.21.15.6:0/87214206 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a300cb080 con 0x7f3a300c33c0 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2f7fe700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a300c33c0 0x7f3a3015e370 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400eaa0 tx=0x7f3a2400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2cff9700 1 -- 172.21.15.6:0/87214206 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24005e50 con 0x7f3a300c33c0 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2cff9700 1 -- 172.21.15.6:0/87214206 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2400ee20 con 0x7f3a300c33c0 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a369cb700 1 -- 172.21.15.6:0/87214206 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a300ce870 con 0x7f3a300c33c0 2023-11-28T13:07:39.406 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a2cff9700 1 -- 172.21.15.6:0/87214206 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2400ca20 con 0x7f3a300c33c0 2023-11-28T13:07:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.399+0000 7f3a369cb700 1 -- 172.21.15.6:0/87214206 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a300cf070 con 0x7f3a300c33c0 2023-11-28T13:07:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.400+0000 7f3a369cb700 1 -- 172.21.15.6:0/87214206 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a300675a0 con 0x7f3a300c33c0 2023-11-28T13:07:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.401+0000 7f3a2cff9700 1 -- 172.21.15.6:0/87214206 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3a2400cb80 con 0x7f3a300c33c0 2023-11-28T13:07:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.401+0000 7f3a2cff9700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a1c0682f0 0x7f3a1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.401+0000 7f3a2cff9700 1 -- 172.21.15.6:0/87214206 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f3a24085b80 con 0x7f3a300c33c0 2023-11-28T13:07:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.401+0000 7f3a2ffff700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a1c0682f0 0x7f3a1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.402+0000 7f3a2ffff700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a1c0682f0 0x7f3a1c06a7a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3a200090b0 tx=0x7f3a2001f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:39.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.405+0000 7f3a2cff9700 1 -- 172.21.15.6:0/87214206 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3a24052260 con 0x7f3a300c33c0 2023-11-28T13:07:39.600 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.599+0000 7f3a369cb700 1 -- 172.21.15.6:0/87214206 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a300cdc50 con 0x7f3a300c33c0 2023-11-28T13:07:39.601 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.600+0000 7f3a2cff9700 1 -- 172.21.15.6:0/87214206 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18127 (secure 0 0 0) 0x7f3a24055880 con 0x7f3a300c33c0 2023-11-28T13:07:39.602 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:39.602 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":68,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:04:11.062741+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:03:14.665009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T13:03:15.497374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:39.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 -- 172.21.15.6:0/87214206 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a1c0682f0 msgr2=0x7f3a1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.603 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a1c0682f0 0x7f3a1c06a7a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3a200090b0 tx=0x7f3a2001f040 comp rx=0 tx=0).stop 2023-11-28T13:07:39.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 -- 172.21.15.6:0/87214206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a300c33c0 msgr2=0x7f3a3015e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a300c33c0 0x7f3a3015e370 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400eaa0 tx=0x7f3a2400edb0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 -- 172.21.15.6:0/87214206 shutdown_connections 2023-11-28T13:07:39.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f3a1c0682f0 0x7f3a1c06a7a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f3a3015e8b0 0x7f3a3015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3a18003a30 0x7f3a30154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 --2- 172.21.15.6:0/87214206 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a300c33c0 0x7f3a3015e370 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 -- 172.21.15.6:0/87214206 >> 172.21.15.6:0/87214206 conn(0x7f3a300b1650 msgr2=0x7f3a3006de20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:39.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 -- 172.21.15.6:0/87214206 shutdown_connections 2023-11-28T13:07:39.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.602+0000 7f3a369cb700 1 -- 172.21.15.6:0/87214206 wait complete. 2023-11-28T13:07:39.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T13:00:14.560915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T13:00:14.998729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T13:03:14.665009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T13:03:15.497374+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T13:07:39.616 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T13:07:39.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.742+0000 7f9ae7d15700 1 Processor -- start 2023-11-28T13:07:39.744 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.743+0000 7f9ae7d15700 1 -- start start 2023-11-28T13:07:39.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.743+0000 7f9ae7d15700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9ae00b1630 0x7f9ae00cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.743+0000 7f9ae7d15700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00b1a00 0x7f9ae00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.743+0000 7f9ae7d15700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae00ce550 con 0x7f9ae00cb3e0 2023-11-28T13:07:39.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.743+0000 7f9ae7d15700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae00ce830 con 0x7f9ae00b1630 2023-11-28T13:07:39.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.743+0000 7f9ae7d15700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae00ceb10 con 0x7f9ae00b1a00 2023-11-28T13:07:39.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae52b0700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00b1a00 0x7f9ae00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae62b2700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f9ae00cb3e0 0x7f9ae0160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:40924/0 (socket says 172.21.15.6:40924) 2023-11-28T13:07:39.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae5ab1700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f9ae00b1630 0x7f9ae00cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.745 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae62b2700 1 -- 172.21.15.6:0/2619737015 learned_addr learned my addr 172.21.15.6:0/2619737015 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:39.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae4aaf700 1 -- 172.21.15.6:0/2619737015 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705152490 0 0) 0x7f9ae00ce550 con 0x7f9ae00cb3e0 2023-11-28T13:07:39.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae52b0700 1 -- 172.21.15.6:0/2619737015 >> v2:172.21.15.61:3300/0 conn(0x7f9ae00b1630 msgr2=0x7f9ae00cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:39.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae52b0700 1 --2- 172.21.15.6:0/2619737015 >> v2:172.21.15.61:3300/0 conn(0x7f9ae00b1630 0x7f9ae00cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae52b0700 1 -- 172.21.15.6:0/2619737015 >> v1:172.21.15.61:6789/0 conn(0x7f9ae00cb3e0 legacy=0x7f9ae0160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae52b0700 1 -- 172.21.15.6:0/2619737015 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae00cedf0 con 0x7f9ae00b1a00 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae5ab1700 1 --2- 172.21.15.6:0/2619737015 >> v2:172.21.15.61:3300/0 conn(0x7f9ae00b1630 0x7f9ae00cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae52b0700 1 --2- 172.21.15.6:0/2619737015 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00b1a00 0x7f9ae00caea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009840 tx=0x7f9ad4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c29f611750c42dc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.744+0000 7f9ae4aaf700 1 -- 172.21.15.6:0/2619737015 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4011830 con 0x7f9ae00b1a00 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.745+0000 7f9ae4aaf700 1 -- 172.21.15.6:0/2619737015 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad4011990 con 0x7f9ae00b1a00 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.745+0000 7f9ae4aaf700 1 -- 172.21.15.6:0/2619737015 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4011d20 con 0x7f9ae00b1a00 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.745+0000 7f9ae7d15700 1 -- 172.21.15.6:0/2619737015 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00b1a00 msgr2=0x7f9ae00caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.745+0000 7f9ae7d15700 1 --2- 172.21.15.6:0/2619737015 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00b1a00 0x7f9ae00caea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009840 tx=0x7f9ad4009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.745+0000 7f9ae7d15700 1 -- 172.21.15.6:0/2619737015 shutdown_connections 2023-11-28T13:07:39.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.745+0000 7f9ae7d15700 1 --2- 172.21.15.6:0/2619737015 >> v2:172.21.15.61:3300/0 conn(0x7f9ae00b1630 0x7f9ae00cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.745+0000 7f9ae7d15700 1 --2- 172.21.15.6:0/2619737015 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00b1a00 0x7f9ae00caea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.745+0000 7f9ae7d15700 1 -- 172.21.15.6:0/2619737015 >> 172.21.15.6:0/2619737015 conn(0x7f9ae00cc630 msgr2=0x7f9ae00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:39.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.745+0000 7f9ae7d15700 1 -- 172.21.15.6:0/2619737015 shutdown_connections 2023-11-28T13:07:39.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.745+0000 7f9ae7d15700 1 -- 172.21.15.6:0/2619737015 wait complete. 2023-11-28T13:07:39.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.746+0000 7f9ae7d15700 1 Processor -- start 2023-11-28T13:07:39.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.746+0000 7f9ae7d15700 1 -- start start 2023-11-28T13:07:39.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.747+0000 7f9ae7d15700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ae00b1630 0x7f9ae014efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.747+0000 7f9ae7d15700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae00b1a00 0x7f9ae00b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.747+0000 7f9ae7d15700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00cb3e0 0x7f9ae00b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.747+0000 7f9ae7d15700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae00c4190 con 0x7f9ae00b1a00 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.747+0000 7f9ae7d15700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae00c4470 con 0x7f9ae00b1630 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.747+0000 7f9ae7d15700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae00c4750 con 0x7f9ae00cb3e0 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.747+0000 7f9ae52b0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae00b1a00 0x7f9ae00b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.747+0000 7f9ae52b0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae00b1a00 0x7f9ae00b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35346/0 (socket says 172.21.15.6:35346) 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.747+0000 7f9ae52b0700 1 -- 172.21.15.6:0/1566109086 learned_addr learned my addr 172.21.15.6:0/1566109086 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ae62b2700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00cb3e0 0x7f9ae00b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ae5ab1700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ae00b1630 0x7f9ae014efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ae52b0700 1 -- 172.21.15.6:0/1566109086 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00cb3e0 msgr2=0x7f9ae00b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ae52b0700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00cb3e0 0x7f9ae00b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ae52b0700 1 -- 172.21.15.6:0/1566109086 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ae00b1630 msgr2=0x7f9ae014efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ae52b0700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ae00b1630 0x7f9ae014efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ae52b0700 1 -- 172.21.15.6:0/1566109086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc0094a0 con 0x7f9ae00b1a00 2023-11-28T13:07:39.753 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ae52b0700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae00b1a00 0x7f9ae00b45b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009b90 tx=0x7f9ad4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ad27fc700 1 -- 172.21.15.6:0/1566109086 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40038a0 con 0x7f9ae00b1a00 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ad27fc700 1 -- 172.21.15.6:0/1566109086 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad401f070 con 0x7f9ae00b1a00 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ae7d15700 1 -- 172.21.15.6:0/1566109086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad40094a0 con 0x7f9ae00b1a00 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ad27fc700 1 -- 172.21.15.6:0/1566109086 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4003c20 con 0x7f9ae00b1a00 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.748+0000 7f9ae7d15700 1 -- 172.21.15.6:0/1566109086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae00b1e70 con 0x7f9ae00b1a00 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.749+0000 7f9ae7d15700 1 -- 172.21.15.6:0/1566109086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae00cf0d0 con 0x7f9ae00b1a00 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.749+0000 7f9ad27fc700 1 -- 172.21.15.6:0/1566109086 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ad400f050 con 0x7f9ae00b1a00 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.750+0000 7f9ad27fc700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ac40682f0 0x7f9ac406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.750+0000 7f9ad27fc700 1 -- 172.21.15.6:0/1566109086 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9ad4021070 con 0x7f9ae00b1a00 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.750+0000 7f9ae5ab1700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ac40682f0 0x7f9ac406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:39.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.750+0000 7f9ae5ab1700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ac40682f0 0x7f9ac406a7a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00c40c0 tx=0x7f9adc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:39.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.754+0000 7f9ad27fc700 1 -- 172.21.15.6:0/1566109086 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ad4053f00 con 0x7f9ae00b1a00 2023-11-28T13:07:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.944+0000 7f9ae7d15700 1 -- 172.21.15.6:0/1566109086 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9ae00ce140 con 0x7f9ae00b1a00 2023-11-28T13:07:39.946 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.945+0000 7f9ad27fc700 1 -- 172.21.15.6:0/1566109086 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9ad4057520 con 0x7f9ae00b1a00 2023-11-28T13:07:39.947 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 8 2023-11-28T13:07:39.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.946+0000 7f9ae7d15700 1 -- 172.21.15.6:0/1566109086 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ac40682f0 msgr2=0x7f9ac406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.946+0000 7f9ae7d15700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ac40682f0 0x7f9ac406a7a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00c40c0 tx=0x7f9adc008040 comp rx=0 tx=0).stop 2023-11-28T13:07:39.948 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.946+0000 7f9ae7d15700 1 -- 172.21.15.6:0/1566109086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae00b1a00 msgr2=0x7f9ae00b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:39.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.946+0000 7f9ae7d15700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae00b1a00 0x7f9ae00b45b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009b90 tx=0x7f9ad4004750 comp rx=0 tx=0).stop 2023-11-28T13:07:39.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.947+0000 7f9ae7d15700 1 -- 172.21.15.6:0/1566109086 shutdown_connections 2023-11-28T13:07:39.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.947+0000 7f9ae7d15700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f9ac40682f0 0x7f9ac406a7a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.947+0000 7f9ae7d15700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9ae00cb3e0 0x7f9ae00b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.947+0000 7f9ae7d15700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9ae00b1630 0x7f9ae014efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.947+0000 7f9ae7d15700 1 --2- 172.21.15.6:0/1566109086 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae00b1a00 0x7f9ae00b45b0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:39.949 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.947+0000 7f9ae7d15700 1 -- 172.21.15.6:0/1566109086 >> 172.21.15.6:0/1566109086 conn(0x7f9ae00cc630 msgr2=0x7f9ae00cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:39.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.947+0000 7f9ae7d15700 1 -- 172.21.15.6:0/1566109086 shutdown_connections 2023-11-28T13:07:39.950 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:39.947+0000 7f9ae7d15700 1 -- 172.21.15.6:0/1566109086 wait complete. 2023-11-28T13:07:39.960 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T13:07:40.088 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.087+0000 7f9696eda700 1 Processor -- start 2023-11-28T13:07:40.089 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.087+0000 7f9696eda700 1 -- start start 2023-11-28T13:07:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f9696eda700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f96900b62d0 0x7f96900cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f9696eda700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f969006f8b0 0x7f96900b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f9696eda700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96900b2fe0 con 0x7f96900b2bc0 2023-11-28T13:07:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f9696eda700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96900cd2a0 con 0x7f969006f8b0 2023-11-28T13:07:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f9696eda700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96900d0400 con 0x7f96900b62d0 2023-11-28T13:07:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f968ffff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f969006f8b0 0x7f96900b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f9695477700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f96900b2bc0 0x7f96900ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50904/0 (socket says 172.21.15.6:50904) 2023-11-28T13:07:40.091 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f9695477700 1 -- 172.21.15.6:0/2990959400 learned_addr learned my addr 172.21.15.6:0/2990959400 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f9694c76700 1 --2- 172.21.15.6:0/2990959400 >> v2:172.21.15.61:3300/0 conn(0x7f96900b62d0 0x7f96900cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f968ffff700 1 -- 172.21.15.6:0/2990959400 >> v2:172.21.15.61:3300/0 conn(0x7f96900b62d0 msgr2=0x7f96900cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f968ffff700 1 --2- 172.21.15.6:0/2990959400 >> v2:172.21.15.61:3300/0 conn(0x7f96900b62d0 0x7f96900cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f968ffff700 1 -- 172.21.15.6:0/2990959400 >> v1:172.21.15.6:6789/0 conn(0x7f96900b2bc0 legacy=0x7f96900ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.088+0000 7f968f7fe700 1 -- 172.21.15.6:0/2990959400 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327672232 0 0) 0x7f96900b2fe0 con 0x7f96900b2bc0 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f968ffff700 1 -- 172.21.15.6:0/2990959400 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96900d06e0 con 0x7f969006f8b0 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f968ffff700 1 --2- 172.21.15.6:0/2990959400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 0x7f96900b2680 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f96900720d0 tx=0x7f967c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86569fe0a8dc6dbc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f968f7fe700 1 -- 172.21.15.6:0/2990959400 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c011830 con 0x7f969006f8b0 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f968f7fe700 1 -- 172.21.15.6:0/2990959400 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c011990 con 0x7f969006f8b0 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f968f7fe700 1 -- 172.21.15.6:0/2990959400 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c011d20 con 0x7f969006f8b0 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f9696eda700 1 -- 172.21.15.6:0/2990959400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 msgr2=0x7f96900b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f9696eda700 1 --2- 172.21.15.6:0/2990959400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 0x7f96900b2680 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f96900720d0 tx=0x7f967c009b20 comp rx=0 tx=0).stop 2023-11-28T13:07:40.092 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f9696eda700 1 -- 172.21.15.6:0/2990959400 shutdown_connections 2023-11-28T13:07:40.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f9696eda700 1 --2- 172.21.15.6:0/2990959400 >> v2:172.21.15.61:3300/0 conn(0x7f96900b62d0 0x7f96900cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f9696eda700 1 --2- 172.21.15.6:0/2990959400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 0x7f96900b2680 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f9696eda700 1 -- 172.21.15.6:0/2990959400 >> 172.21.15.6:0/2990959400 conn(0x7f96900cd680 msgr2=0x7f96900cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:40.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.089+0000 7f9696eda700 1 -- 172.21.15.6:0/2990959400 shutdown_connections 2023-11-28T13:07:40.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.090+0000 7f9696eda700 1 -- 172.21.15.6:0/2990959400 wait complete. 2023-11-28T13:07:40.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.091+0000 7f9696eda700 1 Processor -- start 2023-11-28T13:07:40.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.091+0000 7f9696eda700 1 -- start start 2023-11-28T13:07:40.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9696eda700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 0x7f969015d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9696eda700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96900b2bc0 0x7f96901e3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9696eda700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96900b62d0 0x7f96901e3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9696eda700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900b1fd0 con 0x7f969006f8b0 2023-11-28T13:07:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9696eda700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900b0f70 con 0x7f96900b62d0 2023-11-28T13:07:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9696eda700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96900b1210 con 0x7f96900b2bc0 2023-11-28T13:07:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9694c76700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 0x7f969015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9694c76700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 0x7f969015d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35356/0 (socket says 172.21.15.6:35356) 2023-11-28T13:07:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9694c76700 1 -- 172.21.15.6:0/3908476106 learned_addr learned my addr 172.21.15.6:0/3908476106 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9695477700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96900b62d0 0x7f96901e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f968ffff700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96900b2bc0 0x7f96901e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9694c76700 1 -- 172.21.15.6:0/3908476106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96900b2bc0 msgr2=0x7f96901e3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9694c76700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96900b2bc0 0x7f96901e3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9694c76700 1 -- 172.21.15.6:0/3908476106 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96900b62d0 msgr2=0x7f96901e3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9694c76700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96900b62d0 0x7f96901e3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9694c76700 1 -- 172.21.15.6:0/3908476106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c0094a0 con 0x7f969006f8b0 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9695477700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96900b62d0 0x7f96901e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9694c76700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 0x7f969015d360 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f968400d800 tx=0x7f968400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f968d7fa700 1 -- 172.21.15.6:0/3908476106 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96840125f0 con 0x7f969006f8b0 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f968d7fa700 1 -- 172.21.15.6:0/3908476106 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9684012750 con 0x7f969006f8b0 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9696eda700 1 -- 172.21.15.6:0/3908476106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969006fd30 con 0x7f969006f8b0 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f968d7fa700 1 -- 172.21.15.6:0/3908476106 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684012b00 con 0x7f969006f8b0 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.092+0000 7f9696eda700 1 -- 172.21.15.6:0/3908476106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96900cffb0 con 0x7f969006f8b0 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.093+0000 7f9696eda700 1 -- 172.21.15.6:0/3908476106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96900b66a0 con 0x7f969006f8b0 2023-11-28T13:07:40.098 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.094+0000 7f968d7fa700 1 -- 172.21.15.6:0/3908476106 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9684012c60 con 0x7f969006f8b0 2023-11-28T13:07:40.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.094+0000 7f968d7fa700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96740682f0 0x7f967406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.094+0000 7f968d7fa700 1 -- 172.21.15.6:0/3908476106 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9684086300 con 0x7f969006f8b0 2023-11-28T13:07:40.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.094+0000 7f968ffff700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96740682f0 0x7f967406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.095+0000 7f968ffff700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96740682f0 0x7f967406a7a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f96900b22a0 tx=0x7f967c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:40.099 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.099+0000 7f968d7fa700 1 -- 172.21.15.6:0/3908476106 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f96840529e0 con 0x7f969006f8b0 2023-11-28T13:07:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.292+0000 7f9696eda700 1 -- 172.21.15.6:0/3908476106 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96900b1e20 con 0x7f969006f8b0 2023-11-28T13:07:40.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.292+0000 7f968d7fa700 1 -- 172.21.15.6:0/3908476106 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9684056000 con 0x7f969006f8b0 2023-11-28T13:07:40.294 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 1 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 -- 172.21.15.6:0/3908476106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96740682f0 msgr2=0x7f967406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96740682f0 0x7f967406a7a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f96900b22a0 tx=0x7f967c008030 comp rx=0 tx=0).stop 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 -- 172.21.15.6:0/3908476106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 msgr2=0x7f969015d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 0x7f969015d360 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f968400d800 tx=0x7f968400dbc0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 -- 172.21.15.6:0/3908476106 shutdown_connections 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96900b62d0 0x7f96901e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f969006f8b0 0x7f969015d360 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f96740682f0 0x7f967406a7a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 --2- 172.21.15.6:0/3908476106 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96900b2bc0 0x7f96901e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 -- 172.21.15.6:0/3908476106 >> 172.21.15.6:0/3908476106 conn(0x7f96900cd680 msgr2=0x7f96900c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 -- 172.21.15.6:0/3908476106 shutdown_connections 2023-11-28T13:07:40.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.294+0000 7f9696eda700 1 -- 172.21.15.6:0/3908476106 wait complete. 2023-11-28T13:07:40.308 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T13:07:40.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.434+0000 7f398ddd8700 1 Processor -- start 2023-11-28T13:07:40.435 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.434+0000 7f398ddd8700 1 -- start start 2023-11-28T13:07:40.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.434+0000 7f398ddd8700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f39880b62d0 0x7f39880cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f398ddd8700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39880cd2a0 con 0x7f39880b2d90 2023-11-28T13:07:40.437 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f398ddd8700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39880d0400 con 0x7f398806f8b0 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f398ddd8700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39880d06e0 con 0x7f39880b62d0 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f39877fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f39880b62d0 0x7f39880cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f3987fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f39880b2d90 0x7f39880ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50920/0 (socket says 172.21.15.6:50920) 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f3987fff700 1 -- 172.21.15.6:0/1421429527 learned_addr learned my addr 172.21.15.6:0/1421429527 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f39867fc700 1 -- 172.21.15.6:0/1421429527 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036681470 0 0) 0x7f39880cd2a0 con 0x7f39880b2d90 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f39867fc700 1 -- 172.21.15.6:0/1421429527 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39700035b0 con 0x7f39880b2d90 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f39877fe700 1 -- 172.21.15.6:0/1421429527 >> v1:172.21.15.61:6789/0 conn(0x7f398806f8b0 legacy=0x7f39880b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f39877fe700 1 -- 172.21.15.6:0/1421429527 >> v1:172.21.15.6:6789/0 conn(0x7f39880b2d90 legacy=0x7f39880ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f39877fe700 1 -- 172.21.15.6:0/1421429527 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39880cffb0 con 0x7f39880b62d0 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f39877fe700 1 --2- 172.21.15.6:0/1421429527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39880b62d0 0x7f39880cb630 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3978009840 tx=0x7f3978009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ebd994f4852b9d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:40.438 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f39867fc700 1 -- 172.21.15.6:0/1421429527 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978004230 con 0x7f39880b62d0 2023-11-28T13:07:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f39867fc700 1 -- 172.21.15.6:0/1421429527 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3978004390 con 0x7f39880b62d0 2023-11-28T13:07:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.435+0000 7f39867fc700 1 -- 172.21.15.6:0/1421429527 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39780036c0 con 0x7f39880b62d0 2023-11-28T13:07:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.436+0000 7f398ddd8700 1 -- 172.21.15.6:0/1421429527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39880b62d0 msgr2=0x7f39880cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.436+0000 7f398ddd8700 1 --2- 172.21.15.6:0/1421429527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39880b62d0 0x7f39880cb630 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f3978009840 tx=0x7f3978009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.436+0000 7f398ddd8700 1 -- 172.21.15.6:0/1421429527 shutdown_connections 2023-11-28T13:07:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.436+0000 7f398ddd8700 1 --2- 172.21.15.6:0/1421429527 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f39880b62d0 0x7f39880cb630 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.436+0000 7f398ddd8700 1 -- 172.21.15.6:0/1421429527 >> 172.21.15.6:0/1421429527 conn(0x7f39880cd680 msgr2=0x7f39880cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:40.439 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.436+0000 7f398ddd8700 1 -- 172.21.15.6:0/1421429527 shutdown_connections 2023-11-28T13:07:40.440 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.436+0000 7f398ddd8700 1 -- 172.21.15.6:0/1421429527 wait complete. 2023-11-28T13:07:40.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.437+0000 7f398ddd8700 1 Processor -- start 2023-11-28T13:07:40.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.437+0000 7f398ddd8700 1 -- start start 2023-11-28T13:07:40.442 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f398ddd8700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398806f8b0 0x7f398815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f398ddd8700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39880b2d90 0x7f39880c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f398ddd8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39880b62d0 0x7f39880c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f398ddd8700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39881468c0 con 0x7f398806f8b0 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f398ddd8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3988148c90 con 0x7f39880b62d0 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f398ddd8700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3988148f30 con 0x7f39880b2d90 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f39877fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398806f8b0 0x7f398815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f39877fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398806f8b0 0x7f398815a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35370/0 (socket says 172.21.15.6:35370) 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f39877fe700 1 -- 172.21.15.6:0/2609532025 learned_addr learned my addr 172.21.15.6:0/2609532025 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f3986ffd700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39880b2d90 0x7f39880c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f3987fff700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39880b62d0 0x7f39880c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f39877fe700 1 -- 172.21.15.6:0/2609532025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39880b2d90 msgr2=0x7f39880c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.443 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f39877fe700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39880b2d90 0x7f39880c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f39877fe700 1 -- 172.21.15.6:0/2609532025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39880b62d0 msgr2=0x7f39880c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f39877fe700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39880b62d0 0x7f39880c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.438+0000 7f39877fe700 1 -- 172.21.15.6:0/2609532025 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39780094a0 con 0x7f398806f8b0 2023-11-28T13:07:40.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.439+0000 7f39877fe700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398806f8b0 0x7f398815a9e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f39780090b0 tx=0x7f3978004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:40.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.439+0000 7f396ffff700 1 -- 172.21.15.6:0/2609532025 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978011630 con 0x7f398806f8b0 2023-11-28T13:07:40.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.439+0000 7f396ffff700 1 -- 172.21.15.6:0/2609532025 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3978011790 con 0x7f398806f8b0 2023-11-28T13:07:40.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.439+0000 7f398ddd8700 1 -- 172.21.15.6:0/2609532025 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39880b66a0 con 0x7f398806f8b0 2023-11-28T13:07:40.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.439+0000 7f396ffff700 1 -- 172.21.15.6:0/2609532025 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978011b00 con 0x7f398806f8b0 2023-11-28T13:07:40.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.439+0000 7f398ddd8700 1 -- 172.21.15.6:0/2609532025 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398806fd30 con 0x7f398806f8b0 2023-11-28T13:07:40.444 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.440+0000 7f398ddd8700 1 -- 172.21.15.6:0/2609532025 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39880cce50 con 0x7f398806f8b0 2023-11-28T13:07:40.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.440+0000 7f396ffff700 1 -- 172.21.15.6:0/2609532025 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f397802d070 con 0x7f398806f8b0 2023-11-28T13:07:40.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.441+0000 7f396ffff700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39740682f0 0x7f397406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.441+0000 7f396ffff700 1 -- 172.21.15.6:0/2609532025 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f39780868b0 con 0x7f398806f8b0 2023-11-28T13:07:40.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.441+0000 7f3986ffd700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39740682f0 0x7f397406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.445 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.441+0000 7f3986ffd700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39740682f0 0x7f397406a7a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f39880c48e0 tx=0x7f397c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:40.446 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.445+0000 7f396ffff700 1 -- 172.21.15.6:0/2609532025 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3978052f90 con 0x7f398806f8b0 2023-11-28T13:07:40.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.639+0000 7f398ddd8700 1 -- 172.21.15.6:0/2609532025 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39880b1b10 con 0x7f398806f8b0 2023-11-28T13:07:40.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.639+0000 7f396ffff700 1 -- 172.21.15.6:0/2609532025 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39780565b0 con 0x7f398806f8b0 2023-11-28T13:07:40.641 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 32 2023-11-28T13:07:40.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.641+0000 7f398ddd8700 1 -- 172.21.15.6:0/2609532025 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39740682f0 msgr2=0x7f397406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.641+0000 7f398ddd8700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39740682f0 0x7f397406a7a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f39880c48e0 tx=0x7f397c009040 comp rx=0 tx=0).stop 2023-11-28T13:07:40.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.641+0000 7f398ddd8700 1 -- 172.21.15.6:0/2609532025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398806f8b0 msgr2=0x7f398815a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.641+0000 7f398ddd8700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398806f8b0 0x7f398815a9e0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f39780090b0 tx=0x7f3978004750 comp rx=0 tx=0).stop 2023-11-28T13:07:40.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.641+0000 7f398ddd8700 1 -- 172.21.15.6:0/2609532025 shutdown_connections 2023-11-28T13:07:40.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.641+0000 7f398ddd8700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f39880b62d0 0x7f39880c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.641+0000 7f398ddd8700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398806f8b0 0x7f398815a9e0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.641+0000 7f398ddd8700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39740682f0 0x7f397406a7a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.641+0000 7f398ddd8700 1 --2- 172.21.15.6:0/2609532025 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39880b2d90 0x7f39880c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.641+0000 7f398ddd8700 1 -- 172.21.15.6:0/2609532025 >> 172.21.15.6:0/2609532025 conn(0x7f39880cd680 msgr2=0x7f39880b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:40.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.642+0000 7f398ddd8700 1 -- 172.21.15.6:0/2609532025 shutdown_connections 2023-11-28T13:07:40.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.642+0000 7f398ddd8700 1 -- 172.21.15.6:0/2609532025 wait complete. 2023-11-28T13:07:40.655 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T13:07:40.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.781+0000 7fb8860be700 1 Processor -- start 2023-11-28T13:07:40.782 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.781+0000 7fb8860be700 1 -- start start 2023-11-28T13:07:40.784 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb8860be700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb8801589c0 0x7fb8800ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb8860be700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88006e470 con 0x7fb880158d90 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb8860be700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88006f470 con 0x7fb88006e900 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb8860be700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88006de60 con 0x7fb8801589c0 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb87f7fe700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb8801589c0 0x7fb8800ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb877fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb880158d90 0x7fb88006aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50930/0 (socket says 172.21.15.6:50930) 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb877fff700 1 -- 172.21.15.6:0/3187687052 learned_addr learned my addr 172.21.15.6:0/3187687052 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb87f7fe700 1 -- 172.21.15.6:0/3187687052 >> v1:172.21.15.61:6789/0 conn(0x7fb88006e900 legacy=0x7fb88006ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb87f7fe700 1 -- 172.21.15.6:0/3187687052 >> v1:172.21.15.6:6789/0 conn(0x7fb880158d90 legacy=0x7fb88006aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb87f7fe700 1 -- 172.21.15.6:0/3187687052 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8800ceb40 con 0x7fb8801589c0 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb87effd700 1 -- 172.21.15.6:0/3187687052 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759672556 0 0) 0x7fb88006e470 con 0x7fb880158d90 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb87f7fe700 1 --2- 172.21.15.6:0/3187687052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 0x7fb8800ce560 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fb870009840 tx=0x7fb870009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6324737a5231c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:40.785 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.782+0000 7fb87effd700 1 -- 172.21.15.6:0/3187687052 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870011830 con 0x7fb8801589c0 2023-11-28T13:07:40.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.783+0000 7fb87effd700 1 -- 172.21.15.6:0/3187687052 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb870011990 con 0x7fb8801589c0 2023-11-28T13:07:40.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.783+0000 7fb87effd700 1 -- 172.21.15.6:0/3187687052 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870011d20 con 0x7fb8801589c0 2023-11-28T13:07:40.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.783+0000 7fb8860be700 1 -- 172.21.15.6:0/3187687052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 msgr2=0x7fb8800ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.783+0000 7fb8860be700 1 --2- 172.21.15.6:0/3187687052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 0x7fb8800ce560 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fb870009840 tx=0x7fb870009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:40.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.783+0000 7fb8860be700 1 -- 172.21.15.6:0/3187687052 shutdown_connections 2023-11-28T13:07:40.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.783+0000 7fb8860be700 1 --2- 172.21.15.6:0/3187687052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 0x7fb8800ce560 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.783+0000 7fb8860be700 1 -- 172.21.15.6:0/3187687052 >> 172.21.15.6:0/3187687052 conn(0x7fb8800b0df0 msgr2=0x7fb8800b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:40.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.783+0000 7fb8860be700 1 -- 172.21.15.6:0/3187687052 shutdown_connections 2023-11-28T13:07:40.786 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.783+0000 7fb8860be700 1 -- 172.21.15.6:0/3187687052 wait complete. 2023-11-28T13:07:40.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.784+0000 7fb8860be700 1 Processor -- start 2023-11-28T13:07:40.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.784+0000 7fb8860be700 1 -- start start 2023-11-28T13:07:40.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb8860be700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb88006e900 0x7fb8800b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb8860be700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 0x7fb88013ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.793 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb8860be700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb880158d90 0x7fb88013d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb8860be700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800b2490 con 0x7fb8801589c0 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb8860be700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800b3b90 con 0x7fb880158d90 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb8860be700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800b4410 con 0x7fb88006e900 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb877fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 0x7fb88013ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb877fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 0x7fb88013ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35386/0 (socket says 172.21.15.6:35386) 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb877fff700 1 -- 172.21.15.6:0/917710849 learned_addr learned my addr 172.21.15.6:0/917710849 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb87ffff700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb880158d90 0x7fb88013d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb87f7fe700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb88006e900 0x7fb8800b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb877fff700 1 -- 172.21.15.6:0/917710849 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb88006e900 msgr2=0x7fb8800b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb877fff700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb88006e900 0x7fb8800b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.794 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb877fff700 1 -- 172.21.15.6:0/917710849 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb880158d90 msgr2=0x7fb88013d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb877fff700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb880158d90 0x7fb88013d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb877fff700 1 -- 172.21.15.6:0/917710849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700094a0 con 0x7fb8801589c0 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.785+0000 7fb87ffff700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb880158d90 0x7fb88013d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.786+0000 7fb877fff700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 0x7fb88013ceb0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb86800c520 tx=0x7fb86800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.786+0000 7fb87cff9700 1 -- 172.21.15.6:0/917710849 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868004c80 con 0x7fb8801589c0 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.786+0000 7fb87cff9700 1 -- 172.21.15.6:0/917710849 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb868004de0 con 0x7fb8801589c0 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.786+0000 7fb87cff9700 1 -- 172.21.15.6:0/917710849 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868007310 con 0x7fb8801589c0 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.786+0000 7fb8860be700 1 -- 172.21.15.6:0/917710849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88006e100 con 0x7fb8801589c0 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.786+0000 7fb8860be700 1 -- 172.21.15.6:0/917710849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8800cee20 con 0x7fb8801589c0 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.787+0000 7fb8860be700 1 -- 172.21.15.6:0/917710849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8800c9d90 con 0x7fb8801589c0 2023-11-28T13:07:40.795 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.787+0000 7fb87cff9700 1 -- 172.21.15.6:0/917710849 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb86801a020 con 0x7fb8801589c0 2023-11-28T13:07:40.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.787+0000 7fb87cff9700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb8600682f0 0x7fb86006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:40.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.788+0000 7fb87cff9700 1 -- 172.21.15.6:0/917710849 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fb8680859d0 con 0x7fb8801589c0 2023-11-28T13:07:40.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.788+0000 7fb87f7fe700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb8600682f0 0x7fb86006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:40.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.788+0000 7fb87f7fe700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb8600682f0 0x7fb86006a7a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb8800c9230 tx=0x7fb87001c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:40.796 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.792+0000 7fb87cff9700 1 -- 172.21.15.6:0/917710849 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb8680520b0 con 0x7fb8801589c0 2023-11-28T13:07:40.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.984+0000 7fb8860be700 1 -- 172.21.15.6:0/917710849 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8800739f0 con 0x7fb8801589c0 2023-11-28T13:07:40.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.985+0000 7fb87cff9700 1 -- 172.21.15.6:0/917710849 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8680556d0 con 0x7fb8801589c0 2023-11-28T13:07:40.986 INFO:teuthology.orchestra.run.smithi006.stdout:pg_num: 64 2023-11-28T13:07:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.986+0000 7fb8860be700 1 -- 172.21.15.6:0/917710849 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb8600682f0 msgr2=0x7fb86006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.986+0000 7fb8860be700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb8600682f0 0x7fb86006a7a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb8800c9230 tx=0x7fb87001c040 comp rx=0 tx=0).stop 2023-11-28T13:07:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.986+0000 7fb8860be700 1 -- 172.21.15.6:0/917710849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 msgr2=0x7fb88013ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.986+0000 7fb8860be700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 0x7fb88013ceb0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb86800c520 tx=0x7fb86800c8e0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.986+0000 7fb8860be700 1 -- 172.21.15.6:0/917710849 shutdown_connections 2023-11-28T13:07:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.986+0000 7fb8860be700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb880158d90 0x7fb88013d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.986+0000 7fb8860be700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb8600682f0 0x7fb86006a7a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.986+0000 7fb8860be700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb88006e900 0x7fb8800b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.986+0000 7fb8860be700 1 --2- 172.21.15.6:0/917710849 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8801589c0 0x7fb88013ceb0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:40.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.986+0000 7fb8860be700 1 -- 172.21.15.6:0/917710849 >> 172.21.15.6:0/917710849 conn(0x7fb8800b0df0 msgr2=0x7fb880157840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:40.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.987+0000 7fb8860be700 1 -- 172.21.15.6:0/917710849 shutdown_connections 2023-11-28T13:07:40.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:40.987+0000 7fb8860be700 1 -- 172.21.15.6:0/917710849 wait complete. 2023-11-28T13:07:41.000 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:07:41.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.124+0000 7f3592dec700 1 Processor -- start 2023-11-28T13:07:41.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.124+0000 7f3592dec700 1 -- start start 2023-11-28T13:07:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3592dec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f358c15c990 0x7f358c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3592dec700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f358c0cc310 0x7f358c0cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3592dec700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c0ce340 con 0x7f358c161090 2023-11-28T13:07:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3592dec700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c0cebe0 con 0x7f358c0cc310 2023-11-28T13:07:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3592dec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0cccf0 con 0x7f358c15c990 2023-11-28T13:07:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3591389700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f358c0cc310 0x7f358c0cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3591389700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f358c0cc310 0x7f358c0cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35396/0 (socket says 172.21.15.6:35396) 2023-11-28T13:07:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3591389700 1 -- 172.21.15.6:0/259605059 learned_addr learned my addr 172.21.15.6:0/259605059 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:41.128 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3590b88700 1 --2- 172.21.15.6:0/259605059 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f358c15c990 0x7f358c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3591389700 1 -- 172.21.15.6:0/259605059 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f358c15c990 msgr2=0x7f358c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3591389700 1 --2- 172.21.15.6:0/259605059 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f358c15c990 0x7f358c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3591389700 1 -- 172.21.15.6:0/259605059 >> v1:172.21.15.61:6789/0 conn(0x7f358c161090 legacy=0x7f358c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3591389700 1 -- 172.21.15.6:0/259605059 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0cb3f0 con 0x7f358c0cc310 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.125+0000 7f3591389700 1 --2- 172.21.15.6:0/259605059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c0cc310 0x7f358c0cc7b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3584009e50 tx=0x7f358400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b47a4080ba6e68a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f358b7fe700 1 -- 172.21.15.6:0/259605059 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358400e070 con 0x7f358c0cc310 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f358b7fe700 1 -- 172.21.15.6:0/259605059 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358400b910 con 0x7f358c0cc310 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f358b7fe700 1 -- 172.21.15.6:0/259605059 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358400bcc0 con 0x7f358c0cc310 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f3592dec700 1 -- 172.21.15.6:0/259605059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c0cc310 msgr2=0x7f358c0cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f3592dec700 1 --2- 172.21.15.6:0/259605059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c0cc310 0x7f358c0cc7b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3584009e50 tx=0x7f358400b330 comp rx=0 tx=0).stop 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f3592dec700 1 -- 172.21.15.6:0/259605059 shutdown_connections 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f3592dec700 1 --2- 172.21.15.6:0/259605059 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c0cc310 0x7f358c0cc7b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.129 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f3592dec700 1 --2- 172.21.15.6:0/259605059 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f358c15c990 0x7f358c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f3592dec700 1 -- 172.21.15.6:0/259605059 >> 172.21.15.6:0/259605059 conn(0x7f358c0b1650 msgr2=0x7f358c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:41.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f3592dec700 1 -- 172.21.15.6:0/259605059 shutdown_connections 2023-11-28T13:07:41.130 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.126+0000 7f3592dec700 1 -- 172.21.15.6:0/259605059 wait complete. 2023-11-28T13:07:41.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.128+0000 7f3592dec700 1 Processor -- start 2023-11-28T13:07:41.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.128+0000 7f3592dec700 1 -- start start 2023-11-28T13:07:41.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3592dec700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f358c0cc310 0x7f358c1f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3592dec700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f358c15c990 0x7f358c1f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3592dec700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c161090 0x7f358c15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3592dec700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1f3330 con 0x7f358c161090 2023-11-28T13:07:41.133 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3592dec700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1f2a90 con 0x7f358c0cc310 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3592dec700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c1f2d70 con 0x7f358c15c990 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3591389700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c161090 0x7f358c15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3591389700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c161090 0x7f358c15f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35402/0 (socket says 172.21.15.6:35402) 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3591389700 1 -- 172.21.15.6:0/1536575406 learned_addr learned my addr 172.21.15.6:0/1536575406 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3590b88700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f358c0cc310 0x7f358c1f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f358bfff700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f358c15c990 0x7f358c1f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3591389700 1 -- 172.21.15.6:0/1536575406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f358c15c990 msgr2=0x7f358c1f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3591389700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f358c15c990 0x7f358c1f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3591389700 1 -- 172.21.15.6:0/1536575406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f358c0cc310 msgr2=0x7f358c1f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3591389700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f358c0cc310 0x7f358c1f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.129+0000 7f3591389700 1 -- 172.21.15.6:0/1536575406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3584009b30 con 0x7f358c161090 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.130+0000 7f3591389700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c161090 0x7f358c15f7e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f358c06eac0 tx=0x7f3584004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.130+0000 7f35897fa700 1 -- 172.21.15.6:0/1536575406 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840047c0 con 0x7f358c161090 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.130+0000 7f35897fa700 1 -- 172.21.15.6:0/1536575406 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35840099b0 con 0x7f358c161090 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.130+0000 7f35897fa700 1 -- 172.21.15.6:0/1536575406 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358401c680 con 0x7f358c161090 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.130+0000 7f3592dec700 1 -- 172.21.15.6:0/1536575406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c0ce870 con 0x7f358c161090 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.130+0000 7f3592dec700 1 -- 172.21.15.6:0/1536575406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c0ce590 con 0x7f358c161090 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.131+0000 7f3592dec700 1 -- 172.21.15.6:0/1536575406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c0cb080 con 0x7f358c161090 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.131+0000 7f35897fa700 1 -- 172.21.15.6:0/1536575406 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f358402c070 con 0x7f358c161090 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.132+0000 7f35897fa700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35700682f0 0x7f357006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.132+0000 7f35897fa700 1 -- 172.21.15.6:0/1536575406 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f3584087860 con 0x7f358c161090 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.132+0000 7f3590b88700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35700682f0 0x7f357006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.132+0000 7f3590b88700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35700682f0 0x7f357006a7a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3580009be0 tx=0x7f3580009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:41.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.136+0000 7f35897fa700 1 -- 172.21.15.6:0/1536575406 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3584053f40 con 0x7f358c161090 2023-11-28T13:07:41.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.376+0000 7f3592dec700 1 -- 172.21.15.6:0/1536575406 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f358c0675a0 con 0x7f358c161090 2023-11-28T13:07:41.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.376+0000 7f35897fa700 1 -- 172.21.15.6:0/1536575406 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4641 (secure 0 0 0) 0x7f3584057560 con 0x7f358c161090 2023-11-28T13:07:41.378 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:41.378 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1198233336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1198233336},{"type":"v1","addr":"172.21.15.6:6836","nonce":1198233336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/235204249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":235204249},{"type":"v1","addr":"172.21.15.6:6837","nonce":235204249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2675481701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2675481701},{"type":"v1","addr":"172.21.15.61:6833","nonce":2675481701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:03:16.530114+0000","modified":"2023-11-28T13:03:29.044885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.61:6835/3076579711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":3076579711},{"type":"v1","addr":"172.21.15.61:6835","nonce":3076579711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:07:41.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 -- 172.21.15.6:0/1536575406 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35700682f0 msgr2=0x7f357006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35700682f0 0x7f357006a7a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3580009be0 tx=0x7f3580009210 comp rx=0 tx=0).stop 2023-11-28T13:07:41.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 -- 172.21.15.6:0/1536575406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c161090 msgr2=0x7f358c15f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c161090 0x7f358c15f7e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f358c06eac0 tx=0x7f3584004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 -- 172.21.15.6:0/1536575406 shutdown_connections 2023-11-28T13:07:41.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f358c161090 0x7f358c15f7e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f358c0cc310 0x7f358c1f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f35700682f0 0x7f357006a7a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 --2- 172.21.15.6:0/1536575406 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f358c15c990 0x7f358c1f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 -- 172.21.15.6:0/1536575406 >> 172.21.15.6:0/1536575406 conn(0x7f358c0b1650 msgr2=0x7f358c06cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:41.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 -- 172.21.15.6:0/1536575406 shutdown_connections 2023-11-28T13:07:41.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.378+0000 7f3592dec700 1 -- 172.21.15.6:0/1536575406 wait complete. 2023-11-28T13:07:41.381 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 33 2023-11-28T13:07:41.392 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:41.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.515+0000 7fbae8263700 1 Processor -- start 2023-11-28T13:07:41.517 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.516+0000 7fbae8263700 1 -- start start 2023-11-28T13:07:41.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.516+0000 7fbae8263700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbae0161090 0x7fbae00ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae8263700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbae00caeb0 0x7fbae00cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae8263700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae00ce340 con 0x7fbae015c990 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae8263700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae00cebe0 con 0x7fbae00caeb0 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae8263700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae00cf780 con 0x7fbae0161090 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae6800700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbae00caeb0 0x7fbae00cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae5fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fbae015c990 0x7fbae00cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50958/0 (socket says 172.21.15.6:50958) 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae5fff700 1 -- 172.21.15.6:0/2186776540 learned_addr learned my addr 172.21.15.6:0/2186776540 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae57fe700 1 --2- 172.21.15.6:0/2186776540 >> v2:172.21.15.61:3300/0 conn(0x7fbae0161090 0x7fbae00ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae6800700 1 -- 172.21.15.6:0/2186776540 >> v2:172.21.15.61:3300/0 conn(0x7fbae0161090 msgr2=0x7fbae00ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae6800700 1 --2- 172.21.15.6:0/2186776540 >> v2:172.21.15.61:3300/0 conn(0x7fbae0161090 0x7fbae00ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae6800700 1 -- 172.21.15.6:0/2186776540 >> v1:172.21.15.6:6789/0 conn(0x7fbae015c990 legacy=0x7fbae00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae4ffd700 1 -- 172.21.15.6:0/2186776540 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933499777 0 0) 0x7fbae00ce340 con 0x7fbae015c990 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae6800700 1 -- 172.21.15.6:0/2186776540 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae00cb3f0 con 0x7fbae00caeb0 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae6800700 1 --2- 172.21.15.6:0/2186776540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 0x7fbae00cc310 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fbae006ab20 tx=0x7fbad800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78354df1864cdfdb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae4ffd700 1 -- 172.21.15.6:0/2186776540 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad800e070 con 0x7fbae00caeb0 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae4ffd700 1 -- 172.21.15.6:0/2186776540 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad800bbd0 con 0x7fbae00caeb0 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.517+0000 7fbae4ffd700 1 -- 172.21.15.6:0/2186776540 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad8012430 con 0x7fbae00caeb0 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.518+0000 7fbae8263700 1 -- 172.21.15.6:0/2186776540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 msgr2=0x7fbae00cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.518+0000 7fbae8263700 1 --2- 172.21.15.6:0/2186776540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 0x7fbae00cc310 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fbae006ab20 tx=0x7fbad800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.518+0000 7fbae8263700 1 -- 172.21.15.6:0/2186776540 shutdown_connections 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.518+0000 7fbae8263700 1 --2- 172.21.15.6:0/2186776540 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 0x7fbae00cc310 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.518+0000 7fbae8263700 1 --2- 172.21.15.6:0/2186776540 >> v2:172.21.15.61:3300/0 conn(0x7fbae0161090 0x7fbae00ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.518+0000 7fbae8263700 1 -- 172.21.15.6:0/2186776540 >> 172.21.15.6:0/2186776540 conn(0x7fbae00b1650 msgr2=0x7fbae00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.518+0000 7fbae8263700 1 -- 172.21.15.6:0/2186776540 shutdown_connections 2023-11-28T13:07:41.521 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.518+0000 7fbae8263700 1 -- 172.21.15.6:0/2186776540 wait complete. 2023-11-28T13:07:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.519+0000 7fbae8263700 1 Processor -- start 2023-11-28T13:07:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.519+0000 7fbae8263700 1 -- start start 2023-11-28T13:07:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae8263700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 0x7fbae014ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae8263700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbae015c990 0x7fbae0157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae8263700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbae0161090 0x7fbae01f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae8263700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae00b2950 con 0x7fbae00caeb0 2023-11-28T13:07:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae8263700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0157e30 con 0x7fbae0161090 2023-11-28T13:07:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae8263700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0158110 con 0x7fbae015c990 2023-11-28T13:07:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae5fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 0x7fbae014ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae5fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 0x7fbae014ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35412/0 (socket says 172.21.15.6:35412) 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae5fff700 1 -- 172.21.15.6:0/1591629274 learned_addr learned my addr 172.21.15.6:0/1591629274 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae5fff700 1 -- 172.21.15.6:0/1591629274 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbae015c990 msgr2=0x7fbae0157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae5fff700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbae015c990 0x7fbae0157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae5fff700 1 -- 172.21.15.6:0/1591629274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbae0161090 msgr2=0x7fbae01f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae5fff700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbae0161090 0x7fbae01f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.520+0000 7fbae5fff700 1 -- 172.21.15.6:0/1591629274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad8009a70 con 0x7fbae00caeb0 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.521+0000 7fbae5fff700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 0x7fbae014ae80 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0011e0 tx=0x7fbadc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.521+0000 7fbad67fc700 1 -- 172.21.15.6:0/1591629274 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004c80 con 0x7fbae00caeb0 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.521+0000 7fbad67fc700 1 -- 172.21.15.6:0/1591629274 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc004e00 con 0x7fbae00caeb0 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.521+0000 7fbae8263700 1 -- 172.21.15.6:0/1591629274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae00ce870 con 0x7fbae00caeb0 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.521+0000 7fbad67fc700 1 -- 172.21.15.6:0/1591629274 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc007310 con 0x7fbae00caeb0 2023-11-28T13:07:41.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.521+0000 7fbae8263700 1 -- 172.21.15.6:0/1591629274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae00ce590 con 0x7fbae00caeb0 2023-11-28T13:07:41.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.522+0000 7fbae8263700 1 -- 172.21.15.6:0/1591629274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae00cc850 con 0x7fbae00caeb0 2023-11-28T13:07:41.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.522+0000 7fbad67fc700 1 -- 172.21.15.6:0/1591629274 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbadc01a020 con 0x7fbae00caeb0 2023-11-28T13:07:41.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.523+0000 7fbad67fc700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbac40682f0 0x7fbac406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.523+0000 7fbad67fc700 1 -- 172.21.15.6:0/1591629274 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fbadc085bd0 con 0x7fbae00caeb0 2023-11-28T13:07:41.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.523+0000 7fbae57fe700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbac40682f0 0x7fbac406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.523+0000 7fbae57fe700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbac40682f0 0x7fbac406a7a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbad00098d0 tx=0x7fbad0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:41.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.527+0000 7fbad67fc700 1 -- 172.21.15.6:0/1591629274 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbadc052140 con 0x7fbae00caeb0 2023-11-28T13:07:41.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.722+0000 7fbae8263700 1 -- 172.21.15.6:0/1591629274 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbae01582d0 con 0x7fbae00caeb0 2023-11-28T13:07:41.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.723+0000 7fbad67fc700 1 -- 172.21.15.6:0/1591629274 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18127 (secure 0 0 0) 0x7fbadc055760 con 0x7fbae00caeb0 2023-11-28T13:07:41.724 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:41.725 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":68,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:04:11.062741+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:03:14.665009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T13:03:15.497374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 -- 172.21.15.6:0/1591629274 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbac40682f0 msgr2=0x7fbac406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbac40682f0 0x7fbac406a7a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbad00098d0 tx=0x7fbad0008040 comp rx=0 tx=0).stop 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 -- 172.21.15.6:0/1591629274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 msgr2=0x7fbae014ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 0x7fbae014ae80 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0011e0 tx=0x7fbadc00cae0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 -- 172.21.15.6:0/1591629274 shutdown_connections 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbae0161090 0x7fbae01f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbae00caeb0 0x7fbae014ae80 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbac40682f0 0x7fbac406a7a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 --2- 172.21.15.6:0/1591629274 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbae015c990 0x7fbae0157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 -- 172.21.15.6:0/1591629274 >> 172.21.15.6:0/1591629274 conn(0x7fbae00b1650 msgr2=0x7fbae00c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:41.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 -- 172.21.15.6:0/1591629274 shutdown_connections 2023-11-28T13:07:41.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.725+0000 7fbae8263700 1 -- 172.21.15.6:0/1591629274 wait complete. 2023-11-28T13:07:41.739 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:07:41.739 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:07:41.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.862+0000 7fa7d6438700 1 Processor -- start 2023-11-28T13:07:41.863 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.862+0000 7fa7d6438700 1 -- start start 2023-11-28T13:07:41.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.863+0000 7fa7d6438700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fa7d00b2660 0x7fa7d00b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.863+0000 7fa7d6438700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d00cfb90 con 0x7fa7d00cb230 2023-11-28T13:07:41.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.863+0000 7fa7d6438700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d00cb680 con 0x7fa7d015a6c0 2023-11-28T13:07:41.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.863+0000 7fa7d6438700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00b3020 con 0x7fa7d00b2660 2023-11-28T13:07:41.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.863+0000 7fa7cf7fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fa7d00cb230 0x7fa7d00ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50972/0 (socket says 172.21.15.6:50972) 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.863+0000 7fa7cf7fe700 1 -- 172.21.15.6:0/2712368693 learned_addr learned my addr 172.21.15.6:0/2712368693 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7d49d5700 1 --2- 172.21.15.6:0/2712368693 >> v2:172.21.15.61:3300/0 conn(0x7fa7d00b2660 0x7fa7d00b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952395700 0 0) 0x7fa7d00cfb90 con 0x7fa7d00cb230 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7bc0035b0 con 0x7fa7d00cb230 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 608812193 0 0) 0x7fa7bc0035b0 con 0x7fa7d00cb230 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 >> v2:172.21.15.61:3300/0 conn(0x7fa7d00b2660 msgr2=0x7fa7d00b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 --2- 172.21.15.6:0/2712368693 >> v2:172.21.15.61:3300/0 conn(0x7fa7d00b2660 0x7fa7d00b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 >> v1:172.21.15.61:6789/0 conn(0x7fa7d015a6c0 legacy=0x7fa7d00d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00cb930 con 0x7fa7d00cb230 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fa7b8001670 con 0x7fa7d00cb230 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 >> v1:172.21.15.6:6789/0 conn(0x7fa7d00cb230 legacy=0x7fa7d00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa7bc003a00 0x7fa7bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7bc0043f0 0x7fa7bc0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7bc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00b3020 con 0x7fa7bc0050b0 2023-11-28T13:07:41.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc00bf50 con 0x7fa7bc003a00 2023-11-28T13:07:41.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc00c110 con 0x7fa7bc0043f0 2023-11-28T13:07:41.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7d49d5700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7bc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7cffff700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa7bc003a00 0x7fa7bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.864+0000 7fa7cf7fe700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7bc0043f0 0x7fa7bc0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d49d5700 1 -- 172.21.15.6:0/2712368693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7bc0043f0 msgr2=0x7fa7bc0048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d49d5700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7bc0043f0 0x7fa7bc0048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d49d5700 1 -- 172.21.15.6:0/2712368693 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa7bc003a00 msgr2=0x7fa7bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d49d5700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa7bc003a00 0x7fa7bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d49d5700 1 -- 172.21.15.6:0/2712368693 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0035b0 con 0x7fa7bc0050b0 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7cf7fe700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7bc0043f0 0x7fa7bc0048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d49d5700 1 -- 172.21.15.6:0/2712368693 reap_dead start 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d49d5700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7bc00ba10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8004bb0 tx=0x7fa7bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72c3234707b1ff0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8003c20 con 0x7fa7bc0050b0 2023-11-28T13:07:41.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c800f040 con 0x7fa7bc0050b0 2023-11-28T13:07:41.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7ceffd700 1 -- 172.21.15.6:0/2712368693 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80133e0 con 0x7fa7bc0050b0 2023-11-28T13:07:41.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d6438700 1 -- 172.21.15.6:0/2712368693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 msgr2=0x7fa7bc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d6438700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7bc00ba10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8004bb0 tx=0x7fa7bc005880 comp rx=0 tx=0).stop 2023-11-28T13:07:41.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d6438700 1 -- 172.21.15.6:0/2712368693 shutdown_connections 2023-11-28T13:07:41.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d6438700 1 --2- 172.21.15.6:0/2712368693 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7bc00ba10 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.865+0000 7fa7d6438700 1 -- 172.21.15.6:0/2712368693 >> 172.21.15.6:0/2712368693 conn(0x7fa7d00cd660 msgr2=0x7fa7d00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:41.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.866+0000 7fa7d6438700 1 -- 172.21.15.6:0/2712368693 shutdown_connections 2023-11-28T13:07:41.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.866+0000 7fa7d6438700 1 -- 172.21.15.6:0/2712368693 wait complete. 2023-11-28T13:07:41.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.867+0000 7fa7d6438700 1 Processor -- start 2023-11-28T13:07:41.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.867+0000 7fa7d6438700 1 -- start start 2023-11-28T13:07:41.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7d6438700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7d015c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7d6438700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7d01589c0 0x7fa7d00b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7d6438700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa7d00b5f60 0x7fa7d01efa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7d6438700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d015cb50 con 0x7fa7bc0050b0 2023-11-28T13:07:41.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7d6438700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00b63d0 con 0x7fa7d00b5f60 2023-11-28T13:07:41.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7d6438700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00c8ea0 con 0x7fa7d01589c0 2023-11-28T13:07:41.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7cffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7d015c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7cffff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7d015c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35430/0 (socket says 172.21.15.6:35430) 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7cffff700 1 -- 172.21.15.6:0/3052342153 learned_addr learned my addr 172.21.15.6:0/3052342153 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7d49d5700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa7d00b5f60 0x7fa7d01efa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7cf7fe700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7d01589c0 0x7fa7d00b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7cffff700 1 -- 172.21.15.6:0/3052342153 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7d01589c0 msgr2=0x7fa7d00b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7cffff700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7d01589c0 0x7fa7d00b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7cffff700 1 -- 172.21.15.6:0/3052342153 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa7d00b5f60 msgr2=0x7fa7d01efa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7cffff700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa7d00b5f60 0x7fa7d01efa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7cffff700 1 -- 172.21.15.6:0/3052342153 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00cf540 con 0x7fa7bc0050b0 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7cffff700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7d015c610 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000f7b0 tx=0x7fa7c000fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7ccff9700 1 -- 172.21.15.6:0/3052342153 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00052e0 con 0x7fa7bc0050b0 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7ccff9700 1 -- 172.21.15.6:0/3052342153 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c0005460 con 0x7fa7bc0050b0 2023-11-28T13:07:41.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7d6438700 1 -- 172.21.15.6:0/3052342153 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d00cf260 con 0x7fa7bc0050b0 2023-11-28T13:07:41.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7ccff9700 1 -- 172.21.15.6:0/3052342153 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00038c0 con 0x7fa7bc0050b0 2023-11-28T13:07:41.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.868+0000 7fa7d6438700 1 -- 172.21.15.6:0/3052342153 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d00cf820 con 0x7fa7bc0050b0 2023-11-28T13:07:41.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.869+0000 7fa7d6438700 1 -- 172.21.15.6:0/3052342153 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d00675a0 con 0x7fa7bc0050b0 2023-11-28T13:07:41.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.870+0000 7fa7ccff9700 1 -- 172.21.15.6:0/3052342153 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa7c0003a20 con 0x7fa7bc0050b0 2023-11-28T13:07:41.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.870+0000 7fa7ccff9700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa7b00682f0 0x7fa7b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:41.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.870+0000 7fa7ccff9700 1 -- 172.21.15.6:0/3052342153 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa7c0085df0 con 0x7fa7bc0050b0 2023-11-28T13:07:41.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.870+0000 7fa7cf7fe700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa7b00682f0 0x7fa7b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:41.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.871+0000 7fa7cf7fe700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa7b00682f0 0x7fa7b006a7a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8000c00 tx=0x7fa7b8001b40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:41.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:41.874+0000 7fa7ccff9700 1 -- 172.21.15.6:0/3052342153 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7c00524d0 con 0x7fa7bc0050b0 2023-11-28T13:07:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.112+0000 7fa7d6438700 1 -- 172.21.15.6:0/3052342153 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7d015ccf0 con 0x7fa7bc0050b0 2023-11-28T13:07:42.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.112+0000 7fa7ccff9700 1 -- 172.21.15.6:0/3052342153 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4641 (secure 0 0 0) 0x7fa7c0003cb0 con 0x7fa7bc0050b0 2023-11-28T13:07:42.113 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:42.114 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1198233336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1198233336},{"type":"v1","addr":"172.21.15.6:6836","nonce":1198233336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/235204249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":235204249},{"type":"v1","addr":"172.21.15.6:6837","nonce":235204249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2675481701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2675481701},{"type":"v1","addr":"172.21.15.61:6833","nonce":2675481701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:03:16.530114+0000","modified":"2023-11-28T13:03:29.044885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.61:6835/3076579711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":3076579711},{"type":"v1","addr":"172.21.15.61:6835","nonce":3076579711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 -- 172.21.15.6:0/3052342153 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa7b00682f0 msgr2=0x7fa7b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa7b00682f0 0x7fa7b006a7a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8000c00 tx=0x7fa7b8001b40 comp rx=0 tx=0).stop 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 -- 172.21.15.6:0/3052342153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 msgr2=0x7fa7d015c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7d015c610 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000f7b0 tx=0x7fa7c000fb70 comp rx=0 tx=0).stop 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 -- 172.21.15.6:0/3052342153 shutdown_connections 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa7d00b5f60 0x7fa7d01efa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa7bc0050b0 0x7fa7d015c610 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fa7b00682f0 0x7fa7b006a7a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 --2- 172.21.15.6:0/3052342153 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fa7d01589c0 0x7fa7d00b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 -- 172.21.15.6:0/3052342153 >> 172.21.15.6:0/3052342153 conn(0x7fa7d00cd660 msgr2=0x7fa7d00ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 -- 172.21.15.6:0/3052342153 shutdown_connections 2023-11-28T13:07:42.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.114+0000 7fa7d6438700 1 -- 172.21.15.6:0/3052342153 wait complete. 2023-11-28T13:07:42.117 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 33 2023-11-28T13:07:42.128 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:07:42.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.252+0000 7fda8137d700 1 Processor -- start 2023-11-28T13:07:42.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.252+0000 7fda8137d700 1 -- start start 2023-11-28T13:07:42.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.252+0000 7fda8137d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fda7c06e400 0x7fda7c15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.252+0000 7fda8137d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c0ce890 0x7fda7c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.252+0000 7fda8137d700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7c15e0b0 con 0x7fda7c159590 2023-11-28T13:07:42.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.252+0000 7fda8137d700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7c0cf250 con 0x7fda7c06e400 2023-11-28T13:07:42.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.252+0000 7fda8137d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c15d120 con 0x7fda7c0ce890 2023-11-28T13:07:42.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda7a7fc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fda7c06e400 0x7fda7c15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda7affd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fda7c159590 0x7fda7c0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:50990/0 (socket says 172.21.15.6:50990) 2023-11-28T13:07:42.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda7affd700 1 -- 172.21.15.6:0/3503295260 learned_addr learned my addr 172.21.15.6:0/3503295260 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:42.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda7b7fe700 1 --2- 172.21.15.6:0/3503295260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c0ce890 0x7fda7c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda7a7fc700 1 -- 172.21.15.6:0/3503295260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c0ce890 msgr2=0x7fda7c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda7a7fc700 1 --2- 172.21.15.6:0/3503295260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c0ce890 0x7fda7c0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda7a7fc700 1 -- 172.21.15.6:0/3503295260 >> v1:172.21.15.6:6789/0 conn(0x7fda7c159590 legacy=0x7fda7c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda79ffb700 1 -- 172.21.15.6:0/3503295260 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739097714 0 0) 0x7fda6c001110 con 0x7fda7c159590 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda7a7fc700 1 -- 172.21.15.6:0/3503295260 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c15d6e0 con 0x7fda7c06e400 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda7b7fe700 1 --2- 172.21.15.6:0/3503295260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c0ce890 0x7fda7c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda7a7fc700 1 --2- 172.21.15.6:0/3503295260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c06e400 0x7fda7c15db70 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fda70009840 tx=0x7fda70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcf538669afc080 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda79ffb700 1 -- 172.21.15.6:0/3503295260 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70011830 con 0x7fda7c06e400 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda79ffb700 1 -- 172.21.15.6:0/3503295260 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda70011990 con 0x7fda7c06e400 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.253+0000 7fda79ffb700 1 -- 172.21.15.6:0/3503295260 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70011d20 con 0x7fda7c06e400 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.254+0000 7fda8137d700 1 -- 172.21.15.6:0/3503295260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c06e400 msgr2=0x7fda7c15db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.254+0000 7fda8137d700 1 --2- 172.21.15.6:0/3503295260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c06e400 0x7fda7c15db70 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fda70009840 tx=0x7fda70009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:42.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.254+0000 7fda8137d700 1 -- 172.21.15.6:0/3503295260 shutdown_connections 2023-11-28T13:07:42.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.254+0000 7fda8137d700 1 --2- 172.21.15.6:0/3503295260 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c0ce890 0x7fda7c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.254+0000 7fda8137d700 1 --2- 172.21.15.6:0/3503295260 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c06e400 0x7fda7c15db70 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.254+0000 7fda8137d700 1 -- 172.21.15.6:0/3503295260 >> 172.21.15.6:0/3503295260 conn(0x7fda7c0b1650 msgr2=0x7fda7c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:42.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.254+0000 7fda8137d700 1 -- 172.21.15.6:0/3503295260 shutdown_connections 2023-11-28T13:07:42.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.254+0000 7fda8137d700 1 -- 172.21.15.6:0/3503295260 wait complete. 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.256+0000 7fda8137d700 1 Processor -- start 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.256+0000 7fda8137d700 1 -- start start 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda8137d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda7c06e400 0x7fda7c1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda8137d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c0ce890 0x7fda7c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda8137d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c159590 0x7fda7c15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda8137d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c15bbf0 con 0x7fda7c0ce890 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda8137d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1587d0 con 0x7fda7c06e400 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda8137d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c1547e0 con 0x7fda7c159590 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7a7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c0ce890 0x7fda7c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7a7fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c0ce890 0x7fda7c15a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35446/0 (socket says 172.21.15.6:35446) 2023-11-28T13:07:42.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7a7fc700 1 -- 172.21.15.6:0/3259533445 learned_addr learned my addr 172.21.15.6:0/3259533445 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:42.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7b7fe700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c159590 0x7fda7c15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7affd700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda7c06e400 0x7fda7c1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7a7fc700 1 -- 172.21.15.6:0/3259533445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c159590 msgr2=0x7fda7c15c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7a7fc700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c159590 0x7fda7c15c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7a7fc700 1 -- 172.21.15.6:0/3259533445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda7c06e400 msgr2=0x7fda7c1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7a7fc700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda7c06e400 0x7fda7c1542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7a7fc700 1 -- 172.21.15.6:0/3259533445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda700094a0 con 0x7fda7c0ce890 2023-11-28T13:07:42.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7b7fe700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c159590 0x7fda7c15c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:42.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.257+0000 7fda7a7fc700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c0ce890 0x7fda7c15a800 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fda70000c00 tx=0x7fda70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.258+0000 7fda637fe700 1 -- 172.21.15.6:0/3259533445 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda700045f0 con 0x7fda7c0ce890 2023-11-28T13:07:42.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.258+0000 7fda637fe700 1 -- 172.21.15.6:0/3259533445 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7001f070 con 0x7fda7c0ce890 2023-11-28T13:07:42.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.258+0000 7fda8137d700 1 -- 172.21.15.6:0/3259533445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c0af240 con 0x7fda7c0ce890 2023-11-28T13:07:42.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.258+0000 7fda637fe700 1 -- 172.21.15.6:0/3259533445 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70003ae0 con 0x7fda7c0ce890 2023-11-28T13:07:42.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.258+0000 7fda8137d700 1 -- 172.21.15.6:0/3259533445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c0aef60 con 0x7fda7c0ce890 2023-11-28T13:07:42.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.259+0000 7fda8137d700 1 -- 172.21.15.6:0/3259533445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c15d370 con 0x7fda7c0ce890 2023-11-28T13:07:42.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.259+0000 7fda637fe700 1 -- 172.21.15.6:0/3259533445 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fda7000f050 con 0x7fda7c0ce890 2023-11-28T13:07:42.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.259+0000 7fda637fe700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda680682f0 0x7fda6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.260+0000 7fda637fe700 1 -- 172.21.15.6:0/3259533445 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fda70021070 con 0x7fda7c0ce890 2023-11-28T13:07:42.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.260+0000 7fda7affd700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda680682f0 0x7fda6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.260+0000 7fda7affd700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda680682f0 0x7fda6806a7a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000f60 tx=0x7fda6c007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.264+0000 7fda637fe700 1 -- 172.21.15.6:0/3259533445 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fda70053f70 con 0x7fda7c0ce890 2023-11-28T13:07:42.500 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.499+0000 7fda8137d700 1 -- 172.21.15.6:0/3259533445 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda7c15ad40 con 0x7fda7c0ce890 2023-11-28T13:07:42.501 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.499+0000 7fda637fe700 1 -- 172.21.15.6:0/3259533445 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4641 (secure 0 0 0) 0x7fda70015090 con 0x7fda7c0ce890 2023-11-28T13:07:42.501 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:42.501 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1198233336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1198233336},{"type":"v1","addr":"172.21.15.6:6836","nonce":1198233336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/235204249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":235204249},{"type":"v1","addr":"172.21.15.6:6837","nonce":235204249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2675481701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2675481701},{"type":"v1","addr":"172.21.15.61:6833","nonce":2675481701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T13:03:16.530114+0000","modified":"2023-11-28T13:03:29.044885+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6471},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6471":{"gid":6471,"name":"b","rank":0,"incarnation":28,"state":"up:active","state_seq":6,"addr":"172.21.15.61:6835/3076579711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":3076579711},{"type":"v1","addr":"172.21.15.61:6835","nonce":3076579711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T13:07:42.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.501+0000 7fda8137d700 1 -- 172.21.15.6:0/3259533445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda680682f0 msgr2=0x7fda6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.501+0000 7fda8137d700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda680682f0 0x7fda6806a7a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fda6c000f60 tx=0x7fda6c007c90 comp rx=0 tx=0).stop 2023-11-28T13:07:42.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.501+0000 7fda8137d700 1 -- 172.21.15.6:0/3259533445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c0ce890 msgr2=0x7fda7c15a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.501+0000 7fda8137d700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c0ce890 0x7fda7c15a800 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fda70000c00 tx=0x7fda70004750 comp rx=0 tx=0).stop 2023-11-28T13:07:42.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.502+0000 7fda8137d700 1 -- 172.21.15.6:0/3259533445 shutdown_connections 2023-11-28T13:07:42.503 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.502+0000 7fda8137d700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fda680682f0 0x7fda6806a7a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.502+0000 7fda8137d700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fda7c159590 0x7fda7c15c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.502+0000 7fda8137d700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda7c06e400 0x7fda7c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.502+0000 7fda8137d700 1 --2- 172.21.15.6:0/3259533445 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fda7c0ce890 0x7fda7c15a800 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.502+0000 7fda8137d700 1 -- 172.21.15.6:0/3259533445 >> 172.21.15.6:0/3259533445 conn(0x7fda7c0b1650 msgr2=0x7fda7c0b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.502+0000 7fda8137d700 1 -- 172.21.15.6:0/3259533445 shutdown_connections 2023-11-28T13:07:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.502+0000 7fda8137d700 1 -- 172.21.15.6:0/3259533445 wait complete. 2023-11-28T13:07:42.504 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 33 2023-11-28T13:07:42.515 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:42.639 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.638+0000 7fc1a9e9d700 1 Processor -- start 2023-11-28T13:07:42.640 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.638+0000 7fc1a9e9d700 1 -- start start 2023-11-28T13:07:42.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.639+0000 7fc1a9e9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc1a40cefa0 0x7fc1a415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.639+0000 7fc1a9e9d700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a415e0b0 con 0x7fc1a4159590 2023-11-28T13:07:42.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.639+0000 7fc1a9e9d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a415d120 con 0x7fc1a406e400 2023-11-28T13:07:42.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.639+0000 7fc1a9e9d700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a415d6e0 con 0x7fc1a40cefa0 2023-11-28T13:07:42.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.639+0000 7fc1a3fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc1a40cefa0 0x7fc1a415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.639+0000 7fc1a37fe700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fc1a4159590 0x7fc1a40ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51004/0 (socket says 172.21.15.6:51004) 2023-11-28T13:07:42.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.639+0000 7fc1a37fe700 1 -- 172.21.15.6:0/2936505684 learned_addr learned my addr 172.21.15.6:0/2936505684 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:42.642 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.639+0000 7fc1a3fff700 1 -- 172.21.15.6:0/2936505684 >> v1:172.21.15.61:6789/0 conn(0x7fc1a406e400 legacy=0x7fc1a40ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a3fff700 1 -- 172.21.15.6:0/2936505684 >> v1:172.21.15.6:6789/0 conn(0x7fc1a4159590 legacy=0x7fc1a40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a27fc700 1 -- 172.21.15.6:0/2936505684 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780445371 0 0) 0x7fc18c001110 con 0x7fc1a4159590 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a3fff700 1 -- 172.21.15.6:0/2936505684 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40aef60 con 0x7fc1a40cefa0 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a3fff700 1 --2- 172.21.15.6:0/2936505684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 0x7fc1a415db70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009d90 tx=0x7fc19c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b9005a34aadc1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a27fc700 1 -- 172.21.15.6:0/2936505684 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00e070 con 0x7fc1a40cefa0 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a27fc700 1 -- 172.21.15.6:0/2936505684 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c00bb90 con 0x7fc1a40cefa0 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a27fc700 1 -- 172.21.15.6:0/2936505684 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c003a20 con 0x7fc1a40cefa0 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2936505684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 msgr2=0x7fc1a415db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.643 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a9e9d700 1 --2- 172.21.15.6:0/2936505684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 0x7fc1a415db70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009d90 tx=0x7fc19c00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:42.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2936505684 shutdown_connections 2023-11-28T13:07:42.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a9e9d700 1 --2- 172.21.15.6:0/2936505684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 0x7fc1a415db70 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.640+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2936505684 >> 172.21.15.6:0/2936505684 conn(0x7fc1a40b1650 msgr2=0x7fc1a40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:42.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.641+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2936505684 shutdown_connections 2023-11-28T13:07:42.644 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.641+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2936505684 wait complete. 2023-11-28T13:07:42.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.642+0000 7fc1a9e9d700 1 Processor -- start 2023-11-28T13:07:42.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.642+0000 7fc1a9e9d700 1 -- start start 2023-11-28T13:07:42.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a9e9d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1a406e400 0x7fc1a4158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a9e9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 0x7fc1a40afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a9e9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1a4159590 0x7fc1a40aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a9e9d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40c79d0 con 0x7fc1a40cefa0 2023-11-28T13:07:42.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a9e9d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40c7c70 con 0x7fc1a4159590 2023-11-28T13:07:42.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a9e9d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40c7f50 con 0x7fc1a406e400 2023-11-28T13:07:42.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 0x7fc1a40afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a2ffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 0x7fc1a40afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35460/0 (socket says 172.21.15.6:35460) 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a2ffd700 1 -- 172.21.15.6:0/2618964680 learned_addr learned my addr 172.21.15.6:0/2618964680 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a3fff700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1a4159590 0x7fc1a40aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a37fe700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1a406e400 0x7fc1a4158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a2ffd700 1 -- 172.21.15.6:0/2618964680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1a406e400 msgr2=0x7fc1a4158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a2ffd700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1a406e400 0x7fc1a4158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a2ffd700 1 -- 172.21.15.6:0/2618964680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1a4159590 msgr2=0x7fc1a40aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a2ffd700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1a4159590 0x7fc1a40aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a2ffd700 1 -- 172.21.15.6:0/2618964680 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c009a70 con 0x7fc1a40cefa0 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a3fff700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1a4159590 0x7fc1a40aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:42.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a2ffd700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 0x7fc1a40afa20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b4a0 tx=0x7fc19400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc19bfff700 1 -- 172.21.15.6:0/2618964680 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194011830 con 0x7fc1a40cefa0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc19bfff700 1 -- 172.21.15.6:0/2618964680 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc194011990 con 0x7fc1a40cefa0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc19bfff700 1 -- 172.21.15.6:0/2618964680 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194011d40 con 0x7fc1a40cefa0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.643+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2618964680 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a415d370 con 0x7fc1a40cefa0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.644+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2618964680 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a40af240 con 0x7fc1a40cefa0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.645+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2618964680 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a4159960 con 0x7fc1a40cefa0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.645+0000 7fc19bfff700 1 -- 172.21.15.6:0/2618964680 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc19400f050 con 0x7fc1a40cefa0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.645+0000 7fc19bfff700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc1840682f0 0x7fc18406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.645+0000 7fc19bfff700 1 -- 172.21.15.6:0/2618964680 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fc194085950 con 0x7fc1a40cefa0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.645+0000 7fc1a37fe700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc1840682f0 0x7fc18406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.646+0000 7fc1a37fe700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc1840682f0 0x7fc18406a7a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000f60 tx=0x7fc18c007c20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.649+0000 7fc19bfff700 1 -- 172.21.15.6:0/2618964680 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc194052030 con 0x7fc1a40cefa0 2023-11-28T13:07:42.842 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.841+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2618964680 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1a40675a0 con 0x7fc1a40cefa0 2023-11-28T13:07:42.843 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.842+0000 7fc19bfff700 1 -- 172.21.15.6:0/2618964680 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18127 (secure 0 0 0) 0x7fc194015090 con 0x7fc1a40cefa0 2023-11-28T13:07:42.844 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:42.844 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":68,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:04:11.062741+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T13:03:14.665009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T13:03:15.497374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:42.845 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2618964680 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc1840682f0 msgr2=0x7fc18406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc1840682f0 0x7fc18406a7a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc18c000f60 tx=0x7fc18c007c20 comp rx=0 tx=0).stop 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2618964680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 msgr2=0x7fc1a40afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 0x7fc1a40afa20 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc19400b4a0 tx=0x7fc19400b7b0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2618964680 shutdown_connections 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1a4159590 0x7fc1a40aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc1840682f0 0x7fc18406a7a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc1a406e400 0x7fc1a4158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 --2- 172.21.15.6:0/2618964680 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc1a40cefa0 0x7fc1a40afa20 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2618964680 >> 172.21.15.6:0/2618964680 conn(0x7fc1a40b1650 msgr2=0x7fc1a40ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2618964680 shutdown_connections 2023-11-28T13:07:42.846 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.844+0000 7fc1a9e9d700 1 -- 172.21.15.6:0/2618964680 wait complete. 2023-11-28T13:07:42.858 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T13:07:42.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.982+0000 7f1a0605b700 1 Processor -- start 2023-11-28T13:07:42.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.982+0000 7f1a0605b700 1 -- start start 2023-11-28T13:07:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.983+0000 7f1a0605b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0015a230 0x7f1a0015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.983+0000 7f1a0605b700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0006abb0 con 0x7f1a000b4fe0 2023-11-28T13:07:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.983+0000 7f1a0605b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0006a280 con 0x7f1a000cd4a0 2023-11-28T13:07:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.983+0000 7f1a0605b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0015a600 con 0x7f1a0015a230 2023-11-28T13:07:42.986 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.983+0000 7f19fffff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1a000b4fe0 0x7f1a000d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51022/0 (socket says 172.21.15.6:51022) 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.983+0000 7f19fffff700 1 -- 172.21.15.6:0/343755697 learned_addr learned my addr 172.21.15.6:0/343755697 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19ff7fe700 1 --2- 172.21.15.6:0/343755697 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0015a230 0x7f1a0015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19fe7fc700 1 -- 172.21.15.6:0/343755697 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144945357 0 0) 0x7f1a0006abb0 con 0x7f1a000b4fe0 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19fe7fc700 1 -- 172.21.15.6:0/343755697 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e80035b0 con 0x7f1a000b4fe0 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19ff7fe700 1 -- 172.21.15.6:0/343755697 >> v1:172.21.15.61:6789/0 conn(0x7f1a000cd4a0 legacy=0x7f1a000b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19fe7fc700 1 -- 172.21.15.6:0/343755697 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025537114 0 0) 0x7f1a0006a280 con 0x7f1a000cd4a0 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19ff7fe700 1 -- 172.21.15.6:0/343755697 >> v1:172.21.15.6:6789/0 conn(0x7f1a000b4fe0 legacy=0x7f1a000d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19ff7fe700 1 -- 172.21.15.6:0/343755697 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0006a560 con 0x7f1a0015a230 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19ff7fe700 1 --2- 172.21.15.6:0/343755697 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0015a230 0x7f1a0015d6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009870 tx=0x7f19f4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7720ad30289981 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19fe7fc700 1 -- 172.21.15.6:0/343755697 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f80019f0 con 0x7f1a0015a230 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19fe7fc700 1 -- 172.21.15.6:0/343755697 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f4004030 con 0x7f1a0015a230 2023-11-28T13:07:42.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.984+0000 7f19fe7fc700 1 -- 172.21.15.6:0/343755697 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f40043c0 con 0x7f1a0015a230 2023-11-28T13:07:42.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.985+0000 7f1a0605b700 1 -- 172.21.15.6:0/343755697 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0015a230 msgr2=0x7f1a0015d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.985+0000 7f1a0605b700 1 --2- 172.21.15.6:0/343755697 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0015a230 0x7f1a0015d6b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009870 tx=0x7f19f4009b80 comp rx=0 tx=0).stop 2023-11-28T13:07:42.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.985+0000 7f1a0605b700 1 -- 172.21.15.6:0/343755697 shutdown_connections 2023-11-28T13:07:42.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.985+0000 7f1a0605b700 1 --2- 172.21.15.6:0/343755697 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a0015a230 0x7f1a0015d6b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.985+0000 7f1a0605b700 1 -- 172.21.15.6:0/343755697 >> 172.21.15.6:0/343755697 conn(0x7f1a000cf8d0 msgr2=0x7f1a000cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:42.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.985+0000 7f1a0605b700 1 -- 172.21.15.6:0/343755697 shutdown_connections 2023-11-28T13:07:42.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.985+0000 7f1a0605b700 1 -- 172.21.15.6:0/343755697 wait complete. 2023-11-28T13:07:42.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.986+0000 7f1a0605b700 1 Processor -- start 2023-11-28T13:07:42.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.986+0000 7f1a0605b700 1 -- start start 2023-11-28T13:07:42.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f1a0605b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a000b4fe0 0x7f1a00157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f1a0605b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a000cd4a0 0x7f1a000d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f1a0605b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0015a230 0x7f1a000cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f1a0605b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000d1d90 con 0x7f1a000b4fe0 2023-11-28T13:07:42.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f1a0605b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000d1180 con 0x7f1a0015a230 2023-11-28T13:07:42.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f1a0605b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000d1460 con 0x7f1a000cd4a0 2023-11-28T13:07:42.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a000b4fe0 0x7f1a00157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19ff7fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a000b4fe0 0x7f1a00157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35472/0 (socket says 172.21.15.6:35472) 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19ff7fe700 1 -- 172.21.15.6:0/3981399014 learned_addr learned my addr 172.21.15.6:0/3981399014 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19feffd700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a000cd4a0 0x7f1a000d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19fffff700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0015a230 0x7f1a000cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19ff7fe700 1 -- 172.21.15.6:0/3981399014 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a000cd4a0 msgr2=0x7f1a000d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19ff7fe700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a000cd4a0 0x7f1a000d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19ff7fe700 1 -- 172.21.15.6:0/3981399014 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0015a230 msgr2=0x7f1a000cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19ff7fe700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0015a230 0x7f1a000cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19ff7fe700 1 -- 172.21.15.6:0/3981399014 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40094a0 con 0x7f1a000b4fe0 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.987+0000 7f19fffff700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0015a230 0x7f1a000cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.988+0000 7f19ff7fe700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a000b4fe0 0x7f1a00157f70 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f19f40097c0 tx=0x7f19f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.988+0000 7f19e7fff700 1 -- 172.21.15.6:0/3981399014 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4010590 con 0x7f1a000b4fe0 2023-11-28T13:07:42.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.988+0000 7f19e7fff700 1 -- 172.21.15.6:0/3981399014 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f40106f0 con 0x7f1a000b4fe0 2023-11-28T13:07:42.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.988+0000 7f1a0605b700 1 -- 172.21.15.6:0/3981399014 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a000cd910 con 0x7f1a000b4fe0 2023-11-28T13:07:42.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.988+0000 7f19e7fff700 1 -- 172.21.15.6:0/3981399014 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4010aa0 con 0x7f1a000b4fe0 2023-11-28T13:07:42.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.988+0000 7f1a0605b700 1 -- 172.21.15.6:0/3981399014 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0006a840 con 0x7f1a000b4fe0 2023-11-28T13:07:42.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.989+0000 7f1a0605b700 1 -- 172.21.15.6:0/3981399014 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0006b040 con 0x7f1a000b4fe0 2023-11-28T13:07:42.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.989+0000 7f19e7fff700 1 -- 172.21.15.6:0/3981399014 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19f400f050 con 0x7f1a000b4fe0 2023-11-28T13:07:42.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.990+0000 7f19e7fff700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19ec0682f0 0x7f19ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:42.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.990+0000 7f19e7fff700 1 -- 172.21.15.6:0/3981399014 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f19f4087590 con 0x7f1a000b4fe0 2023-11-28T13:07:42.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.990+0000 7f19feffd700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19ec0682f0 0x7f19ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:42.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.990+0000 7f19feffd700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19ec0682f0 0x7f19ec06a7a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1a001f62a0 tx=0x7f19f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:42.995 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:42.994+0000 7f19e7fff700 1 -- 172.21.15.6:0/3981399014 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19f4053c70 con 0x7f1a000b4fe0 2023-11-28T13:07:43.234 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:43.233+0000 7f1a0605b700 1 -- 172.21.15.6:0/3981399014 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1a000675a0 con 0x7f1a000b4fe0 2023-11-28T13:07:44.124 INFO:tasks.ceph.mon.a.smithi006.stderr:2023-11-28T13:07:44.123+0000 7fa6275a1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T13:07:44.131 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.130+0000 7f19e7fff700 1 -- 172.21.15.6:0/3981399014 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f19f4057290 con 0x7f1a000b4fe0 2023-11-28T13:07:44.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 -- 172.21.15.6:0/3981399014 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19ec0682f0 msgr2=0x7f19ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19ec0682f0 0x7f19ec06a7a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1a001f62a0 tx=0x7f19f0009040 comp rx=0 tx=0).stop 2023-11-28T13:07:44.134 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 -- 172.21.15.6:0/3981399014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a000b4fe0 msgr2=0x7f1a00157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a000b4fe0 0x7f1a00157f70 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f19f40097c0 tx=0x7f19f4004750 comp rx=0 tx=0).stop 2023-11-28T13:07:44.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 -- 172.21.15.6:0/3981399014 shutdown_connections 2023-11-28T13:07:44.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1a0015a230 0x7f1a000cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a000b4fe0 0x7f1a00157f70 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f19ec0682f0 0x7f19ec06a7a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 --2- 172.21.15.6:0/3981399014 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f1a000cd4a0 0x7f1a000d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 -- 172.21.15.6:0/3981399014 >> 172.21.15.6:0/3981399014 conn(0x7f1a000cf8d0 msgr2=0x7f1a000b74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:44.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 -- 172.21.15.6:0/3981399014 shutdown_connections 2023-11-28T13:07:44.135 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.133+0000 7f1a0605b700 1 -- 172.21.15.6:0/3981399014 wait complete. 2023-11-28T13:07:44.136 INFO:teuthology.orchestra.run.smithi006.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T13:07:44.150 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T13:07:44.173 INFO:tasks.ceph.mds.b.smithi061.stdout:starting mds.b at 2023-11-28T13:07:44.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.283+0000 7f7720fb4700 1 Processor -- start 2023-11-28T13:07:44.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.284+0000 7f7720fb4700 1 -- start start 2023-11-28T13:07:44.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.284+0000 7f7720fb4700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f771c0b1d40 0x7f771c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.284+0000 7f7720fb4700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f771c15dbf0 0x7f771c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.284+0000 7f7720fb4700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771c0ceff0 con 0x7f771c159500 2023-11-28T13:07:44.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.284+0000 7f7720fb4700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771c15e090 con 0x7f771c15dbf0 2023-11-28T13:07:44.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.284+0000 7f7720fb4700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771c0cede0 con 0x7f771c0b1d40 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771ad9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f771c15dbf0 0x7f771c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771a59c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f771c159500 0x7f771c0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51032/0 (socket says 172.21.15.6:51032) 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771a59c700 1 -- 172.21.15.6:0/951095391 learned_addr learned my addr 172.21.15.6:0/951095391 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f7719d9b700 1 --2- 172.21.15.6:0/951095391 >> v2:172.21.15.61:3300/0 conn(0x7f771c0b1d40 0x7f771c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771ad9d700 1 -- 172.21.15.6:0/951095391 >> v2:172.21.15.61:3300/0 conn(0x7f771c0b1d40 msgr2=0x7f771c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771ad9d700 1 --2- 172.21.15.6:0/951095391 >> v2:172.21.15.61:3300/0 conn(0x7f771c0b1d40 0x7f771c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771ad9d700 1 -- 172.21.15.6:0/951095391 >> v1:172.21.15.6:6789/0 conn(0x7f771c159500 legacy=0x7f771c0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771ad9d700 1 -- 172.21.15.6:0/951095391 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c0cf2d0 con 0x7f771c15dbf0 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771959a700 1 -- 172.21.15.6:0/951095391 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007763894 0 0) 0x7f771c0ceff0 con 0x7f771c159500 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771ad9d700 1 --2- 172.21.15.6:0/951095391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c15dbf0 0x7f771c0ce8a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f771c0d0c50 tx=0x7f771400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad13e1fdb466ca80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771959a700 1 -- 172.21.15.6:0/951095391 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771400e070 con 0x7f771c15dbf0 2023-11-28T13:07:44.288 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771959a700 1 -- 172.21.15.6:0/951095391 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771400bbd0 con 0x7f771c15dbf0 2023-11-28T13:07:44.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.285+0000 7f771959a700 1 -- 172.21.15.6:0/951095391 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714012430 con 0x7f771c15dbf0 2023-11-28T13:07:44.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.286+0000 7f7720fb4700 1 -- 172.21.15.6:0/951095391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c15dbf0 msgr2=0x7f771c0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.286+0000 7f7720fb4700 1 --2- 172.21.15.6:0/951095391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c15dbf0 0x7f771c0ce8a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f771c0d0c50 tx=0x7f771400b330 comp rx=0 tx=0).stop 2023-11-28T13:07:44.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.286+0000 7f7720fb4700 1 -- 172.21.15.6:0/951095391 shutdown_connections 2023-11-28T13:07:44.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.286+0000 7f7720fb4700 1 --2- 172.21.15.6:0/951095391 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c15dbf0 0x7f771c0ce8a0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.286+0000 7f7720fb4700 1 --2- 172.21.15.6:0/951095391 >> v2:172.21.15.61:3300/0 conn(0x7f771c0b1d40 0x7f771c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.286+0000 7f7720fb4700 1 -- 172.21.15.6:0/951095391 >> 172.21.15.6:0/951095391 conn(0x7f771c0cc630 msgr2=0x7f771c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:44.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.286+0000 7f7720fb4700 1 -- 172.21.15.6:0/951095391 shutdown_connections 2023-11-28T13:07:44.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.286+0000 7f7720fb4700 1 -- 172.21.15.6:0/951095391 wait complete. 2023-11-28T13:07:44.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.287+0000 7f7720fb4700 1 Processor -- start 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.287+0000 7f7720fb4700 1 -- start start 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f7720fb4700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f771c0b1d40 0x7f771c0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f7720fb4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c159500 0x7f771c146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f7720fb4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f771c15dbf0 0x7f771c0c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f7720fb4700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c0c5500 con 0x7f771c159500 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f7720fb4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c0c57c0 con 0x7f771c15dbf0 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f7720fb4700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c0c5aa0 con 0x7f771c0b1d40 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f7719d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c159500 0x7f771c146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f7719d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c159500 0x7f771c146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35488/0 (socket says 172.21.15.6:35488) 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f7719d9b700 1 -- 172.21.15.6:0/3910855858 learned_addr learned my addr 172.21.15.6:0/3910855858 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f771ad9d700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f771c15dbf0 0x7f771c0c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.288+0000 7f771a59c700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f771c0b1d40 0x7f771c0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f7719d9b700 1 -- 172.21.15.6:0/3910855858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f771c0b1d40 msgr2=0x7f771c0c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f7719d9b700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f771c0b1d40 0x7f771c0c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f7719d9b700 1 -- 172.21.15.6:0/3910855858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f771c15dbf0 msgr2=0x7f771c0c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f7719d9b700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f771c15dbf0 0x7f771c0c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f7719d9b700 1 -- 172.21.15.6:0/3910855858 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7714009a70 con 0x7f771c159500 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f7719d9b700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c159500 0x7f771c146630 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f770c00d800 tx=0x7f770c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f770affd700 1 -- 172.21.15.6:0/3910855858 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0125f0 con 0x7f771c159500 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f770affd700 1 -- 172.21.15.6:0/3910855858 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c012750 con 0x7f771c159500 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f7720fb4700 1 -- 172.21.15.6:0/3910855858 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771c15d3f0 con 0x7f771c159500 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f770affd700 1 -- 172.21.15.6:0/3910855858 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c012b00 con 0x7f771c159500 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.289+0000 7f7720fb4700 1 -- 172.21.15.6:0/3910855858 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771c15d110 con 0x7f771c159500 2023-11-28T13:07:44.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.290+0000 7f7720fb4700 1 -- 172.21.15.6:0/3910855858 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771c15d6d0 con 0x7f771c159500 2023-11-28T13:07:44.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.290+0000 7f770affd700 1 -- 172.21.15.6:0/3910855858 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f770c012c60 con 0x7f771c159500 2023-11-28T13:07:44.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.291+0000 7f770affd700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76fc0682f0 0x7f76fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:44.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.291+0000 7f771a59c700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76fc0682f0 0x7f76fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:44.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.291+0000 7f770affd700 1 -- 172.21.15.6:0/3910855858 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f770c085f60 con 0x7f771c159500 2023-11-28T13:07:44.294 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.291+0000 7f771a59c700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76fc0682f0 0x7f76fc06a7a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7710007fd0 tx=0x7f7710007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:44.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.295+0000 7f770affd700 1 -- 172.21.15.6:0/3910855858 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f770c0525f0 con 0x7f771c159500 2023-11-28T13:07:44.537 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:44.536+0000 7f7720fb4700 1 -- 172.21.15.6:0/3910855858 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f771c0739f0 con 0x7f771c159500 2023-11-28T13:07:45.137 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.136+0000 7f770affd700 1 -- 172.21.15.6:0/3910855858 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7f770c055c10 con 0x7f771c159500 2023-11-28T13:07:45.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 -- 172.21.15.6:0/3910855858 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76fc0682f0 msgr2=0x7f76fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76fc0682f0 0x7f76fc06a7a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7710007fd0 tx=0x7f7710007de0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 -- 172.21.15.6:0/3910855858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c159500 msgr2=0x7f771c146630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c159500 0x7f771c146630 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f770c00d800 tx=0x7f770c00db10 comp rx=0 tx=0).stop 2023-11-28T13:07:45.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 -- 172.21.15.6:0/3910855858 shutdown_connections 2023-11-28T13:07:45.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f771c15dbf0 0x7f771c0c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f76fc0682f0 0x7f76fc06a7a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f771c0b1d40 0x7f771c0c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 --2- 172.21.15.6:0/3910855858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f771c159500 0x7f771c146630 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.141 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 -- 172.21.15.6:0/3910855858 >> 172.21.15.6:0/3910855858 conn(0x7f771c0cc630 msgr2=0x7f771c1538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:45.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 -- 172.21.15.6:0/3910855858 shutdown_connections 2023-11-28T13:07:45.142 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.139+0000 7f7720fb4700 1 -- 172.21.15.6:0/3910855858 wait complete. 2023-11-28T13:07:45.156 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T13:07:45.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.291+0000 7f487e76e700 1 Processor -- start 2023-11-28T13:07:45.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.292+0000 7f487e76e700 1 -- start start 2023-11-28T13:07:45.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.292+0000 7f487e76e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4878160ca0 0x7f48780ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.292+0000 7f487e76e700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f48780cae50 0x7f48780ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.292+0000 7f487e76e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f48780aec70 0x7f48780b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.292+0000 7f487e76e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48780cb2c0 con 0x7f48780aec70 2023-11-28T13:07:45.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.292+0000 7f487e76e700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4878164740 con 0x7f48780cae50 2023-11-28T13:07:45.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.292+0000 7f487e76e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4878164a20 con 0x7f4878160ca0 2023-11-28T13:07:45.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f487cd0b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f48780aec70 0x7f48780b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f487cd0b700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f48780aec70 0x7f48780b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35498/0 (socket says 172.21.15.6:35498) 2023-11-28T13:07:45.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f487cd0b700 1 -- 172.21.15.6:0/4089723099 learned_addr learned my addr 172.21.15.6:0/4089723099 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:45.296 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f4877fff700 1 --2- 172.21.15.6:0/4089723099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4878160ca0 0x7f48780ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f48777fe700 1 --2- 172.21.15.6:0/4089723099 >> v2:172.21.15.61:3300/0 conn(0x7f48780cae50 0x7f48780ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f487cd0b700 1 -- 172.21.15.6:0/4089723099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4878160ca0 msgr2=0x7f48780ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f487cd0b700 1 --2- 172.21.15.6:0/4089723099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4878160ca0 0x7f48780ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f487cd0b700 1 -- 172.21.15.6:0/4089723099 >> v2:172.21.15.61:3300/0 conn(0x7f48780cae50 msgr2=0x7f48780ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f487cd0b700 1 --2- 172.21.15.6:0/4089723099 >> v2:172.21.15.61:3300/0 conn(0x7f48780cae50 0x7f48780ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f487cd0b700 1 -- 172.21.15.6:0/4089723099 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4878164d00 con 0x7f48780aec70 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f487cd0b700 1 --2- 172.21.15.6:0/4089723099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780aec70 0x7f48780b1530 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4878071610 tx=0x7f487000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732f3b06f6dd636b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f4876ffd700 1 -- 172.21.15.6:0/4089723099 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000e070 con 0x7f48780aec70 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f4876ffd700 1 -- 172.21.15.6:0/4089723099 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487000b910 con 0x7f48780aec70 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.293+0000 7f4876ffd700 1 -- 172.21.15.6:0/4089723099 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000bcc0 con 0x7f48780aec70 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.294+0000 7f487e76e700 1 -- 172.21.15.6:0/4089723099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780aec70 msgr2=0x7f48780b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.297 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.294+0000 7f487e76e700 1 --2- 172.21.15.6:0/4089723099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780aec70 0x7f48780b1530 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4878071610 tx=0x7f487000b330 comp rx=0 tx=0).stop 2023-11-28T13:07:45.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.294+0000 7f487e76e700 1 -- 172.21.15.6:0/4089723099 shutdown_connections 2023-11-28T13:07:45.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.294+0000 7f487e76e700 1 --2- 172.21.15.6:0/4089723099 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780aec70 0x7f48780b1530 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.294+0000 7f487e76e700 1 --2- 172.21.15.6:0/4089723099 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4878160ca0 0x7f48780ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.294+0000 7f487e76e700 1 --2- 172.21.15.6:0/4089723099 >> v2:172.21.15.61:3300/0 conn(0x7f48780cae50 0x7f48780ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.294+0000 7f487e76e700 1 -- 172.21.15.6:0/4089723099 >> 172.21.15.6:0/4089723099 conn(0x7f487806dae0 msgr2=0x7f487806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:45.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.294+0000 7f487e76e700 1 -- 172.21.15.6:0/4089723099 shutdown_connections 2023-11-28T13:07:45.298 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.294+0000 7f487e76e700 1 -- 172.21.15.6:0/4089723099 wait complete. 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.295+0000 7f487e76e700 1 Processor -- start 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.295+0000 7f487e76e700 1 -- start start 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f487e76e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48780aec70 0x7f48780c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f487e76e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780cae50 0x7f487815ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f487e76e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4878160ca0 0x7f487815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f487e76e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48780c9480 con 0x7f48780cae50 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f487e76e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48780c35e0 con 0x7f4878160ca0 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f487e76e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48781fab20 con 0x7f48780aec70 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f48777fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780cae50 0x7f487815ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f48777fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780cae50 0x7f487815ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35502/0 (socket says 172.21.15.6:35502) 2023-11-28T13:07:45.301 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f48777fe700 1 -- 172.21.15.6:0/2904066955 learned_addr learned my addr 172.21.15.6:0/2904066955 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f487cd0b700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4878160ca0 0x7f487815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f4877fff700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48780aec70 0x7f48780c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f48777fe700 1 -- 172.21.15.6:0/2904066955 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48780aec70 msgr2=0x7f48780c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f48777fe700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48780aec70 0x7f48780c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f48777fe700 1 -- 172.21.15.6:0/2904066955 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4878160ca0 msgr2=0x7f487815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.296+0000 7f48777fe700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4878160ca0 0x7f487815ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.297+0000 7f48777fe700 1 -- 172.21.15.6:0/2904066955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870009b30 con 0x7f48780cae50 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.297+0000 7f48777fe700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780cae50 0x7f487815ae40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f486c00e9b0 tx=0x7f486c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.297+0000 7f4874ff9700 1 -- 172.21.15.6:0/2904066955 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c009710 con 0x7f48780cae50 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.297+0000 7f4874ff9700 1 -- 172.21.15.6:0/2904066955 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c00ee50 con 0x7f48780cae50 2023-11-28T13:07:45.302 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.297+0000 7f487e76e700 1 -- 172.21.15.6:0/2904066955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48780af080 con 0x7f48780cae50 2023-11-28T13:07:45.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.297+0000 7f4874ff9700 1 -- 172.21.15.6:0/2904066955 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c00ca50 con 0x7f48780cae50 2023-11-28T13:07:45.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.297+0000 7f487e76e700 1 -- 172.21.15.6:0/2904066955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4878164fe0 con 0x7f48780cae50 2023-11-28T13:07:45.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.298+0000 7f487e76e700 1 -- 172.21.15.6:0/2904066955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4878066c60 con 0x7f48780cae50 2023-11-28T13:07:45.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.298+0000 7f4874ff9700 1 -- 172.21.15.6:0/2904066955 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f486c019030 con 0x7f48780cae50 2023-11-28T13:07:45.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.299+0000 7f4874ff9700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48640682f0 0x7f486406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:45.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.299+0000 7f4874ff9700 1 -- 172.21.15.6:0/2904066955 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f486c085ce0 con 0x7f48780cae50 2023-11-28T13:07:45.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.299+0000 7f4877fff700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48640682f0 0x7f486406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:45.303 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.299+0000 7f4877fff700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48640682f0 0x7f486406a7a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4868009b60 tx=0x7f4868009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:45.304 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.303+0000 7f4874ff9700 1 -- 172.21.15.6:0/2904066955 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f486c052370 con 0x7f48780cae50 2023-11-28T13:07:45.497 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:45.495+0000 7f487e76e700 1 -- 172.21.15.6:0/2904066955 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48780739f0 con 0x7f48780cae50 2023-11-28T13:07:46.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.201+0000 7f4874ff9700 1 -- 172.21.15.6:0/2904066955 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v70) v1 ==== 181+0+0 (secure 0 0 0) 0x7f486c055990 con 0x7f48780cae50 2023-11-28T13:07:46.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.206+0000 7f487e76e700 1 -- 172.21.15.6:0/2904066955 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48640682f0 msgr2=0x7f486406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.206+0000 7f487e76e700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48640682f0 0x7f486406a7a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4868009b60 tx=0x7f4868009210 comp rx=0 tx=0).stop 2023-11-28T13:07:46.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.206+0000 7f487e76e700 1 -- 172.21.15.6:0/2904066955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780cae50 msgr2=0x7f487815ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.206+0000 7f487e76e700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780cae50 0x7f487815ae40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f486c00e9b0 tx=0x7f486c00ecc0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.206+0000 7f487e76e700 1 -- 172.21.15.6:0/2904066955 shutdown_connections 2023-11-28T13:07:46.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.206+0000 7f487e76e700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4878160ca0 0x7f487815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.206+0000 7f487e76e700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f48640682f0 0x7f486406a7a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.206+0000 7f487e76e700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f48780aec70 0x7f48780c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.206+0000 7f487e76e700 1 --2- 172.21.15.6:0/2904066955 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f48780cae50 0x7f487815ae40 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.206+0000 7f487e76e700 1 -- 172.21.15.6:0/2904066955 >> 172.21.15.6:0/2904066955 conn(0x7f487806dae0 msgr2=0x7f487806deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:46.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.207+0000 7f487e76e700 1 -- 172.21.15.6:0/2904066955 shutdown_connections 2023-11-28T13:07:46.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.207+0000 7f487e76e700 1 -- 172.21.15.6:0/2904066955 wait complete. 2023-11-28T13:07:46.209 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_metadata' removed 2023-11-28T13:07:46.224 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T13:07:46.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.359+0000 7f40d4f58700 1 Processor -- start 2023-11-28T13:07:46.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.359+0000 7f40d4f58700 1 -- start start 2023-11-28T13:07:46.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.360+0000 7f40d4f58700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40d0161090 0x7f40d00ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.360+0000 7f40d4f58700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f40d00caeb0 0x7f40d00cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.360+0000 7f40d4f58700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d00ce340 con 0x7f40d015c990 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.360+0000 7f40d4f58700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d00cebe0 con 0x7f40d00caeb0 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.360+0000 7f40d4f58700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00cf780 con 0x7f40d0161090 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40ced9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f40d00caeb0 0x7f40d00cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40ce59c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f40d015c990 0x7f40d00cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51060/0 (socket says 172.21.15.6:51060) 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40ce59c700 1 -- 172.21.15.6:0/245296858 learned_addr learned my addr 172.21.15.6:0/245296858 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40ced9d700 1 -- 172.21.15.6:0/245296858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40d0161090 msgr2=0x7f40d00ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40ced9d700 1 --2- 172.21.15.6:0/245296858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40d0161090 0x7f40d00ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40ced9d700 1 -- 172.21.15.6:0/245296858 >> v1:172.21.15.6:6789/0 conn(0x7f40d015c990 legacy=0x7f40d00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40ced9d700 1 -- 172.21.15.6:0/245296858 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00cb3f0 con 0x7f40d00caeb0 2023-11-28T13:07:46.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40ced9d700 1 --2- 172.21.15.6:0/245296858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d00caeb0 0x7f40d00cc310 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009d90 tx=0x7f40c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbd79630de3616c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40cd59a700 1 -- 172.21.15.6:0/245296858 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c800e070 con 0x7f40d00caeb0 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40cd59a700 1 -- 172.21.15.6:0/245296858 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c800bad0 con 0x7f40d00caeb0 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.361+0000 7f40cd59a700 1 -- 172.21.15.6:0/245296858 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c800be80 con 0x7f40d00caeb0 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.362+0000 7f40d4f58700 1 -- 172.21.15.6:0/245296858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d00caeb0 msgr2=0x7f40d00cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.362+0000 7f40d4f58700 1 --2- 172.21.15.6:0/245296858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d00caeb0 0x7f40d00cc310 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009d90 tx=0x7f40c800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.362+0000 7f40d4f58700 1 -- 172.21.15.6:0/245296858 shutdown_connections 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.362+0000 7f40d4f58700 1 --2- 172.21.15.6:0/245296858 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d00caeb0 0x7f40d00cc310 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.362+0000 7f40d4f58700 1 --2- 172.21.15.6:0/245296858 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40d0161090 0x7f40d00ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.362+0000 7f40d4f58700 1 -- 172.21.15.6:0/245296858 >> 172.21.15.6:0/245296858 conn(0x7f40d00b1650 msgr2=0x7f40d00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.362+0000 7f40d4f58700 1 -- 172.21.15.6:0/245296858 shutdown_connections 2023-11-28T13:07:46.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.362+0000 7f40d4f58700 1 -- 172.21.15.6:0/245296858 wait complete. 2023-11-28T13:07:46.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.363+0000 7f40d4f58700 1 Processor -- start 2023-11-28T13:07:46.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.363+0000 7f40d4f58700 1 -- start start 2023-11-28T13:07:46.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40d4f58700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40d00caeb0 0x7f40d00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40d4f58700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40d015c990 0x7f40d0153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40d4f58700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d0161090 0x7f40d00c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40d4f58700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00c9a60 con 0x7f40d0161090 2023-11-28T13:07:46.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40d4f58700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00c8ee0 con 0x7f40d015c990 2023-11-28T13:07:46.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40d4f58700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00c91c0 con 0x7f40d00caeb0 2023-11-28T13:07:46.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40ced9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d0161090 0x7f40d00c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40ced9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d0161090 0x7f40d00c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35518/0 (socket says 172.21.15.6:35518) 2023-11-28T13:07:46.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40ced9d700 1 -- 172.21.15.6:0/1335888195 learned_addr learned my addr 172.21.15.6:0/1335888195 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:46.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40ce59c700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40d00caeb0 0x7f40d00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40cdd9b700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40d015c990 0x7f40d0153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40ced9d700 1 -- 172.21.15.6:0/1335888195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40d00caeb0 msgr2=0x7f40d00c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40ced9d700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40d00caeb0 0x7f40d00c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40ced9d700 1 -- 172.21.15.6:0/1335888195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40d015c990 msgr2=0x7f40d0153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:46.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40ced9d700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40d015c990 0x7f40d0153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:46.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40ced9d700 1 -- 172.21.15.6:0/1335888195 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8009a70 con 0x7f40d0161090 2023-11-28T13:07:46.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40cdd9b700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40d015c990 0x7f40d0153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:46.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.364+0000 7f40ce59c700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40d00caeb0 0x7f40d00c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.365+0000 7f40ced9d700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d0161090 0x7f40d00c5360 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f40d006eac0 tx=0x7f40c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.365+0000 7f40c6ffd700 1 -- 172.21.15.6:0/1335888195 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8012630 con 0x7f40d0161090 2023-11-28T13:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.365+0000 7f40c6ffd700 1 -- 172.21.15.6:0/1335888195 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c80127b0 con 0x7f40d0161090 2023-11-28T13:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.365+0000 7f40d4f58700 1 -- 172.21.15.6:0/1335888195 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d00ce870 con 0x7f40d0161090 2023-11-28T13:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.365+0000 7f40c6ffd700 1 -- 172.21.15.6:0/1335888195 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8012b60 con 0x7f40d0161090 2023-11-28T13:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.365+0000 7f40d4f58700 1 -- 172.21.15.6:0/1335888195 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d00ce590 con 0x7f40d0161090 2023-11-28T13:07:46.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.366+0000 7f40d4f58700 1 -- 172.21.15.6:0/1335888195 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d00cc850 con 0x7f40d0161090 2023-11-28T13:07:46.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.366+0000 7f40c6ffd700 1 -- 172.21.15.6:0/1335888195 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f40c8012cc0 con 0x7f40d0161090 2023-11-28T13:07:46.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.367+0000 7f40c6ffd700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40b00682f0 0x7f40b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:46.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.367+0000 7f40c6ffd700 1 -- 172.21.15.6:0/1335888195 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6088+0+0 (secure 0 0 0) 0x7f40c8088180 con 0x7f40d0161090 2023-11-28T13:07:46.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.367+0000 7f40ce59c700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40b00682f0 0x7f40b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:46.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.367+0000 7f40ce59c700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40b00682f0 0x7f40b006a7a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f40c0000f60 tx=0x7f40c0007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:46.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.371+0000 7f40c6ffd700 1 -- 172.21.15.6:0/1335888195 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f40c80549c0 con 0x7f40d0161090 2023-11-28T13:07:46.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:46.574+0000 7f40d4f58700 1 -- 172.21.15.6:0/1335888195 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f40d0066700 con 0x7f40d0161090 2023-11-28T13:07:47.207 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.206+0000 7f40c6ffd700 1 -- 172.21.15.6:0/1335888195 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v71) v1 ==== 169+0+0 (secure 0 0 0) 0x7f40c8057fe0 con 0x7f40d0161090 2023-11-28T13:07:47.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 -- 172.21.15.6:0/1335888195 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40b00682f0 msgr2=0x7f40b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40b00682f0 0x7f40b006a7a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f40c0000f60 tx=0x7f40c0007e30 comp rx=0 tx=0).stop 2023-11-28T13:07:47.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 -- 172.21.15.6:0/1335888195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d0161090 msgr2=0x7f40d00c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d0161090 0x7f40d00c5360 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f40d006eac0 tx=0x7f40c8004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 -- 172.21.15.6:0/1335888195 shutdown_connections 2023-11-28T13:07:47.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40d0161090 0x7f40d00c5360 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40b00682f0 0x7f40b006a7a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f40d00caeb0 0x7f40d00c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 --2- 172.21.15.6:0/1335888195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40d015c990 0x7f40d0153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 -- 172.21.15.6:0/1335888195 >> 172.21.15.6:0/1335888195 conn(0x7f40d00b1650 msgr2=0x7f40d00cf070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:47.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 -- 172.21.15.6:0/1335888195 shutdown_connections 2023-11-28T13:07:47.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.210+0000 7f40d4f58700 1 -- 172.21.15.6:0/1335888195 wait complete. 2023-11-28T13:07:47.212 INFO:teuthology.orchestra.run.smithi006.stderr:pool 'cephfs_data' removed 2023-11-28T13:07:47.227 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T13:07:47.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.360+0000 7f97f82ab700 1 Processor -- start 2023-11-28T13:07:47.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.360+0000 7f97f82ab700 1 -- start start 2023-11-28T13:07:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.361+0000 7f97f82ab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97f00b1630 0x7f97f00cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.361+0000 7f97f82ab700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f97f006afc0 0x7f97f006b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.361+0000 7f97f82ab700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00ce550 con 0x7f97f00b1a00 2023-11-28T13:07:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.361+0000 7f97f82ab700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f00ce830 con 0x7f97f006afc0 2023-11-28T13:07:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.361+0000 7f97f82ab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00ceb10 con 0x7f97f00b1630 2023-11-28T13:07:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.361+0000 7f97f6848700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f97f006afc0 0x7f97f006b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.362+0000 7f97f5846700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f97f00b1a00 0x7f97f00caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51074/0 (socket says 172.21.15.6:51074) 2023-11-28T13:07:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.362+0000 7f97f5846700 1 -- 172.21.15.6:0/2902362962 learned_addr learned my addr 172.21.15.6:0/2902362962 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:47.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.362+0000 7f97f6848700 1 -- 172.21.15.6:0/2902362962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97f00b1630 msgr2=0x7f97f00cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.362+0000 7f97f6848700 1 --2- 172.21.15.6:0/2902362962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97f00b1630 0x7f97f00cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.362+0000 7f97f6848700 1 -- 172.21.15.6:0/2902362962 >> v1:172.21.15.6:6789/0 conn(0x7f97f00b1a00 legacy=0x7f97f00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.362+0000 7f97f6848700 1 -- 172.21.15.6:0/2902362962 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f00cedf0 con 0x7f97f006afc0 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.362+0000 7f97f6848700 1 --2- 172.21.15.6:0/2902362962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 0x7f97f006b440 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009d90 tx=0x7f97e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48caffd26b97fb9d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.362+0000 7f97f5045700 1 -- 172.21.15.6:0/2902362962 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e800e070 con 0x7f97f006afc0 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.362+0000 7f97f5045700 1 -- 172.21.15.6:0/2902362962 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e800bbd0 con 0x7f97f006afc0 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.362+0000 7f97f5045700 1 -- 172.21.15.6:0/2902362962 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e8012430 con 0x7f97f006afc0 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.363+0000 7f97f82ab700 1 -- 172.21.15.6:0/2902362962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 msgr2=0x7f97f006b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.363+0000 7f97f82ab700 1 --2- 172.21.15.6:0/2902362962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 0x7f97f006b440 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009d90 tx=0x7f97e800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.363+0000 7f97f82ab700 1 -- 172.21.15.6:0/2902362962 shutdown_connections 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.363+0000 7f97f82ab700 1 --2- 172.21.15.6:0/2902362962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 0x7f97f006b440 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.363+0000 7f97f82ab700 1 --2- 172.21.15.6:0/2902362962 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97f00b1630 0x7f97f00cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.363+0000 7f97f82ab700 1 -- 172.21.15.6:0/2902362962 >> 172.21.15.6:0/2902362962 conn(0x7f97f00cc630 msgr2=0x7f97f00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:47.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.363+0000 7f97f82ab700 1 -- 172.21.15.6:0/2902362962 shutdown_connections 2023-11-28T13:07:47.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.363+0000 7f97f82ab700 1 -- 172.21.15.6:0/2902362962 wait complete. 2023-11-28T13:07:47.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.364+0000 7f97f82ab700 1 Processor -- start 2023-11-28T13:07:47.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.364+0000 7f97f82ab700 1 -- start start 2023-11-28T13:07:47.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f82ab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 0x7f97f015bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f82ab700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97f00b1630 0x7f97f00c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f82ab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97f00b1a00 0x7f97f00c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.368 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f82ab700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00c5100 con 0x7f97f006afc0 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f82ab700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00c44f0 con 0x7f97f00b1630 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f82ab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00c47d0 con 0x7f97f00b1a00 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f6047700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 0x7f97f015bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f6047700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 0x7f97f015bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35528/0 (socket says 172.21.15.6:35528) 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f6047700 1 -- 172.21.15.6:0/14000934 learned_addr learned my addr 172.21.15.6:0/14000934 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f6848700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97f00b1a00 0x7f97f00c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f5846700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97f00b1630 0x7f97f00c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f6047700 1 -- 172.21.15.6:0/14000934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97f00b1a00 msgr2=0x7f97f00c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f6047700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97f00b1a00 0x7f97f00c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f6047700 1 -- 172.21.15.6:0/14000934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97f00b1630 msgr2=0x7f97f00c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f6047700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97f00b1630 0x7f97f00c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f6047700 1 -- 172.21.15.6:0/14000934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8009a70 con 0x7f97f006afc0 2023-11-28T13:07:47.369 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.365+0000 7f97f6047700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 0x7f97f015bba0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f97e000b530 tx=0x7f97e000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:47.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.366+0000 7f97e6ffd700 1 -- 172.21.15.6:0/14000934 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0011830 con 0x7f97f006afc0 2023-11-28T13:07:47.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.366+0000 7f97e6ffd700 1 -- 172.21.15.6:0/14000934 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e0011990 con 0x7f97f006afc0 2023-11-28T13:07:47.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.366+0000 7f97f82ab700 1 -- 172.21.15.6:0/14000934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f00b1e70 con 0x7f97f006afc0 2023-11-28T13:07:47.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.366+0000 7f97e6ffd700 1 -- 172.21.15.6:0/14000934 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0011d40 con 0x7f97f006afc0 2023-11-28T13:07:47.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.366+0000 7f97f82ab700 1 -- 172.21.15.6:0/14000934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f00cf0d0 con 0x7f97f006afc0 2023-11-28T13:07:47.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.367+0000 7f97f82ab700 1 -- 172.21.15.6:0/14000934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f00cb5b0 con 0x7f97f006afc0 2023-11-28T13:07:47.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.367+0000 7f97e6ffd700 1 -- 172.21.15.6:0/14000934 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97e0021070 con 0x7f97f006afc0 2023-11-28T13:07:47.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.367+0000 7f97e6ffd700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97d40682f0 0x7f97d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.367+0000 7f97e6ffd700 1 -- 172.21.15.6:0/14000934 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f97e0085970 con 0x7f97f006afc0 2023-11-28T13:07:47.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.368+0000 7f97f5846700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97d40682f0 0x7f97d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.368+0000 7f97f5846700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97d40682f0 0x7f97d406a7a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f97f00c3570 tx=0x7f97ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:47.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.372+0000 7f97e6ffd700 1 -- 172.21.15.6:0/14000934 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f97e00523f0 con 0x7f97f006afc0 2023-11-28T13:07:47.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.565+0000 7f97f82ab700 1 -- 172.21.15.6:0/14000934 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f97f00739f0 con 0x7f97f006afc0 2023-11-28T13:07:47.570 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.569+0000 7f97e6ffd700 1 -- 172.21.15.6:0/14000934 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f97e00150a0 con 0x7f97f006afc0 2023-11-28T13:07:47.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 -- 172.21.15.6:0/14000934 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97d40682f0 msgr2=0x7f97d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97d40682f0 0x7f97d406a7a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f97f00c3570 tx=0x7f97ec009040 comp rx=0 tx=0).stop 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 -- 172.21.15.6:0/14000934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 msgr2=0x7f97f015bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 0x7f97f015bba0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f97e000b530 tx=0x7f97e000b840 comp rx=0 tx=0).stop 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 -- 172.21.15.6:0/14000934 shutdown_connections 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f97d40682f0 0x7f97d406a7a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f97f00b1a00 0x7f97f00c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f97f006afc0 0x7f97f015bba0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 --2- 172.21.15.6:0/14000934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97f00b1630 0x7f97f00c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 -- 172.21.15.6:0/14000934 >> 172.21.15.6:0/14000934 conn(0x7f97f00cc630 msgr2=0x7f97f00caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 -- 172.21.15.6:0/14000934 shutdown_connections 2023-11-28T13:07:47.573 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.571+0000 7f97f82ab700 1 -- 172.21.15.6:0/14000934 wait complete. 2023-11-28T13:07:47.584 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T13:07:47.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.711+0000 7fc2e8336700 1 Processor -- start 2023-11-28T13:07:47.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.711+0000 7fc2e8336700 1 -- start start 2023-11-28T13:07:47.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e8336700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc2e015a650 0x7fc2e00b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e8336700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2e015aa20 0x7fc2e00ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e8336700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e00ca990 con 0x7fc2e00b5500 2023-11-28T13:07:47.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e8336700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e00caf50 con 0x7fc2e015a650 2023-11-28T13:07:47.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e8336700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00aec70 con 0x7fc2e015aa20 2023-11-28T13:07:47.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e60d2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc2e015a650 0x7fc2e00b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e60d2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc2e015a650 0x7fc2e00b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35536/0 (socket says 172.21.15.6:35536) 2023-11-28T13:07:47.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e60d2700 1 -- 172.21.15.6:0/2901534859 learned_addr learned my addr 172.21.15.6:0/2901534859 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e58d1700 1 --2- 172.21.15.6:0/2901534859 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2e015aa20 0x7fc2e00ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e60d2700 1 -- 172.21.15.6:0/2901534859 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2e015aa20 msgr2=0x7fc2e00ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e60d2700 1 --2- 172.21.15.6:0/2901534859 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2e015aa20 0x7fc2e00ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e60d2700 1 -- 172.21.15.6:0/2901534859 >> v1:172.21.15.61:6789/0 conn(0x7fc2e00b5500 legacy=0x7fc2e00cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.712+0000 7fc2e60d2700 1 -- 172.21.15.6:0/2901534859 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00aef10 con 0x7fc2e015a650 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e60d2700 1 --2- 172.21.15.6:0/2901534859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015a650 0x7fc2e00b2950 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00098f0 tx=0x7fc2d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90de317793c6c7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e50d0700 1 -- 172.21.15.6:0/2901534859 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000b8a0 con 0x7fc2e015a650 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e50d0700 1 -- 172.21.15.6:0/2901534859 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d000ba00 con 0x7fc2e015a650 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e50d0700 1 -- 172.21.15.6:0/2901534859 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000bd90 con 0x7fc2e015a650 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e8336700 1 -- 172.21.15.6:0/2901534859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015a650 msgr2=0x7fc2e00b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e8336700 1 --2- 172.21.15.6:0/2901534859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015a650 0x7fc2e00b2950 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00098f0 tx=0x7fc2d0009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e8336700 1 -- 172.21.15.6:0/2901534859 shutdown_connections 2023-11-28T13:07:47.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e8336700 1 --2- 172.21.15.6:0/2901534859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015a650 0x7fc2e00b2950 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e8336700 1 --2- 172.21.15.6:0/2901534859 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2e015aa20 0x7fc2e00ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e8336700 1 -- 172.21.15.6:0/2901534859 >> 172.21.15.6:0/2901534859 conn(0x7fc2e00b1650 msgr2=0x7fc2e00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:47.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e8336700 1 -- 172.21.15.6:0/2901534859 shutdown_connections 2023-11-28T13:07:47.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.713+0000 7fc2e8336700 1 -- 172.21.15.6:0/2901534859 wait complete. 2023-11-28T13:07:47.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.715+0000 7fc2e8336700 1 Processor -- start 2023-11-28T13:07:47.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.715+0000 7fc2e8336700 1 -- start start 2023-11-28T13:07:47.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.715+0000 7fc2e8336700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e00b5500 0x7fc2e015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.715+0000 7fc2e8336700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2e015a650 0x7fc2e006b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.715+0000 7fc2e8336700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015aa20 0x7fc2e014f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e8336700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015d5d0 con 0x7fc2e015aa20 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e8336700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015b280 con 0x7fc2e00b5500 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e8336700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015b560 con 0x7fc2e015a650 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e68d3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015aa20 0x7fc2e014f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e60d2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e00b5500 0x7fc2e015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e68d3700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015aa20 0x7fc2e014f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35546/0 (socket says 172.21.15.6:35546) 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e60d2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e00b5500 0x7fc2e015d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.6:39260/0 (socket says 172.21.15.6:39260) 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e68d3700 1 -- 172.21.15.6:0/882950062 learned_addr learned my addr 172.21.15.6:0/882950062 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e58d1700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2e015a650 0x7fc2e006b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e68d3700 1 -- 172.21.15.6:0/882950062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2e015a650 msgr2=0x7fc2e006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e68d3700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2e015a650 0x7fc2e006b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e68d3700 1 -- 172.21.15.6:0/882950062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e00b5500 msgr2=0x7fc2e015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e68d3700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e00b5500 0x7fc2e015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e68d3700 1 -- 172.21.15.6:0/882950062 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d0009550 con 0x7fc2e015aa20 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e68d3700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015aa20 0x7fc2e014f3a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800cb50 tx=0x7fc2d800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2d6ffd700 1 -- 172.21.15.6:0/882950062 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8003c00 con 0x7fc2e015aa20 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2d6ffd700 1 -- 172.21.15.6:0/882950062 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d8003d60 con 0x7fc2e015aa20 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e8336700 1 -- 172.21.15.6:0/882950062 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e00cabe0 con 0x7fc2e015aa20 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2d6ffd700 1 -- 172.21.15.6:0/882950062 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d800aa20 con 0x7fc2e015aa20 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.716+0000 7fc2e8336700 1 -- 172.21.15.6:0/882950062 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e00af1f0 con 0x7fc2e015aa20 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.717+0000 7fc2e8336700 1 -- 172.21.15.6:0/882950062 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e00ccbc0 con 0x7fc2e015aa20 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.718+0000 7fc2d6ffd700 1 -- 172.21.15.6:0/882950062 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc2d8010050 con 0x7fc2e015aa20 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.718+0000 7fc2d6ffd700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c4068320 0x7fc2c406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:47.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.718+0000 7fc2d6ffd700 1 -- 172.21.15.6:0/882950062 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fc2d8085960 con 0x7fc2e015aa20 2023-11-28T13:07:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.718+0000 7fc2e60d2700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c4068320 0x7fc2c406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:47.722 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.719+0000 7fc2e60d2700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c4068320 0x7fc2c406a7d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009c40 tx=0x7fc2d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:47.723 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.722+0000 7fc2d6ffd700 1 -- 172.21.15.6:0/882950062 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc2d8052380 con 0x7fc2e015aa20 2023-11-28T13:07:48.000 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:47.999+0000 7fc2e8336700 1 -- 172.21.15.6:0/882950062 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fc2e00ca5c0 con 0x7fc2e015aa20 2023-11-28T13:07:48.206 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.205+0000 7fc2d6ffd700 1 -- 172.21.15.6:0/882950062 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v338) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc2d80559a0 con 0x7fc2e015aa20 2023-11-28T13:07:48.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 -- 172.21.15.6:0/882950062 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c4068320 msgr2=0x7fc2c406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:48.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c4068320 0x7fc2c406a7d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009c40 tx=0x7fc2d0008030 comp rx=0 tx=0).stop 2023-11-28T13:07:48.208 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 -- 172.21.15.6:0/882950062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015aa20 msgr2=0x7fc2e014f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:48.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015aa20 0x7fc2e014f3a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800cb50 tx=0x7fc2d800ce60 comp rx=0 tx=0).stop 2023-11-28T13:07:48.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 -- 172.21.15.6:0/882950062 shutdown_connections 2023-11-28T13:07:48.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc2e015aa20 0x7fc2e014f3a0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:48.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e00b5500 0x7fc2e015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:48.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fc2c4068320 0x7fc2c406a7d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:48.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 --2- 172.21.15.6:0/882950062 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fc2e015a650 0x7fc2e006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:48.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 -- 172.21.15.6:0/882950062 >> 172.21.15.6:0/882950062 conn(0x7fc2e00b1650 msgr2=0x7fc2e00cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:48.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 -- 172.21.15.6:0/882950062 shutdown_connections 2023-11-28T13:07:48.209 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:48.207+0000 7fc2e8336700 1 -- 172.21.15.6:0/882950062 wait complete. 2023-11-28T13:07:48.222 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ERROR 2023-11-28T13:07:48.222 INFO:tasks.cephfs_test_runner: 2023-11-28T13:07:48.222 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T13:07:48.222 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:07:48.222 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T13:07:48.222 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T13:07:48.222 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T13:07:48.222 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=200) 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 200s and 0 retries 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner: 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner:Ran 3 tests in 403.875s 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner: 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner: 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:07:48.223 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T13:07:48.224 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T13:07:48.224 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T13:07:48.224 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T13:07:48.224 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=200) 2023-11-28T13:07:48.224 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T13:07:48.224 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T13:07:48.224 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 200s and 0 retries 2023-11-28T13:07:48.224 INFO:tasks.cephfs_test_runner: 2023-11-28T13:07:48.224 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:07:48.303 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=780162d5c8414688a4c601747ab2870f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_3ae99413970476692dfab1fe38dadb056700a61d/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T13:07:48.305 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T13:07:48.317 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-28T13:07:48.350 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-28T13:07:48.351 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:07:48.351 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:48.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3446 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:07:48.374 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-28T13:07:48.374 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:48.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=37,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3446 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T13:07:48.434 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:07:48.434 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:48.457 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:07:48.457 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-11-28T13:07:48.457 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T13:07:48.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284288k,nr_inodes=4071072,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=40,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18958 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260556k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T13:07:48.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:48.521 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list 2023-11-28T13:07:48.581 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T13:07:48.581 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T13:07:48.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:48.582 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T13:07:48.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:48.661 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T13:07:48.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:48.746 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link delete ceph-brx 2023-11-28T13:07:48.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:48.840 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-28T13:07:48.862 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T13:07:48.862 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T13:07:48.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:48.863 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T13:07:48.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:48.962 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T13:07:49.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T13:07:49.046 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2023-11-28T13:07:49.145 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T13:07:49.154 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T13:07:49.154 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:07:49.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.280+0000 7f408bce9700 1 Processor -- start 2023-11-28T13:07:49.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.280+0000 7f408bce9700 1 -- start start 2023-11-28T13:07:49.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.281+0000 7f408bce9700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f408415c640 0x7f40840cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.281+0000 7f408bce9700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40840b6350 con 0x7f408406bf30 2023-11-28T13:07:49.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.281+0000 7f408bce9700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40840d1470 con 0x7f40840d2820 2023-11-28T13:07:49.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.281+0000 7f408bce9700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408415ca10 con 0x7f408415c640 2023-11-28T13:07:49.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.281+0000 7f4089a85700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f408415c640 0x7f40840cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.281+0000 7f408a286700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f408406bf30 0x7f408406c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51102/0 (socket says 172.21.15.6:51102) 2023-11-28T13:07:49.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.281+0000 7f408a286700 1 -- 172.21.15.6:0/3479027095 learned_addr learned my addr 172.21.15.6:0/3479027095 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:49.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.282+0000 7f4089a85700 1 -- 172.21.15.6:0/3479027095 >> v1:172.21.15.61:6789/0 conn(0x7f40840d2820 legacy=0x7f40840b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:49.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.282+0000 7f4089a85700 1 -- 172.21.15.6:0/3479027095 >> v1:172.21.15.6:6789/0 conn(0x7f408406bf30 legacy=0x7f408406c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.282+0000 7f4089a85700 1 -- 172.21.15.6:0/3479027095 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840cd4e0 con 0x7f408415c640 2023-11-28T13:07:49.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.282+0000 7f4088a83700 1 -- 172.21.15.6:0/3479027095 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601356326 0 0) 0x7f40840b6350 con 0x7f408406bf30 2023-11-28T13:07:49.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.282+0000 7f4089a85700 1 --2- 172.21.15.6:0/3479027095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f408415c640 0x7f40840cf040 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4074009840 tx=0x7f4074009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989363f786c784a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:49.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.282+0000 7f4088a83700 1 -- 172.21.15.6:0/3479027095 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074004230 con 0x7f408415c640 2023-11-28T13:07:49.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.282+0000 7f4088a83700 1 -- 172.21.15.6:0/3479027095 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4074004390 con 0x7f408415c640 2023-11-28T13:07:49.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.282+0000 7f4088a83700 1 -- 172.21.15.6:0/3479027095 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40740036a0 con 0x7f408415c640 2023-11-28T13:07:49.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.283+0000 7f408bce9700 1 -- 172.21.15.6:0/3479027095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f408415c640 msgr2=0x7f40840cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.283+0000 7f408bce9700 1 --2- 172.21.15.6:0/3479027095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f408415c640 0x7f40840cf040 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4074009840 tx=0x7f4074009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:49.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.283+0000 7f408bce9700 1 -- 172.21.15.6:0/3479027095 shutdown_connections 2023-11-28T13:07:49.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.283+0000 7f408bce9700 1 --2- 172.21.15.6:0/3479027095 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f408415c640 0x7f40840cf040 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.286 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.283+0000 7f408bce9700 1 -- 172.21.15.6:0/3479027095 >> 172.21.15.6:0/3479027095 conn(0x7f40840d2df0 msgr2=0x7f40840d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:49.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.283+0000 7f408bce9700 1 -- 172.21.15.6:0/3479027095 shutdown_connections 2023-11-28T13:07:49.287 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.283+0000 7f408bce9700 1 -- 172.21.15.6:0/3479027095 wait complete. 2023-11-28T13:07:49.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.284+0000 7f408bce9700 1 Processor -- start 2023-11-28T13:07:49.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.284+0000 7f408bce9700 1 -- start start 2023-11-28T13:07:49.289 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f408bce9700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f408406bf30 0x7f408415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f408bce9700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d2820 0x7f40841576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f408bce9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f408415c640 0x7f40841559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f408bce9700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40841417b0 con 0x7f40840d2820 2023-11-28T13:07:49.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f408bce9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4084141a90 con 0x7f408415c640 2023-11-28T13:07:49.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f408bce9700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4084141d70 con 0x7f408406bf30 2023-11-28T13:07:49.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f4089284700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d2820 0x7f40841576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.290 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f4089284700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d2820 0x7f40841576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35556/0 (socket says 172.21.15.6:35556) 2023-11-28T13:07:49.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f4089284700 1 -- 172.21.15.6:0/3288903063 learned_addr learned my addr 172.21.15.6:0/3288903063 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:49.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f4089a85700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f408406bf30 0x7f408415f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f408a286700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f408415c640 0x7f40841559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f4089284700 1 -- 172.21.15.6:0/3288903063 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f408406bf30 msgr2=0x7f408415f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f4089284700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f408406bf30 0x7f408415f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f4089284700 1 -- 172.21.15.6:0/3288903063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f408415c640 msgr2=0x7f40841559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f4089284700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f408415c640 0x7f40841559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.291 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f4089284700 1 -- 172.21.15.6:0/3288903063 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40740094a0 con 0x7f40840d2820 2023-11-28T13:07:49.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.285+0000 7f4089284700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d2820 0x7f40841576f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f408000c4d0 tx=0x7f408000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:49.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.286+0000 7f407a7fc700 1 -- 172.21.15.6:0/3288903063 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080011790 con 0x7f40840d2820 2023-11-28T13:07:49.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.286+0000 7f407a7fc700 1 -- 172.21.15.6:0/3288903063 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40800118f0 con 0x7f40840d2820 2023-11-28T13:07:49.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.286+0000 7f408bce9700 1 -- 172.21.15.6:0/3288903063 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40840d1710 con 0x7f40840d2820 2023-11-28T13:07:49.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.286+0000 7f407a7fc700 1 -- 172.21.15.6:0/3288903063 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4080011ca0 con 0x7f40840d2820 2023-11-28T13:07:49.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.286+0000 7f408bce9700 1 -- 172.21.15.6:0/3288903063 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40840cd7c0 con 0x7f40840d2820 2023-11-28T13:07:49.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.286+0000 7f408bce9700 1 -- 172.21.15.6:0/3288903063 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40840cf580 con 0x7f40840d2820 2023-11-28T13:07:49.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.287+0000 7f407a7fc700 1 -- 172.21.15.6:0/3288903063 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4080010050 con 0x7f40840d2820 2023-11-28T13:07:49.292 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.287+0000 7f407a7fc700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40680682f0 0x7f406806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.287+0000 7f407a7fc700 1 -- 172.21.15.6:0/3288903063 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f4080085720 con 0x7f40840d2820 2023-11-28T13:07:49.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.287+0000 7f4089a85700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40680682f0 0x7f406806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.288+0000 7f4089a85700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40680682f0 0x7f406806a7a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f40740090b0 tx=0x7f4074008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:49.293 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.292+0000 7f407a7fc700 1 -- 172.21.15.6:0/3288903063 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f40800520c0 con 0x7f40840d2820 2023-11-28T13:07:49.485 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.483+0000 7f408bce9700 1 -- 172.21.15.6:0/3288903063 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4084068570 con 0x7f40680682f0 2023-11-28T13:07:49.487 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.486+0000 7f407a7fc700 1 -- 172.21.15.6:0/3288903063 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73792 (secure 0 0 0) 0x7f4084068570 con 0x7f40680682f0 2023-11-28T13:07:49.488 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:49.489 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 -- 172.21.15.6:0/3288903063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40680682f0 msgr2=0x7f406806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40680682f0 0x7f406806a7a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f40740090b0 tx=0x7f4074008030 comp rx=0 tx=0).stop 2023-11-28T13:07:49.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 -- 172.21.15.6:0/3288903063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d2820 msgr2=0x7f40841576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d2820 0x7f40841576f0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f408000c4d0 tx=0x7f408000c7e0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 -- 172.21.15.6:0/3288903063 shutdown_connections 2023-11-28T13:07:49.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f408415c640 0x7f40841559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.490 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f40680682f0 0x7f406806a7a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f408406bf30 0x7f408415f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 --2- 172.21.15.6:0/3288903063 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40840d2820 0x7f40841576f0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 -- 172.21.15.6:0/3288903063 >> 172.21.15.6:0/3288903063 conn(0x7f40840d2df0 msgr2=0x7f40840b8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:49.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 -- 172.21.15.6:0/3288903063 shutdown_connections 2023-11-28T13:07:49.491 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.488+0000 7f408bce9700 1 -- 172.21.15.6:0/3288903063 wait complete. 2023-11-28T13:07:49.491 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:07:49.502 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-11-28T13:07:49.051700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81300,"kb_used_data":7700,"kb_used_omap":0,"kb_used_meta":73536,"kb_avail":754893420,"statfs":{"total":773094113280,"available":773010862080,"internally_reserved":0,"allocated":7884800,"data_stored":4335141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75300864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12289,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-74,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001419"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:34.058263+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:06:34.058263+0000","last_peered":"2023-11-28T13:06:34.058263+0000","last_clean":"2023-11-28T13:06:34.058263+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:06:34.058263+0000","last_undegraded":"2023-11-28T13:06:34.058263+0000","last_fullsized":"2023-11-28T13:06:34.058263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:34.048398+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:06:34.048398+0000","last_peered":"2023-11-28T13:06:34.048398+0000","last_clean":"2023-11-28T13:06:34.048398+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:06:34.048398+0000","last_undegraded":"2023-11-28T13:06:34.048398+0000","last_fullsized":"2023-11-28T13:06:34.048398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:29.034409+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:04:29.034409+0000","last_peered":"2023-11-28T13:04:29.034409+0000","last_clean":"2023-11-28T13:04:29.034409+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:04:29.034409+0000","last_undegraded":"2023-11-28T13:04:29.034409+0000","last_fullsized":"2023-11-28T13:04:29.034409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":215,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:19.087021+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:06:19.087021+0000","last_peered":"2023-11-28T13:06:19.087021+0000","last_clean":"2023-11-28T13:06:19.087021+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:06:19.087021+0000","last_undegraded":"2023-11-28T13:06:19.087021+0000","last_fullsized":"2023-11-28T13:06:19.087021+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:34.088663+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:06:34.088663+0000","last_peered":"2023-11-28T13:06:34.088663+0000","last_clean":"2023-11-28T13:06:34.088663+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:06:34.088663+0000","last_undegraded":"2023-11-28T13:06:34.088663+0000","last_fullsized":"2023-11-28T13:06:34.088663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:19.086981+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:06:19.086981+0000","last_peered":"2023-11-28T13:06:19.086981+0000","last_clean":"2023-11-28T13:06:19.086981+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:06:19.086981+0000","last_undegraded":"2023-11-28T13:06:19.086981+0000","last_fullsized":"2023-11-28T13:06:19.086981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":57,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:29.043386+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:04:29.043386+0000","last_peered":"2023-11-28T13:04:29.043386+0000","last_clean":"2023-11-28T13:04:29.043386+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:04:29.043386+0000","last_undegraded":"2023-11-28T13:04:29.043386+0000","last_fullsized":"2023-11-28T13:04:29.043386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:39.047071+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:04:39.047071+0000","last_peered":"2023-11-28T13:04:39.047071+0000","last_clean":"2023-11-28T13:04:39.047071+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:04:39.047071+0000","last_undegraded":"2023-11-28T13:04:39.047071+0000","last_fullsized":"2023-11-28T13:04:39.047071+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:39.047137+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:04:39.047137+0000","last_peered":"2023-11-28T13:04:39.047137+0000","last_clean":"2023-11-28T13:04:39.047137+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:04:39.047137+0000","last_undegraded":"2023-11-28T13:04:39.047137+0000","last_fullsized":"2023-11-28T13:04:39.047137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11076,"kb_used_data":1404,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360764,"statfs":{"total":96636764160,"available":96625422336,"internally_reserved":0,"allocated":1437696,"data_stored":986739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.49199999999999999},"min":{"1min":0.49399999999999999,"5min":0.373,"15min":0.24099999999999999},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.877},"last":0.371},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.53700000000000003},"min":{"1min":0.46899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.52000000000000002},"min":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.313},"max":{"1min":0.87,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.55000000000000004},"min":{"1min":0.50700000000000001,"5min":0.34999999999999998,"15min":0.33000000000000002},"max":{"1min":0.77200000000000002,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.433}]},{"osd":2,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60599999999999998,"15min":0.55000000000000004},"min":{"1min":0.53500000000000003,"5min":0.375,"15min":0.36899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.628,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.875,"15min":0.875},"last":0.41099999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.56100000000000005},"min":{"1min":0.49199999999999999,"5min":0.42699999999999999,"15min":0.39300000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.59099999999999997},"min":{"1min":0.49099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82199999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53700000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81899999999999995,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.61799999999999999,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.28499999999999998},"max":{"1min":0.87,"5min":7.9939999999999998,"15min":7.9939999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68200000000000005,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84799999999999998,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82799999999999996,"5min":4.133,"15min":4.133},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89000000000000001,"5min":8.3000000000000007,"15min":8.3000000000000007},"last":0.623}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":388993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56000000000000005,"15min":0.496},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57299999999999995,"15min":0.51400000000000001},"min":{"1min":0.34300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.30499999999999999},"max":{"1min":0.75700000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.505},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.42999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.79500000000000004,"15min":0.87},"last":0.59599999999999997}]},{"osd":2,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65800000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57399999999999995,"15min":0.53200000000000003},"min":{"1min":0.434,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60799999999999998,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.49299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.71499999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52500000000000002,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.84599999999999997,"15min":0.85399999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55100000000000005,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73299999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.53300000000000003,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68899999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77800000000000002,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94360952,"statfs":{"total":96636764160,"available":96625614848,"internally_reserved":0,"allocated":1441792,"data_stored":990371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.32800000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.373},"min":{"1min":0.29299999999999998,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39800000000000002,"15min":0.36499999999999999},"min":{"1min":0.26200000000000001,"5min":0.252,"15min":0.25},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.78500000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.437,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23699999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33500000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.41799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41599999999999998,"15min":0.38400000000000001},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.24099999999999999},"max":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.69699999999999995},"last":0.35699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79200000000000004,"5min":0.93200000000000005,"15min":1.1259999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.629,"15min":0.58599999999999997},"min":{"1min":0.50600000000000001,"5min":0.46500000000000002,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67100000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61099999999999999,"15min":0.58099999999999996},"min":{"1min":0.48899999999999999,"5min":0.41299999999999998,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.48099999999999998,"5min":0.42699999999999999,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.98299999999999998,"15min":1.157},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.41799999999999998,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.625},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.54300000000000004,"5min":0.46200000000000002,"15min":0.39000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0169999999999999,"15min":1.1759999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.625},"min":{"1min":0.52000000000000002,"5min":0.40500000000000003,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.624},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.41599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.74199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10024,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361816,"statfs":{"total":96636764160,"available":96626499584,"internally_reserved":0,"allocated":819200,"data_stored":387915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.35399999999999998},"min":{"1min":0.223,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.61399999999999999,"5min":0.623,"15min":0.623},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40899999999999997,"15min":0.36399999999999999},"min":{"1min":0.30199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41199999999999998,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.443,"15min":0.39500000000000002},"min":{"1min":0.33700000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.66000000000000003},"last":0.40200000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66700000000000004,"5min":0.746,"15min":0.746},"last":0.435},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47699999999999998,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66900000000000004,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.42799999999999999,"15min":0.42699999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.61399999999999999},"min":{"1min":0.505,"5min":0.44400000000000001,"15min":0.42999999999999999},"max":{"1min":0.83599999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.44900000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":1.5509999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.65200000000000002},"min":{"1min":0.57399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.996},"last":0.624}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66200000000000003,"15min":0.63},"min":{"1min":0.55500000000000005,"5min":0.45200000000000001,"15min":0.41499999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.98299999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.46999999999999997,"15min":0.40200000000000002},"max":{"1min":0.90200000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9792,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362048,"statfs":{"total":96636764160,"available":96626737152,"internally_reserved":0,"allocated":843776,"data_stored":400408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.379,"15min":0.36199999999999999},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.20399999999999999},"max":{"1min":0.46000000000000002,"5min":0.628,"15min":0.70999999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.378},"min":{"1min":0.27800000000000002,"5min":0.24199999999999999,"15min":0.23300000000000001},"max":{"1min":0.47699999999999998,"5min":0.60899999999999999,"15min":0.72299999999999998},"last":0.32200000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.41799999999999998,"15min":0.40200000000000002},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.24099999999999999},"max":{"1min":0.48199999999999998,"5min":0.63200000000000001,"15min":0.65700000000000003},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.443,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.47699999999999998,"5min":0.63200000000000001,"15min":0.69399999999999995},"last":0.36399999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40600000000000003,"15min":0.38100000000000001},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51200000000000001,"5min":0.65400000000000003,"15min":0.73099999999999998},"last":0.308},{"interface":"front","average":{"1min":0.437,"5min":0.47299999999999998,"15min":0.44400000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.53900000000000003,"5min":0.71199999999999997,"15min":0.748},"last":0.374}]},{"osd":4,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.61699999999999999},"min":{"1min":0.46000000000000002,"5min":0.39600000000000002,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":0.93500000000000005,"15min":0.96199999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.624},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.36699999999999999},"max":{"1min":0.874,"5min":0.96299999999999997,"15min":1.0649999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.63400000000000001},"min":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.40300000000000002},"max":{"1min":0.92800000000000005,"5min":0.94099999999999995,"15min":0.96899999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.44600000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.58699999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.52700000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88400000000000001,"5min":1.022,"15min":1.022},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67300000000000004,"15min":0.63900000000000001},"min":{"1min":0.499,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69599999999999995,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86299999999999999,"5min":1.022,"15min":1.0309999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.65200000000000002},"min":{"1min":0.48099999999999998,"5min":0.42499999999999999,"15min":0.38400000000000001},"max":{"1min":0.84299999999999997,"5min":1.004,"15min":1.004},"last":0.56499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9984,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361856,"statfs":{"total":96636764160,"available":96626540544,"internally_reserved":0,"allocated":843776,"data_stored":395883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.189},"max":{"1min":0.47099999999999997,"5min":0.61199999999999999,"15min":0.75800000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39400000000000002,"15min":0.35299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.48999999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.55800000000000005,"5min":0.64300000000000002,"15min":1.1279999999999999},"last":0.317},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.40600000000000003},"min":{"1min":0.318,"5min":0.252,"15min":0.22600000000000001},"max":{"1min":0.52900000000000003,"5min":0.63200000000000001,"15min":0.82799999999999996},"last":0.20899999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.52700000000000002},"min":{"1min":0.314,"5min":0.27000000000000002,"15min":0.24399999999999999},"max":{"1min":0.54600000000000004,"5min":0.59399999999999997,"15min":5.0540000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.432,"15min":0.45100000000000001},"min":{"1min":0.25800000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.67500000000000004,"15min":1.9390000000000001},"last":0.23200000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.64800000000000002},"min":{"1min":0.45400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":1.6919999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68999999999999995,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.97099999999999997,"15min":1.2},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.625,"15min":0.57199999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.92500000000000004,"15min":1.169},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65200000000000002,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.90500000000000003,"5min":0.94599999999999995,"15min":1.6519999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93000000000000005,"5min":1.01,"15min":1.23},"last":0.42499999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.65400000000000003},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86799999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69499999999999995,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90000000000000002,"5min":1.018,"15min":1.7250000000000001},"last":0.40500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":839680,"data_stored":395841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32000000000000001},"max":{"1min":0.77300000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.873,"15min":0.873},"last":0.747}]},{"osd":1,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60899999999999999,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.34000000000000002},"max":{"1min":0.81000000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.53200000000000003},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35199999999999998},"max":{"1min":0.79000000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61699999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.36899999999999999,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.85999999999999999},"last":0.57299999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.55600000000000005},"min":{"1min":0.45600000000000002,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.879},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.58899999999999997},"min":{"1min":0.442,"5min":0.41199999999999998,"15min":0.372},"max":{"1min":0.81000000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51500000000000001,"15min":0.56200000000000006},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66700000000000004,"5min":0.79800000000000004,"15min":0.95199999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54000000000000004,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72299999999999998,"5min":0.77300000000000002,"15min":0.79400000000000004},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.58399999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80000000000000004,"5min":0.82599999999999996,"15min":0.84999999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60199999999999998,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81899999999999995,"5min":0.88500000000000001,"15min":1.117},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59099999999999997,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9776,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362064,"statfs":{"total":96636764160,"available":96626753536,"internally_reserved":0,"allocated":827392,"data_stored":388991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.52300000000000002},"min":{"1min":0.39900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86099999999999999,"5min":0.875,"15min":0.875},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68200000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60799999999999998,"15min":0.55700000000000005},"min":{"1min":0.45600000000000002,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57999999999999996,"15min":0.52300000000000002},"min":{"1min":0.41799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.50700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63600000000000001,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.41699999999999998,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.498}]},{"osd":3,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.59599999999999997},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.44},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65700000000000003,"15min":0.64800000000000002},"min":{"1min":0.52200000000000002,"5min":0.41599999999999998,"15min":0.39200000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60799999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.95999999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.61099999999999999},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.84599999999999997,"15min":1.0029999999999999},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69999999999999996,"5min":0.755,"15min":0.85899999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.85399999999999998,"5min":0.91900000000000004,"15min":1.0349999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56599999999999995,"15min":0.59799999999999998},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.89800000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60299999999999998,"15min":0.65000000000000002},"min":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.89100000000000001,"15min":1.052},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:07:49.505 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:07:49.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.629+0000 7f93bbf5b700 1 Processor -- start 2023-11-28T13:07:49.630 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.629+0000 7f93bbf5b700 1 -- start start 2023-11-28T13:07:49.633 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93bbf5b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 0x7f93b4167590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93bbf5b700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b40b30b0 con 0x7f93b40b2660 2023-11-28T13:07:49.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93bbf5b700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b406c020 con 0x7f93b415c560 2023-11-28T13:07:49.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93bbf5b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40d09c0 con 0x7f93b40ceec0 2023-11-28T13:07:49.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93b94f6700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f93b40b2660 0x7f93b40ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51114/0 (socket says 172.21.15.6:51114) 2023-11-28T13:07:49.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93b94f6700 1 -- 172.21.15.6:0/1975567591 learned_addr learned my addr 172.21.15.6:0/1975567591 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:49.634 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93ba4f8700 1 --2- 172.21.15.6:0/1975567591 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 0x7f93b4167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93b8cf5700 1 -- 172.21.15.6:0/1975567591 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680397214 0 0) 0x7f93b40b30b0 con 0x7f93b40b2660 2023-11-28T13:07:49.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93b8cf5700 1 -- 172.21.15.6:0/1975567591 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93a00035b0 con 0x7f93b40b2660 2023-11-28T13:07:49.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93b8cf5700 1 -- 172.21.15.6:0/1975567591 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627749312 0 0) 0x7f93b406c020 con 0x7f93b415c560 2023-11-28T13:07:49.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93b8cf5700 1 -- 172.21.15.6:0/1975567591 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93b40b30b0 con 0x7f93b415c560 2023-11-28T13:07:49.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93b8cf5700 1 -- 172.21.15.6:0/1975567591 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2538420399 0 0) 0x7f93a00035b0 con 0x7f93b40b2660 2023-11-28T13:07:49.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93ba4f8700 1 -- 172.21.15.6:0/1975567591 >> v1:172.21.15.61:6789/0 conn(0x7f93b415c560 legacy=0x7f93b406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.635 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93ba4f8700 1 -- 172.21.15.6:0/1975567591 >> v1:172.21.15.6:6789/0 conn(0x7f93b40b2660 legacy=0x7f93b40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.630+0000 7f93ba4f8700 1 -- 172.21.15.6:0/1975567591 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b40b2af0 con 0x7f93b40ceec0 2023-11-28T13:07:49.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93ba4f8700 1 --2- 172.21.15.6:0/1975567591 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 0x7f93b4167590 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009dc0 tx=0x7f93ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc1f208ceee671 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:49.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93b8cf5700 1 -- 172.21.15.6:0/1975567591 <== mon.2 v2:172.21.15.61:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00e070 con 0x7f93b40ceec0 2023-11-28T13:07:49.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93b8cf5700 1 -- 172.21.15.6:0/1975567591 <== mon.2 v2:172.21.15.61:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac003a20 con 0x7f93b40ceec0 2023-11-28T13:07:49.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93b8cf5700 1 -- 172.21.15.6:0/1975567591 <== mon.2 v2:172.21.15.61:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8001670 con 0x7f93b40ceec0 2023-11-28T13:07:49.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93bbf5b700 1 -- 172.21.15.6:0/1975567591 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 msgr2=0x7f93b4167590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.636 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93bbf5b700 1 --2- 172.21.15.6:0/1975567591 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 0x7f93b4167590 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009dc0 tx=0x7f93ac00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:49.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93bbf5b700 1 -- 172.21.15.6:0/1975567591 shutdown_connections 2023-11-28T13:07:49.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93bbf5b700 1 --2- 172.21.15.6:0/1975567591 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 0x7f93b4167590 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93bbf5b700 1 -- 172.21.15.6:0/1975567591 >> 172.21.15.6:0/1975567591 conn(0x7f93b40cd660 msgr2=0x7f93b40cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:49.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93bbf5b700 1 -- 172.21.15.6:0/1975567591 shutdown_connections 2023-11-28T13:07:49.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.631+0000 7f93bbf5b700 1 -- 172.21.15.6:0/1975567591 wait complete. 2023-11-28T13:07:49.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.632+0000 7f93bbf5b700 1 Processor -- start 2023-11-28T13:07:49.637 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.633+0000 7f93bbf5b700 1 -- start start 2023-11-28T13:07:49.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.633+0000 7f93bbf5b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93b40b2660 0x7f93b415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.633+0000 7f93bbf5b700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 0x7f93b415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.633+0000 7f93bbf5b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93b415c560 0x7f93b4157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.633+0000 7f93bbf5b700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b415f5a0 con 0x7f93b40b2660 2023-11-28T13:07:49.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.633+0000 7f93bbf5b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b415f050 con 0x7f93b415c560 2023-11-28T13:07:49.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.633+0000 7f93bbf5b700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b415e720 con 0x7f93b40ceec0 2023-11-28T13:07:49.648 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.633+0000 7f93b9cf7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93b40b2660 0x7f93b415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.633+0000 7f93b9cf7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93b40b2660 0x7f93b415eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35568/0 (socket says 172.21.15.6:35568) 2023-11-28T13:07:49.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.633+0000 7f93b9cf7700 1 -- 172.21.15.6:0/3873979083 learned_addr learned my addr 172.21.15.6:0/3873979083 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:49.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93ba4f8700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93b415c560 0x7f93b4157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93b94f6700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 0x7f93b415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93b9cf7700 1 -- 172.21.15.6:0/3873979083 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 msgr2=0x7f93b415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93b9cf7700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 0x7f93b415f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.649 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93b9cf7700 1 -- 172.21.15.6:0/3873979083 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93b415c560 msgr2=0x7f93b4157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93b9cf7700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93b415c560 0x7f93b4157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93b9cf7700 1 -- 172.21.15.6:0/3873979083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac009a70 con 0x7f93b40b2660 2023-11-28T13:07:49.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93ba4f8700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93b415c560 0x7f93b4157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:49.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93b94f6700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 0x7f93b415f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:49.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93b9cf7700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93b40b2660 0x7f93b415eb10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c530 tx=0x7f93b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:49.650 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93a67fc700 1 -- 172.21.15.6:0/3873979083 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00052d0 con 0x7f93b40b2660 2023-11-28T13:07:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93a67fc700 1 -- 172.21.15.6:0/3873979083 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0005450 con 0x7f93b40b2660 2023-11-28T13:07:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93bbf5b700 1 -- 172.21.15.6:0/3873979083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b406bcb0 con 0x7f93b40b2660 2023-11-28T13:07:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93a67fc700 1 -- 172.21.15.6:0/3873979083 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00038c0 con 0x7f93b40b2660 2023-11-28T13:07:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.634+0000 7f93bbf5b700 1 -- 172.21.15.6:0/3873979083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b40b2dd0 con 0x7f93b40b2660 2023-11-28T13:07:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.635+0000 7f93bbf5b700 1 -- 172.21.15.6:0/3873979083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b415c930 con 0x7f93b40b2660 2023-11-28T13:07:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.636+0000 7f93a67fc700 1 -- 172.21.15.6:0/3873979083 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f93b0019030 con 0x7f93b40b2660 2023-11-28T13:07:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.636+0000 7f93a67fc700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93980682f0 0x7f939806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.636+0000 7f93a67fc700 1 -- 172.21.15.6:0/3873979083 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f93b0084e60 con 0x7f93b40b2660 2023-11-28T13:07:49.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.636+0000 7f93b94f6700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93980682f0 0x7f939806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.636+0000 7f93b94f6700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93980682f0 0x7f939806a7a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f93a8001eb0 tx=0x7f93a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:49.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.640+0000 7f93a67fc700 1 -- 172.21.15.6:0/3873979083 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f93b0051880 con 0x7f93b40b2660 2023-11-28T13:07:49.830 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.828+0000 7f93bbf5b700 1 -- 172.21.15.6:0/3873979083 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93b40c5a30 con 0x7f93980682f0 2023-11-28T13:07:49.832 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.831+0000 7f93a67fc700 1 -- 172.21.15.6:0/3873979083 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73792 (secure 0 0 0) 0x7f93b40c5a30 con 0x7f93980682f0 2023-11-28T13:07:49.834 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:49.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 -- 172.21.15.6:0/3873979083 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93980682f0 msgr2=0x7f939806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93980682f0 0x7f939806a7a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f93a8001eb0 tx=0x7f93a8009040 comp rx=0 tx=0).stop 2023-11-28T13:07:49.836 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 -- 172.21.15.6:0/3873979083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93b40b2660 msgr2=0x7f93b415eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93b40b2660 0x7f93b415eb10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c530 tx=0x7f93b000c8f0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 -- 172.21.15.6:0/3873979083 shutdown_connections 2023-11-28T13:07:49.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f93b415c560 0x7f93b4157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.837 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f93b40b2660 0x7f93b415eb10 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f93980682f0 0x7f939806a7a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 --2- 172.21.15.6:0/3873979083 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f93b40ceec0 0x7f93b415f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 -- 172.21.15.6:0/3873979083 >> 172.21.15.6:0/3873979083 conn(0x7f93b40cd660 msgr2=0x7f93b406f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:49.838 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 -- 172.21.15.6:0/3873979083 shutdown_connections 2023-11-28T13:07:49.839 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.833+0000 7f93bbf5b700 1 -- 172.21.15.6:0/3873979083 wait complete. 2023-11-28T13:07:49.839 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:07:49.847 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-11-28T13:07:49.051700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81300,"kb_used_data":7700,"kb_used_omap":0,"kb_used_meta":73536,"kb_avail":754893420,"statfs":{"total":773094113280,"available":773010862080,"internally_reserved":0,"allocated":7884800,"data_stored":4335141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75300864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12289,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-74,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001419"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:34.058263+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:06:34.058263+0000","last_peered":"2023-11-28T13:06:34.058263+0000","last_clean":"2023-11-28T13:06:34.058263+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:06:34.058263+0000","last_undegraded":"2023-11-28T13:06:34.058263+0000","last_fullsized":"2023-11-28T13:06:34.058263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:34.048398+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:06:34.048398+0000","last_peered":"2023-11-28T13:06:34.048398+0000","last_clean":"2023-11-28T13:06:34.048398+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:06:34.048398+0000","last_undegraded":"2023-11-28T13:06:34.048398+0000","last_fullsized":"2023-11-28T13:06:34.048398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:29.034409+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:04:29.034409+0000","last_peered":"2023-11-28T13:04:29.034409+0000","last_clean":"2023-11-28T13:04:29.034409+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:04:29.034409+0000","last_undegraded":"2023-11-28T13:04:29.034409+0000","last_fullsized":"2023-11-28T13:04:29.034409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":215,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:19.087021+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:06:19.087021+0000","last_peered":"2023-11-28T13:06:19.087021+0000","last_clean":"2023-11-28T13:06:19.087021+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:06:19.087021+0000","last_undegraded":"2023-11-28T13:06:19.087021+0000","last_fullsized":"2023-11-28T13:06:19.087021+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:34.088663+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:06:34.088663+0000","last_peered":"2023-11-28T13:06:34.088663+0000","last_clean":"2023-11-28T13:06:34.088663+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:06:34.088663+0000","last_undegraded":"2023-11-28T13:06:34.088663+0000","last_fullsized":"2023-11-28T13:06:34.088663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:19.086981+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:06:19.086981+0000","last_peered":"2023-11-28T13:06:19.086981+0000","last_clean":"2023-11-28T13:06:19.086981+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:06:19.086981+0000","last_undegraded":"2023-11-28T13:06:19.086981+0000","last_fullsized":"2023-11-28T13:06:19.086981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":57,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:29.043386+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:04:29.043386+0000","last_peered":"2023-11-28T13:04:29.043386+0000","last_clean":"2023-11-28T13:04:29.043386+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:04:29.043386+0000","last_undegraded":"2023-11-28T13:04:29.043386+0000","last_fullsized":"2023-11-28T13:04:29.043386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:39.047071+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:04:39.047071+0000","last_peered":"2023-11-28T13:04:39.047071+0000","last_clean":"2023-11-28T13:04:39.047071+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:04:39.047071+0000","last_undegraded":"2023-11-28T13:04:39.047071+0000","last_fullsized":"2023-11-28T13:04:39.047071+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:39.047137+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:04:39.047137+0000","last_peered":"2023-11-28T13:04:39.047137+0000","last_clean":"2023-11-28T13:04:39.047137+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:04:39.047137+0000","last_undegraded":"2023-11-28T13:04:39.047137+0000","last_fullsized":"2023-11-28T13:04:39.047137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11076,"kb_used_data":1404,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360764,"statfs":{"total":96636764160,"available":96625422336,"internally_reserved":0,"allocated":1437696,"data_stored":986739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.49199999999999999},"min":{"1min":0.49399999999999999,"5min":0.373,"15min":0.24099999999999999},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.877},"last":0.371},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.53700000000000003},"min":{"1min":0.46899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.52000000000000002},"min":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.313},"max":{"1min":0.87,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.55000000000000004},"min":{"1min":0.50700000000000001,"5min":0.34999999999999998,"15min":0.33000000000000002},"max":{"1min":0.77200000000000002,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.433}]},{"osd":2,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60599999999999998,"15min":0.55000000000000004},"min":{"1min":0.53500000000000003,"5min":0.375,"15min":0.36899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.628,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.875,"15min":0.875},"last":0.41099999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.56100000000000005},"min":{"1min":0.49199999999999999,"5min":0.42699999999999999,"15min":0.39300000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.59099999999999997},"min":{"1min":0.49099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82199999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53700000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81899999999999995,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.61799999999999999,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.28499999999999998},"max":{"1min":0.87,"5min":7.9939999999999998,"15min":7.9939999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68200000000000005,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84799999999999998,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82799999999999996,"5min":4.133,"15min":4.133},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89000000000000001,"5min":8.3000000000000007,"15min":8.3000000000000007},"last":0.623}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":388993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56000000000000005,"15min":0.496},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57299999999999995,"15min":0.51400000000000001},"min":{"1min":0.34300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.30499999999999999},"max":{"1min":0.75700000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.505},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.42999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.79500000000000004,"15min":0.87},"last":0.59599999999999997}]},{"osd":2,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65800000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57399999999999995,"15min":0.53200000000000003},"min":{"1min":0.434,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60799999999999998,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.49299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.71499999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52500000000000002,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.84599999999999997,"15min":0.85399999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55100000000000005,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73299999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.53300000000000003,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68899999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77800000000000002,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94360952,"statfs":{"total":96636764160,"available":96625614848,"internally_reserved":0,"allocated":1441792,"data_stored":990371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.32800000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.373},"min":{"1min":0.29299999999999998,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39800000000000002,"15min":0.36499999999999999},"min":{"1min":0.26200000000000001,"5min":0.252,"15min":0.25},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.78500000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.437,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23699999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33500000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.41799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41599999999999998,"15min":0.38400000000000001},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.24099999999999999},"max":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.69699999999999995},"last":0.35699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79200000000000004,"5min":0.93200000000000005,"15min":1.1259999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.629,"15min":0.58599999999999997},"min":{"1min":0.50600000000000001,"5min":0.46500000000000002,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67100000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61099999999999999,"15min":0.58099999999999996},"min":{"1min":0.48899999999999999,"5min":0.41299999999999998,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.48099999999999998,"5min":0.42699999999999999,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.98299999999999998,"15min":1.157},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.41799999999999998,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.625},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.54300000000000004,"5min":0.46200000000000002,"15min":0.39000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0169999999999999,"15min":1.1759999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.625},"min":{"1min":0.52000000000000002,"5min":0.40500000000000003,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.624},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.41599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.74199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10024,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361816,"statfs":{"total":96636764160,"available":96626499584,"internally_reserved":0,"allocated":819200,"data_stored":387915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.35399999999999998},"min":{"1min":0.223,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.61399999999999999,"5min":0.623,"15min":0.623},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40899999999999997,"15min":0.36399999999999999},"min":{"1min":0.30199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41199999999999998,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.443,"15min":0.39500000000000002},"min":{"1min":0.33700000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.66000000000000003},"last":0.40200000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66700000000000004,"5min":0.746,"15min":0.746},"last":0.435},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47699999999999998,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66900000000000004,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.42799999999999999,"15min":0.42699999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.61399999999999999},"min":{"1min":0.505,"5min":0.44400000000000001,"15min":0.42999999999999999},"max":{"1min":0.83599999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.44900000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":1.5509999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.65200000000000002},"min":{"1min":0.57399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.996},"last":0.624}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66200000000000003,"15min":0.63},"min":{"1min":0.55500000000000005,"5min":0.45200000000000001,"15min":0.41499999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.98299999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.46999999999999997,"15min":0.40200000000000002},"max":{"1min":0.90200000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9792,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362048,"statfs":{"total":96636764160,"available":96626737152,"internally_reserved":0,"allocated":843776,"data_stored":400408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.379,"15min":0.36199999999999999},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.20399999999999999},"max":{"1min":0.46000000000000002,"5min":0.628,"15min":0.70999999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.378},"min":{"1min":0.27800000000000002,"5min":0.24199999999999999,"15min":0.23300000000000001},"max":{"1min":0.47699999999999998,"5min":0.60899999999999999,"15min":0.72299999999999998},"last":0.32200000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.41799999999999998,"15min":0.40200000000000002},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.24099999999999999},"max":{"1min":0.48199999999999998,"5min":0.63200000000000001,"15min":0.65700000000000003},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.443,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.47699999999999998,"5min":0.63200000000000001,"15min":0.69399999999999995},"last":0.36399999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40600000000000003,"15min":0.38100000000000001},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51200000000000001,"5min":0.65400000000000003,"15min":0.73099999999999998},"last":0.308},{"interface":"front","average":{"1min":0.437,"5min":0.47299999999999998,"15min":0.44400000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.53900000000000003,"5min":0.71199999999999997,"15min":0.748},"last":0.374}]},{"osd":4,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.61699999999999999},"min":{"1min":0.46000000000000002,"5min":0.39600000000000002,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":0.93500000000000005,"15min":0.96199999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.624},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.36699999999999999},"max":{"1min":0.874,"5min":0.96299999999999997,"15min":1.0649999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.63400000000000001},"min":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.40300000000000002},"max":{"1min":0.92800000000000005,"5min":0.94099999999999995,"15min":0.96899999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.44600000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.58699999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.52700000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88400000000000001,"5min":1.022,"15min":1.022},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67300000000000004,"15min":0.63900000000000001},"min":{"1min":0.499,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69599999999999995,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86299999999999999,"5min":1.022,"15min":1.0309999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.65200000000000002},"min":{"1min":0.48099999999999998,"5min":0.42499999999999999,"15min":0.38400000000000001},"max":{"1min":0.84299999999999997,"5min":1.004,"15min":1.004},"last":0.56499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9984,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361856,"statfs":{"total":96636764160,"available":96626540544,"internally_reserved":0,"allocated":843776,"data_stored":395883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.189},"max":{"1min":0.47099999999999997,"5min":0.61199999999999999,"15min":0.75800000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39400000000000002,"15min":0.35299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.48999999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.55800000000000005,"5min":0.64300000000000002,"15min":1.1279999999999999},"last":0.317},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.40600000000000003},"min":{"1min":0.318,"5min":0.252,"15min":0.22600000000000001},"max":{"1min":0.52900000000000003,"5min":0.63200000000000001,"15min":0.82799999999999996},"last":0.20899999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.52700000000000002},"min":{"1min":0.314,"5min":0.27000000000000002,"15min":0.24399999999999999},"max":{"1min":0.54600000000000004,"5min":0.59399999999999997,"15min":5.0540000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.432,"15min":0.45100000000000001},"min":{"1min":0.25800000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.67500000000000004,"15min":1.9390000000000001},"last":0.23200000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.64800000000000002},"min":{"1min":0.45400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":1.6919999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68999999999999995,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.97099999999999997,"15min":1.2},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.625,"15min":0.57199999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.92500000000000004,"15min":1.169},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65200000000000002,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.90500000000000003,"5min":0.94599999999999995,"15min":1.6519999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93000000000000005,"5min":1.01,"15min":1.23},"last":0.42499999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.65400000000000003},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86799999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69499999999999995,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90000000000000002,"5min":1.018,"15min":1.7250000000000001},"last":0.40500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":839680,"data_stored":395841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32000000000000001},"max":{"1min":0.77300000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.873,"15min":0.873},"last":0.747}]},{"osd":1,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60899999999999999,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.34000000000000002},"max":{"1min":0.81000000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.53200000000000003},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35199999999999998},"max":{"1min":0.79000000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61699999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.36899999999999999,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.85999999999999999},"last":0.57299999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.55600000000000005},"min":{"1min":0.45600000000000002,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.879},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.58899999999999997},"min":{"1min":0.442,"5min":0.41199999999999998,"15min":0.372},"max":{"1min":0.81000000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51500000000000001,"15min":0.56200000000000006},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66700000000000004,"5min":0.79800000000000004,"15min":0.95199999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54000000000000004,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72299999999999998,"5min":0.77300000000000002,"15min":0.79400000000000004},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.58399999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80000000000000004,"5min":0.82599999999999996,"15min":0.84999999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60199999999999998,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81899999999999995,"5min":0.88500000000000001,"15min":1.117},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59099999999999997,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9776,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362064,"statfs":{"total":96636764160,"available":96626753536,"internally_reserved":0,"allocated":827392,"data_stored":388991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.52300000000000002},"min":{"1min":0.39900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86099999999999999,"5min":0.875,"15min":0.875},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68200000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60799999999999998,"15min":0.55700000000000005},"min":{"1min":0.45600000000000002,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57999999999999996,"15min":0.52300000000000002},"min":{"1min":0.41799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.50700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63600000000000001,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.41699999999999998,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.498}]},{"osd":3,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.59599999999999997},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.44},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65700000000000003,"15min":0.64800000000000002},"min":{"1min":0.52200000000000002,"5min":0.41599999999999998,"15min":0.39200000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60799999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.95999999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.61099999999999999},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.84599999999999997,"15min":1.0029999999999999},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69999999999999996,"5min":0.755,"15min":0.85899999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.85399999999999998,"5min":0.91900000000000004,"15min":1.0349999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56599999999999995,"15min":0.59799999999999998},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.89800000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60299999999999998,"15min":0.65000000000000002},"min":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.89100000000000001,"15min":1.052},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:07:49.850 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T13:07:49.851 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:07:49.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.974+0000 7f26e6ed2700 1 Processor -- start 2023-11-28T13:07:49.976 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.974+0000 7f26e6ed2700 1 -- start start 2023-11-28T13:07:49.978 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e6ed2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f26e00b2660 0x7f26e00ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e6ed2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f26e00cecf0 0x7f26e006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e6ed2700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e00b30b0 con 0x7f26e015c560 2023-11-28T13:07:49.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e6ed2700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e006c020 con 0x7f26e00cecf0 2023-11-28T13:07:49.979 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e6ed2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e00cf110 con 0x7f26e00b2660 2023-11-28T13:07:49.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e546f700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f26e00cecf0 0x7f26e006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e4c6e700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f26e015c560 0x7f26e006f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51128/0 (socket says 172.21.15.6:51128) 2023-11-28T13:07:49.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e4c6e700 1 -- 172.21.15.6:0/702884765 learned_addr learned my addr 172.21.15.6:0/702884765 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:49.980 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26dffff700 1 --2- 172.21.15.6:0/702884765 >> v2:172.21.15.61:3300/0 conn(0x7f26e00b2660 0x7f26e00ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e546f700 1 -- 172.21.15.6:0/702884765 >> v2:172.21.15.61:3300/0 conn(0x7f26e00b2660 msgr2=0x7f26e00ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:49.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e546f700 1 --2- 172.21.15.6:0/702884765 >> v2:172.21.15.61:3300/0 conn(0x7f26e00b2660 0x7f26e00ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e546f700 1 -- 172.21.15.6:0/702884765 >> v1:172.21.15.6:6789/0 conn(0x7f26e015c560 legacy=0x7f26e006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.981 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26df7fe700 1 -- 172.21.15.6:0/702884765 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980151300 0 0) 0x7f26e00b30b0 con 0x7f26e015c560 2023-11-28T13:07:49.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e546f700 1 -- 172.21.15.6:0/702884765 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e00d09c0 con 0x7f26e00cecf0 2023-11-28T13:07:49.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.975+0000 7f26e546f700 1 --2- 172.21.15.6:0/702884765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00cecf0 0x7f26e006c4b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f26e0071e10 tx=0x7f26d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41d2a285d035d17 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:49.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26df7fe700 1 -- 172.21.15.6:0/702884765 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d800e070 con 0x7f26e00cecf0 2023-11-28T13:07:49.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26df7fe700 1 -- 172.21.15.6:0/702884765 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d800bbd0 con 0x7f26e00cecf0 2023-11-28T13:07:49.982 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26df7fe700 1 -- 172.21.15.6:0/702884765 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8012430 con 0x7f26e00cecf0 2023-11-28T13:07:49.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26e6ed2700 1 -- 172.21.15.6:0/702884765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00cecf0 msgr2=0x7f26e006c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26e6ed2700 1 --2- 172.21.15.6:0/702884765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00cecf0 0x7f26e006c4b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f26e0071e10 tx=0x7f26d800b330 comp rx=0 tx=0).stop 2023-11-28T13:07:49.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26e6ed2700 1 -- 172.21.15.6:0/702884765 shutdown_connections 2023-11-28T13:07:49.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26e6ed2700 1 --2- 172.21.15.6:0/702884765 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00cecf0 0x7f26e006c4b0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26e6ed2700 1 --2- 172.21.15.6:0/702884765 >> v2:172.21.15.61:3300/0 conn(0x7f26e00b2660 0x7f26e00ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26e6ed2700 1 -- 172.21.15.6:0/702884765 >> 172.21.15.6:0/702884765 conn(0x7f26e00cd660 msgr2=0x7f26e00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:49.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26e6ed2700 1 -- 172.21.15.6:0/702884765 shutdown_connections 2023-11-28T13:07:49.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.976+0000 7f26e6ed2700 1 -- 172.21.15.6:0/702884765 wait complete. 2023-11-28T13:07:49.984 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.977+0000 7f26e6ed2700 1 Processor -- start 2023-11-28T13:07:49.985 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.977+0000 7f26e6ed2700 1 -- start start 2023-11-28T13:07:49.987 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.978+0000 7f26e6ed2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00b2660 0x7f26e01f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.978+0000 7f26e6ed2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26e00cecf0 0x7f26e01605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.978+0000 7f26e6ed2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26e015c560 0x7f26e00b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.978+0000 7f26e6ed2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e0160110 con 0x7f26e00b2660 2023-11-28T13:07:49.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.978+0000 7f26e6ed2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e01f2f80 con 0x7f26e00cecf0 2023-11-28T13:07:49.988 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.978+0000 7f26e6ed2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e015d350 con 0x7f26e015c560 2023-11-28T13:07:49.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.978+0000 7f26e4c6e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00b2660 0x7f26e01f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.978+0000 7f26e4c6e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00b2660 0x7f26e01f2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35582/0 (socket says 172.21.15.6:35582) 2023-11-28T13:07:49.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.978+0000 7f26e4c6e700 1 -- 172.21.15.6:0/3748069148 learned_addr learned my addr 172.21.15.6:0/3748069148 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:49.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26dffff700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26e00cecf0 0x7f26e01605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26e546f700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26e015c560 0x7f26e00b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26e4c6e700 1 -- 172.21.15.6:0/3748069148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26e015c560 msgr2=0x7f26e00b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26e4c6e700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26e015c560 0x7f26e00b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.990 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26e4c6e700 1 -- 172.21.15.6:0/3748069148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26e00cecf0 msgr2=0x7f26e01605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26e4c6e700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26e00cecf0 0x7f26e01605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26e4c6e700 1 -- 172.21.15.6:0/3748069148 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d8009a70 con 0x7f26e00b2660 2023-11-28T13:07:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26dffff700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26e00cecf0 0x7f26e01605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26e4c6e700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00b2660 0x7f26e01f2a40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f26d400c550 tx=0x7f26d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:49.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26dd7fa700 1 -- 172.21.15.6:0/3748069148 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4004c80 con 0x7f26e00b2660 2023-11-28T13:07:49.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26dd7fa700 1 -- 172.21.15.6:0/3748069148 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d4004e00 con 0x7f26e00b2660 2023-11-28T13:07:49.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26e6ed2700 1 -- 172.21.15.6:0/3748069148 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e00b2dd0 con 0x7f26e00b2660 2023-11-28T13:07:49.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26dd7fa700 1 -- 172.21.15.6:0/3748069148 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4007310 con 0x7f26e00b2660 2023-11-28T13:07:49.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.979+0000 7f26e6ed2700 1 -- 172.21.15.6:0/3748069148 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e00b2af0 con 0x7f26e00b2660 2023-11-28T13:07:49.992 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.980+0000 7f26e6ed2700 1 -- 172.21.15.6:0/3748069148 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e006bcb0 con 0x7f26e00b2660 2023-11-28T13:07:49.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.980+0000 7f26dd7fa700 1 -- 172.21.15.6:0/3748069148 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f26d401a020 con 0x7f26e00b2660 2023-11-28T13:07:49.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.981+0000 7f26dd7fa700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26c40682f0 0x7f26c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:49.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.981+0000 7f26dd7fa700 1 -- 172.21.15.6:0/3748069148 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f26d4085700 con 0x7f26e00b2660 2023-11-28T13:07:49.993 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.981+0000 7f26dffff700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26c40682f0 0x7f26c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:49.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.981+0000 7f26dffff700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26c40682f0 0x7f26c406a7a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f26e00afb80 tx=0x7f26cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:49.994 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:49.985+0000 7f26dd7fa700 1 -- 172.21.15.6:0/3748069148 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f26d4052170 con 0x7f26e00b2660 2023-11-28T13:07:50.174 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.172+0000 7f26e6ed2700 1 -- 172.21.15.6:0/3748069148 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26e01603f0 con 0x7f26c40682f0 2023-11-28T13:07:50.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.175+0000 7f26dd7fa700 1 -- 172.21.15.6:0/3748069148 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73792 (secure 0 0 0) 0x7f26e01603f0 con 0x7f26c40682f0 2023-11-28T13:07:50.177 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:50.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 -- 172.21.15.6:0/3748069148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26c40682f0 msgr2=0x7f26c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:50.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26c40682f0 0x7f26c406a7a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f26e00afb80 tx=0x7f26cc008040 comp rx=0 tx=0).stop 2023-11-28T13:07:50.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 -- 172.21.15.6:0/3748069148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00b2660 msgr2=0x7f26e01f2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:50.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00b2660 0x7f26e01f2a40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f26d400c550 tx=0x7f26d400c910 comp rx=0 tx=0).stop 2023-11-28T13:07:50.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 -- 172.21.15.6:0/3748069148 shutdown_connections 2023-11-28T13:07:50.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f26c40682f0 0x7f26c406a7a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f26e015c560 0x7f26e00b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e00b2660 0x7f26e01f2a40 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 --2- 172.21.15.6:0/3748069148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26e00cecf0 0x7f26e01605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 -- 172.21.15.6:0/3748069148 >> 172.21.15.6:0/3748069148 conn(0x7f26e00cd660 msgr2=0x7f26e00afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:50.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 -- 172.21.15.6:0/3748069148 shutdown_connections 2023-11-28T13:07:50.180 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.177+0000 7f26e6ed2700 1 -- 172.21.15.6:0/3748069148 wait complete. 2023-11-28T13:07:50.181 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:07:50.191 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-11-28T13:07:49.051700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81300,"kb_used_data":7700,"kb_used_omap":0,"kb_used_meta":73536,"kb_avail":754893420,"statfs":{"total":773094113280,"available":773010862080,"internally_reserved":0,"allocated":7884800,"data_stored":4335141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75300864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12289,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-74,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001419"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:34.058263+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:06:34.058263+0000","last_peered":"2023-11-28T13:06:34.058263+0000","last_clean":"2023-11-28T13:06:34.058263+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:06:34.058263+0000","last_undegraded":"2023-11-28T13:06:34.058263+0000","last_fullsized":"2023-11-28T13:06:34.058263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:34.048398+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:06:34.048398+0000","last_peered":"2023-11-28T13:06:34.048398+0000","last_clean":"2023-11-28T13:06:34.048398+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:06:34.048398+0000","last_undegraded":"2023-11-28T13:06:34.048398+0000","last_fullsized":"2023-11-28T13:06:34.048398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-30T00:45:05.299448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:29.034409+0000","last_change":"2023-11-28T13:00:15.958631+0000","last_active":"2023-11-28T13:04:29.034409+0000","last_peered":"2023-11-28T13:04:29.034409+0000","last_clean":"2023-11-28T13:04:29.034409+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:04:29.034409+0000","last_undegraded":"2023-11-28T13:04:29.034409+0000","last_fullsized":"2023-11-28T13:04:29.034409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:10:41.387175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":215,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:19.087021+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:06:19.087021+0000","last_peered":"2023-11-28T13:06:19.087021+0000","last_clean":"2023-11-28T13:06:19.087021+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:06:19.087021+0000","last_undegraded":"2023-11-28T13:06:19.087021+0000","last_fullsized":"2023-11-28T13:06:19.087021+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:34.088663+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:06:34.088663+0000","last_peered":"2023-11-28T13:06:34.088663+0000","last_clean":"2023-11-28T13:06:34.088663+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:06:34.088663+0000","last_undegraded":"2023-11-28T13:06:34.088663+0000","last_fullsized":"2023-11-28T13:06:34.088663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:06:19.086981+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:06:19.086981+0000","last_peered":"2023-11-28T13:06:19.086981+0000","last_clean":"2023-11-28T13:06:19.086981+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:06:19.086981+0000","last_undegraded":"2023-11-28T13:06:19.086981+0000","last_fullsized":"2023-11-28T13:06:19.086981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":57,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:29.043386+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:04:29.043386+0000","last_peered":"2023-11-28T13:04:29.043386+0000","last_clean":"2023-11-28T13:04:29.043386+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:04:29.043386+0000","last_undegraded":"2023-11-28T13:04:29.043386+0000","last_fullsized":"2023-11-28T13:04:29.043386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:39.047071+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:04:39.047071+0000","last_peered":"2023-11-28T13:04:39.047071+0000","last_clean":"2023-11-28T13:04:39.047071+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:04:39.047071+0000","last_undegraded":"2023-11-28T13:04:39.047071+0000","last_fullsized":"2023-11-28T13:04:39.047071+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T13:04:39.047137+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:04:39.047137+0000","last_peered":"2023-11-28T13:04:39.047137+0000","last_clean":"2023-11-28T13:04:39.047137+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:04:39.047137+0000","last_undegraded":"2023-11-28T13:04:39.047137+0000","last_fullsized":"2023-11-28T13:04:39.047137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11076,"kb_used_data":1404,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360764,"statfs":{"total":96636764160,"available":96625422336,"internally_reserved":0,"allocated":1437696,"data_stored":986739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.49199999999999999},"min":{"1min":0.49399999999999999,"5min":0.373,"15min":0.24099999999999999},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.877},"last":0.371},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.53700000000000003},"min":{"1min":0.46899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.52000000000000002},"min":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.313},"max":{"1min":0.87,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.55000000000000004},"min":{"1min":0.50700000000000001,"5min":0.34999999999999998,"15min":0.33000000000000002},"max":{"1min":0.77200000000000002,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.433}]},{"osd":2,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60599999999999998,"15min":0.55000000000000004},"min":{"1min":0.53500000000000003,"5min":0.375,"15min":0.36899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.628,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.875,"15min":0.875},"last":0.41099999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.56100000000000005},"min":{"1min":0.49199999999999999,"5min":0.42699999999999999,"15min":0.39300000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.59099999999999997},"min":{"1min":0.49099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82199999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.53700000000000003}]},{"osd":4,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81899999999999995,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.61799999999999999,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.28499999999999998},"max":{"1min":0.87,"5min":7.9939999999999998,"15min":7.9939999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68200000000000005,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84799999999999998,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.63900000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82799999999999996,"5min":4.133,"15min":4.133},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89000000000000001,"5min":8.3000000000000007,"15min":8.3000000000000007},"last":0.623}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":388993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56000000000000005,"15min":0.496},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57299999999999995,"15min":0.51400000000000001},"min":{"1min":0.34300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.30499999999999999},"max":{"1min":0.75700000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.505},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.42999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.79500000000000004,"15min":0.87},"last":0.59599999999999997}]},{"osd":2,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65800000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57399999999999995,"15min":0.53200000000000003},"min":{"1min":0.434,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60799999999999998,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.49299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.71499999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52500000000000002,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.84599999999999997,"15min":0.85399999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55100000000000005,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73299999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.53300000000000003,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68899999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77800000000000002,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10888,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94360952,"statfs":{"total":96636764160,"available":96625614848,"internally_reserved":0,"allocated":1441792,"data_stored":990371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.32800000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.373},"min":{"1min":0.29299999999999998,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39800000000000002,"15min":0.36499999999999999},"min":{"1min":0.26200000000000001,"5min":0.252,"15min":0.25},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.78500000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.437,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23699999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33500000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.41799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41599999999999998,"15min":0.38400000000000001},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.24099999999999999},"max":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.69699999999999995},"last":0.35699999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79200000000000004,"5min":0.93200000000000005,"15min":1.1259999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.629,"15min":0.58599999999999997},"min":{"1min":0.50600000000000001,"5min":0.46500000000000002,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.67100000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61099999999999999,"15min":0.58099999999999996},"min":{"1min":0.48899999999999999,"5min":0.41299999999999998,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.48099999999999998,"5min":0.42699999999999999,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.98299999999999998,"15min":1.157},"last":0.67700000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.41799999999999998,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.625},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.54300000000000004,"5min":0.46200000000000002,"15min":0.39000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0169999999999999,"15min":1.1759999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.625},"min":{"1min":0.52000000000000002,"5min":0.40500000000000003,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.624},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.41599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.74199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10024,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361816,"statfs":{"total":96636764160,"available":96626499584,"internally_reserved":0,"allocated":819200,"data_stored":387915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.35399999999999998},"min":{"1min":0.223,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.61399999999999999,"5min":0.623,"15min":0.623},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40899999999999997,"15min":0.36399999999999999},"min":{"1min":0.30199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.35499999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41199999999999998,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.443,"15min":0.39500000000000002},"min":{"1min":0.33700000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.66000000000000003},"last":0.40200000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66700000000000004,"5min":0.746,"15min":0.746},"last":0.435},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47699999999999998,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.45800000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66900000000000004,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.42799999999999999,"15min":0.42699999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.61399999999999999},"min":{"1min":0.505,"5min":0.44400000000000001,"15min":0.42999999999999999},"max":{"1min":0.83599999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.48699999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.44900000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":1.5509999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.65200000000000002},"min":{"1min":0.57399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.996},"last":0.624}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66200000000000003,"15min":0.63},"min":{"1min":0.55500000000000005,"5min":0.45200000000000001,"15min":0.41499999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.98299999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.46999999999999997,"15min":0.40200000000000002},"max":{"1min":0.90200000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9792,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362048,"statfs":{"total":96636764160,"available":96626737152,"internally_reserved":0,"allocated":843776,"data_stored":400408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.379,"15min":0.36199999999999999},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.20399999999999999},"max":{"1min":0.46000000000000002,"5min":0.628,"15min":0.70999999999999996},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.378},"min":{"1min":0.27800000000000002,"5min":0.24199999999999999,"15min":0.23300000000000001},"max":{"1min":0.47699999999999998,"5min":0.60899999999999999,"15min":0.72299999999999998},"last":0.32200000000000001}]},{"osd":1,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.41799999999999998,"15min":0.40200000000000002},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.24099999999999999},"max":{"1min":0.48199999999999998,"5min":0.63200000000000001,"15min":0.65700000000000003},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.443,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.47699999999999998,"5min":0.63200000000000001,"15min":0.69399999999999995},"last":0.36399999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40600000000000003,"15min":0.38100000000000001},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51200000000000001,"5min":0.65400000000000003,"15min":0.73099999999999998},"last":0.308},{"interface":"front","average":{"1min":0.437,"5min":0.47299999999999998,"15min":0.44400000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.53900000000000003,"5min":0.71199999999999997,"15min":0.748},"last":0.374}]},{"osd":4,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.61699999999999999},"min":{"1min":0.46000000000000002,"5min":0.39600000000000002,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":0.93500000000000005,"15min":0.96199999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.624},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.36699999999999999},"max":{"1min":0.874,"5min":0.96299999999999997,"15min":1.0649999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.63400000000000001},"min":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.40300000000000002},"max":{"1min":0.92800000000000005,"5min":0.94099999999999995,"15min":0.96899999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.44600000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.58699999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.52700000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88400000000000001,"5min":1.022,"15min":1.022},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67300000000000004,"15min":0.63900000000000001},"min":{"1min":0.499,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69599999999999995,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86299999999999999,"5min":1.022,"15min":1.0309999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.65200000000000002},"min":{"1min":0.48099999999999998,"5min":0.42499999999999999,"15min":0.38400000000000001},"max":{"1min":0.84299999999999997,"5min":1.004,"15min":1.004},"last":0.56499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9984,"kb_used_data":824,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361856,"statfs":{"total":96636764160,"available":96626540544,"internally_reserved":0,"allocated":843776,"data_stored":395883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.189},"max":{"1min":0.47099999999999997,"5min":0.61199999999999999,"15min":0.75800000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39400000000000002,"15min":0.35299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.48999999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.55800000000000005,"5min":0.64300000000000002,"15min":1.1279999999999999},"last":0.317},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.40600000000000003},"min":{"1min":0.318,"5min":0.252,"15min":0.22600000000000001},"max":{"1min":0.52900000000000003,"5min":0.63200000000000001,"15min":0.82799999999999996},"last":0.20899999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.52700000000000002},"min":{"1min":0.314,"5min":0.27000000000000002,"15min":0.24399999999999999},"max":{"1min":0.54600000000000004,"5min":0.59399999999999997,"15min":5.0540000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.432,"15min":0.45100000000000001},"min":{"1min":0.25800000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.67500000000000004,"15min":1.9390000000000001},"last":0.23200000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.64800000000000002},"min":{"1min":0.45400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":1.6919999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68999999999999995,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.97099999999999997,"15min":1.2},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.625,"15min":0.57199999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.92500000000000004,"15min":1.169},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65200000000000002,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.90500000000000003,"5min":0.94599999999999995,"15min":1.6519999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93000000000000005,"5min":1.01,"15min":1.23},"last":0.42499999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.65400000000000003},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86799999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69499999999999995,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90000000000000002,"5min":1.018,"15min":1.7250000000000001},"last":0.40500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9980,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361860,"statfs":{"total":96636764160,"available":96626544640,"internally_reserved":0,"allocated":839680,"data_stored":395841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32000000000000001},"max":{"1min":0.77300000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.873,"15min":0.873},"last":0.747}]},{"osd":1,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60899999999999999,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.34000000000000002},"max":{"1min":0.81000000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.53200000000000003},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35199999999999998},"max":{"1min":0.79000000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61699999999999999}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.36899999999999999,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.85999999999999999},"last":0.57299999999999995}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.55600000000000005},"min":{"1min":0.45600000000000002,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.879},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.58899999999999997},"min":{"1min":0.442,"5min":0.41199999999999998,"15min":0.372},"max":{"1min":0.81000000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51500000000000001,"15min":0.56200000000000006},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66700000000000004,"5min":0.79800000000000004,"15min":0.95199999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54000000000000004,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72299999999999998,"5min":0.77300000000000002,"15min":0.79400000000000004},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.58399999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80000000000000004,"5min":0.82599999999999996,"15min":0.84999999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60199999999999998,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81899999999999995,"5min":0.88500000000000001,"15min":1.117},"last":0.70399999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59099999999999997,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9776,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362064,"statfs":{"total":96636764160,"available":96626753536,"internally_reserved":0,"allocated":827392,"data_stored":388991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.52300000000000002},"min":{"1min":0.39900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86099999999999999,"5min":0.875,"15min":0.875},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68200000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60799999999999998,"15min":0.55700000000000005},"min":{"1min":0.45600000000000002,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57999999999999996,"15min":0.52300000000000002},"min":{"1min":0.41799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.50700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63600000000000001,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.41699999999999998,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.498}]},{"osd":3,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.59599999999999997},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.44},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65700000000000003,"15min":0.64800000000000002},"min":{"1min":0.52200000000000002,"5min":0.41599999999999998,"15min":0.39200000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60799999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.95999999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.61099999999999999},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.84599999999999997,"15min":1.0029999999999999},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69999999999999996,"5min":0.755,"15min":0.85899999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.85399999999999998,"5min":0.91900000000000004,"15min":1.0349999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56599999999999995,"15min":0.59799999999999998},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.89800000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60299999999999998,"15min":0.65000000000000002},"min":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.89100000000000001,"15min":1.052},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:07:50.194 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T13:07:50.339 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.318+0000 7f42c91cd700 1 Processor -- start 2023-11-28T13:07:50.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.318+0000 7f42c91cd700 1 -- start start 2023-11-28T13:07:50.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.318+0000 7f42c91cd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4161090 0x7f42c40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:50.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.318+0000 7f42c91cd700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f42c40caeb0 0x7f42c40cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:50.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c91cd700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c40ce340 con 0x7f42c415c990 2023-11-28T13:07:50.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c91cd700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c40cebe0 con 0x7f42c40caeb0 2023-11-28T13:07:50.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c91cd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c40cf780 con 0x7f42c4161090 2023-11-28T13:07:50.340 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c2d9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f42c415c990 0x7f42c40cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51142/0 (socket says 172.21.15.6:51142) 2023-11-28T13:07:50.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c2d9d700 1 -- 172.21.15.6:0/3941921111 learned_addr learned my addr 172.21.15.6:0/3941921111 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:50.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875196224 0 0) 0x7f42c40ce340 con 0x7f42c415c990 2023-11-28T13:07:50.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c359e700 1 --2- 172.21.15.6:0/3941921111 >> v2:172.21.15.61:3300/0 conn(0x7f42c40caeb0 0x7f42c40cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:50.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42ac0035b0 con 0x7f42c415c990 2023-11-28T13:07:50.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c259c700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4161090 0x7f42c40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:50.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2567988764 0 0) 0x7f42ac0035b0 con 0x7f42c415c990 2023-11-28T13:07:50.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4161090 msgr2=0x7f42c40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:50.341 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4161090 0x7f42c40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 >> v2:172.21.15.61:3300/0 conn(0x7f42c40caeb0 msgr2=0x7f42c40cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:50.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 --2- 172.21.15.6:0/3941921111 >> v2:172.21.15.61:3300/0 conn(0x7f42c40caeb0 0x7f42c40cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c40cb3f0 con 0x7f42c415c990 2023-11-28T13:07:50.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7f42b4001670 con 0x7f42c415c990 2023-11-28T13:07:50.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 >> v1:172.21.15.6:6789/0 conn(0x7f42c415c990 legacy=0x7f42c40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:50.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42ac003a00 0x7f42ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:50.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 0x7f42ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:50.342 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4161090 0x7f42ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:50.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c40cebe0 con 0x7f42ac004850 2023-11-28T13:07:50.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c40cf780 con 0x7f42ac003a00 2023-11-28T13:07:50.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac00ba90 con 0x7f42c4161090 2023-11-28T13:07:50.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.319+0000 7f42c259c700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 0x7f42ac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:50.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c2d9d700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42ac003a00 0x7f42ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:50.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c359e700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4161090 0x7f42ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:50.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c259c700 1 -- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4161090 msgr2=0x7f42ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:50.343 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c259c700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4161090 0x7f42ac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c259c700 1 -- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42ac003a00 msgr2=0x7f42ac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:50.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c259c700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42ac003a00 0x7f42ac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c259c700 1 -- 172.21.15.6:0/3941921111 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac0035b0 con 0x7f42ac004850 2023-11-28T13:07:50.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c259c700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 0x7f42ac004cc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009b30 tx=0x7f42ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff67cf0672f6037f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:50.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b800e6c0 con 0x7f42ac004850 2023-11-28T13:07:50.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b800e820 con 0x7f42ac004850 2023-11-28T13:07:50.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.320+0000 7f42c1d9b700 1 -- 172.21.15.6:0/3941921111 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b800eb90 con 0x7f42ac004850 2023-11-28T13:07:50.344 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.321+0000 7f42c91cd700 1 -- 172.21.15.6:0/3941921111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 msgr2=0x7f42ac004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:50.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.321+0000 7f42c91cd700 1 --2- 172.21.15.6:0/3941921111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 0x7f42ac004cc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009b30 tx=0x7f42ac005880 comp rx=0 tx=0).stop 2023-11-28T13:07:50.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.321+0000 7f42c359e700 1 -- 172.21.15.6:0/3941921111 reap_dead start 2023-11-28T13:07:50.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.321+0000 7f42c91cd700 1 -- 172.21.15.6:0/3941921111 shutdown_connections 2023-11-28T13:07:50.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.321+0000 7f42c91cd700 1 -- 172.21.15.6:0/3941921111 >> 172.21.15.6:0/3941921111 conn(0x7f42c40b1650 msgr2=0x7f42c40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:50.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.321+0000 7f42c91cd700 1 -- 172.21.15.6:0/3941921111 shutdown_connections 2023-11-28T13:07:50.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.321+0000 7f42c91cd700 1 -- 172.21.15.6:0/3941921111 wait complete. 2023-11-28T13:07:50.345 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.322+0000 7f42c91cd700 1 Processor -- start 2023-11-28T13:07:50.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.322+0000 7f42c91cd700 1 -- start start 2023-11-28T13:07:50.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c91cd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 0x7f42c415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:50.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c91cd700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4150ec0 0x7f42c415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:50.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c91cd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42c40c5600 0x7f42c40c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:50.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c91cd700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c415e6a0 con 0x7f42ac004850 2023-11-28T13:07:50.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c91cd700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c40c5fb0 con 0x7f42c40c5600 2023-11-28T13:07:50.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c91cd700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c415e330 con 0x7f42c4150ec0 2023-11-28T13:07:50.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 0x7f42c415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:50.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c2d9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 0x7f42c415eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35602/0 (socket says 172.21.15.6:35602) 2023-11-28T13:07:50.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c2d9d700 1 -- 172.21.15.6:0/2313472266 learned_addr learned my addr 172.21.15.6:0/2313472266 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:50.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c359e700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42c40c5600 0x7f42c40c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:50.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c259c700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4150ec0 0x7f42c415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:50.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c2d9d700 1 -- 172.21.15.6:0/2313472266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4150ec0 msgr2=0x7f42c415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:50.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c2d9d700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4150ec0 0x7f42c415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c2d9d700 1 -- 172.21.15.6:0/2313472266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42c40c5600 msgr2=0x7f42c40c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:50.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c2d9d700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42c40c5600 0x7f42c40c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c2d9d700 1 -- 172.21.15.6:0/2313472266 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c40ce870 con 0x7f42ac004850 2023-11-28T13:07:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.323+0000 7f42c2d9d700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 0x7f42c415eac0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f42b4000fb0 tx=0x7f42b40075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.324+0000 7f42ab7fe700 1 -- 172.21.15.6:0/2313472266 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4017070 con 0x7f42ac004850 2023-11-28T13:07:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.324+0000 7f42ab7fe700 1 -- 172.21.15.6:0/2313472266 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b4011580 con 0x7f42ac004850 2023-11-28T13:07:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.324+0000 7f42c91cd700 1 -- 172.21.15.6:0/2313472266 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c40ce590 con 0x7f42ac004850 2023-11-28T13:07:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.324+0000 7f42ab7fe700 1 -- 172.21.15.6:0/2313472266 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4011730 con 0x7f42ac004850 2023-11-28T13:07:50.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.324+0000 7f42c91cd700 1 -- 172.21.15.6:0/2313472266 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c40ca970 con 0x7f42ac004850 2023-11-28T13:07:50.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.325+0000 7f42c91cd700 1 -- 172.21.15.6:0/2313472266 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c415e7e0 con 0x7f42ac004850 2023-11-28T13:07:50.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.325+0000 7f42ab7fe700 1 -- 172.21.15.6:0/2313472266 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f42b4011890 con 0x7f42ac004850 2023-11-28T13:07:50.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.326+0000 7f42ab7fe700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b00682f0 0x7f42b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:50.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.326+0000 7f42ab7fe700 1 -- 172.21.15.6:0/2313472266 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f42b4085550 con 0x7f42ac004850 2023-11-28T13:07:50.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.326+0000 7f42c259c700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b00682f0 0x7f42b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:50.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.326+0000 7f42c259c700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b00682f0 0x7f42b006a7a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f42b80090b0 tx=0x7f42b801f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:50.349 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.330+0000 7f42ab7fe700 1 -- 172.21.15.6:0/2313472266 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f42c415e7e0 con 0x7f42ac004850 2023-11-28T13:07:50.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.524+0000 7f42c91cd700 1 -- 172.21.15.6:0/2313472266 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42c415e7e0 con 0x7f42ac004850 2023-11-28T13:07:50.526 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.525+0000 7f42ab7fe700 1 -- 172.21.15.6:0/2313472266 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+14448 (secure 0 0 0) 0x7f42c415e7e0 con 0x7f42ac004850 2023-11-28T13:07:50.526 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:50.527 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":71,"fsid":"dc643718-e974-4d8a-b829-ff6ced554687","created":"2023-11-28T13:00:03.163304+0000","modified":"2023-11-28T13:07:47.197682+0000","last_up_change":"2023-11-28T13:00:13.931581+0000","last_in_change":"2023-11-28T13:00:05.723768+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T13:00:14.560915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T13:00:14.998729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9217a667-026f-41e4-9f6a-4a9bf892353e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6809","nonce":103502},{"type":"v1","addr":"172.21.15.6:6811","nonce":103502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":103502},{"type":"v1","addr":"172.21.15.6:6816","nonce":103502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":103502},{"type":"v1","addr":"172.21.15.6:6823","nonce":103502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6819","nonce":103502},{"type":"v1","addr":"172.21.15.6:6821","nonce":103502}]},"public_addr":"172.21.15.6:6811/103502","cluster_addr":"172.21.15.6:6816/103502","heartbeat_back_addr":"172.21.15.6:6823/103502","heartbeat_front_addr":"172.21.15.6:6821/103502","state":["exists","up"]},{"osd":1,"uuid":"614e9b07-0417-4fd0-b01b-b4ff14d65052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":103501},{"type":"v1","addr":"172.21.15.6:6801","nonce":103501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":103501},{"type":"v1","addr":"172.21.15.6:6803","nonce":103501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":103501},{"type":"v1","addr":"172.21.15.6:6807","nonce":103501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":103501},{"type":"v1","addr":"172.21.15.6:6805","nonce":103501}]},"public_addr":"172.21.15.6:6801/103501","cluster_addr":"172.21.15.6:6803/103501","heartbeat_back_addr":"172.21.15.6:6807/103501","heartbeat_front_addr":"172.21.15.6:6805/103501","state":["exists","up"]},{"osd":2,"uuid":"dc568cdf-3d3f-4255-b5ef-a07ac57d3ec3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":103512},{"type":"v1","addr":"172.21.15.6:6825","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":103512},{"type":"v1","addr":"172.21.15.6:6827","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":103512},{"type":"v1","addr":"172.21.15.6:6831","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":103512},{"type":"v1","addr":"172.21.15.6:6829","nonce":103512}]},"public_addr":"172.21.15.6:6825/103512","cluster_addr":"172.21.15.6:6827/103512","heartbeat_back_addr":"172.21.15.6:6831/103512","heartbeat_front_addr":"172.21.15.6:6829/103512","state":["exists","up"]},{"osd":3,"uuid":"154eaf7a-f54d-4e18-8e19-3a319d24dfc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":103508},{"type":"v1","addr":"172.21.15.6:6810","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":103508},{"type":"v1","addr":"172.21.15.6:6813","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":103508},{"type":"v1","addr":"172.21.15.6:6820","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":103508},{"type":"v1","addr":"172.21.15.6:6817","nonce":103508}]},"public_addr":"172.21.15.6:6810/103508","cluster_addr":"172.21.15.6:6813/103508","heartbeat_back_addr":"172.21.15.6:6820/103508","heartbeat_front_addr":"172.21.15.6:6817/103508","state":["exists","up"]},{"osd":4,"uuid":"9fb332d4-e6e3-49d9-9172-f1d3884c2d5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":103782},{"type":"v1","addr":"172.21.15.61:6817","nonce":103782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":103782},{"type":"v1","addr":"172.21.15.61:6819","nonce":103782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":103782},{"type":"v1","addr":"172.21.15.61:6823","nonce":103782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":103782},{"type":"v1","addr":"172.21.15.61:6821","nonce":103782}]},"public_addr":"172.21.15.61:6817/103782","cluster_addr":"172.21.15.61:6819/103782","heartbeat_back_addr":"172.21.15.61:6823/103782","heartbeat_front_addr":"172.21.15.61:6821/103782","state":["exists","up"]},{"osd":5,"uuid":"bc0d4e9c-c75d-4959-b71c-66a04ca98515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":103780},{"type":"v1","addr":"172.21.15.61:6809","nonce":103780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":103780},{"type":"v1","addr":"172.21.15.61:6811","nonce":103780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":103780},{"type":"v1","addr":"172.21.15.61:6815","nonce":103780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":103780},{"type":"v1","addr":"172.21.15.61:6813","nonce":103780}]},"public_addr":"172.21.15.61:6809/103780","cluster_addr":"172.21.15.61:6811/103780","heartbeat_back_addr":"172.21.15.61:6815/103780","heartbeat_front_addr":"172.21.15.61:6813/103780","state":["exists","up"]},{"osd":6,"uuid":"264f6e4c-db13-4cb0-a749-d507ab7c900d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":103779},{"type":"v1","addr":"172.21.15.61:6801","nonce":103779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":103779},{"type":"v1","addr":"172.21.15.61:6803","nonce":103779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":103779},{"type":"v1","addr":"172.21.15.61:6807","nonce":103779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":103779},{"type":"v1","addr":"172.21.15.61:6805","nonce":103779}]},"public_addr":"172.21.15.61:6801/103779","cluster_addr":"172.21.15.61:6803/103779","heartbeat_back_addr":"172.21.15.61:6807/103779","heartbeat_front_addr":"172.21.15.61:6805/103779","state":["exists","up"]},{"osd":7,"uuid":"5030167b-9ac0-434e-b096-61c9eaac0940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":103806},{"type":"v1","addr":"172.21.15.61:6825","nonce":103806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":103806},{"type":"v1","addr":"172.21.15.61:6827","nonce":103806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":103806},{"type":"v1","addr":"172.21.15.61:6831","nonce":103806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":103806},{"type":"v1","addr":"172.21.15.61:6829","nonce":103806}]},"public_addr":"172.21.15.61:6825/103806","cluster_addr":"172.21.15.61:6827/103806","heartbeat_back_addr":"172.21.15.61:6831/103806","heartbeat_front_addr":"172.21.15.61:6829/103806","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.986813+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.906014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.894230+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.990196+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.873424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.866047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.617537+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T13:00:10.879298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:6835/3076579711":"2023-11-29T13:07:43.234894+0000","172.21.15.61:6834/3076579711":"2023-11-29T13:07:43.234894+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T13:07:50.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 -- 172.21.15.6:0/2313472266 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b00682f0 msgr2=0x7f42b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:50.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b00682f0 0x7f42b006a7a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f42b80090b0 tx=0x7f42b801f040 comp rx=0 tx=0).stop 2023-11-28T13:07:50.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 -- 172.21.15.6:0/2313472266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 msgr2=0x7f42c415eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:50.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 0x7f42c415eac0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f42b4000fb0 tx=0x7f42b40075f0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 -- 172.21.15.6:0/2313472266 shutdown_connections 2023-11-28T13:07:50.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42c40c5600 0x7f42c40c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42ac004850 0x7f42c415eac0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b00682f0 0x7f42b006a7a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 --2- 172.21.15.6:0/2313472266 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42c4150ec0 0x7f42c415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:50.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 -- 172.21.15.6:0/2313472266 >> 172.21.15.6:0/2313472266 conn(0x7f42c40b1650 msgr2=0x7f42c40cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:50.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 -- 172.21.15.6:0/2313472266 shutdown_connections 2023-11-28T13:07:50.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:50.527+0000 7f42c91cd700 1 -- 172.21.15.6:0/2313472266 wait complete. 2023-11-28T13:07:51.542 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T13:07:51.542 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:51.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.684+0000 7ff8f513d700 1 Processor -- start 2023-11-28T13:07:51.686 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.684+0000 7ff8f513d700 1 -- start start 2023-11-28T13:07:51.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.685+0000 7ff8f513d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff8f0159390 0x7ff8f015d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.688 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.685+0000 7ff8f513d700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f006c3a0 con 0x7ff8f00cebe0 2023-11-28T13:07:51.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.685+0000 7ff8f513d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f015dd50 con 0x7ff8f00cdf80 2023-11-28T13:07:51.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.685+0000 7ff8f513d700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f0159760 con 0x7ff8f0159390 2023-11-28T13:07:51.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.685+0000 7ff8eed9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff8f0159390 0x7ff8f015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.685+0000 7ff8eed9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff8f0159390 0x7ff8f015d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35610/0 (socket says 172.21.15.6:35610) 2023-11-28T13:07:51.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.685+0000 7ff8eed9d700 1 -- 172.21.15.6:0/569852006 learned_addr learned my addr 172.21.15.6:0/569852006 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:51.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.685+0000 7ff8ef59e700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7ff8f00cebe0 0x7ff8f00cac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51166/0 (socket says 172.21.15.6:51166) 2023-11-28T13:07:51.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.686+0000 7ff8eed9d700 1 -- 172.21.15.6:0/569852006 >> v1:172.21.15.61:6789/0 conn(0x7ff8f00cdf80 legacy=0x7ff8f00ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.689 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.686+0000 7ff8eed9d700 1 -- 172.21.15.6:0/569852006 >> v1:172.21.15.6:6789/0 conn(0x7ff8f00cebe0 legacy=0x7ff8f00cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.686+0000 7ff8edd9b700 1 -- 172.21.15.6:0/569852006 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139750375 0 0) 0x7ff8f006c3a0 con 0x7ff8f00cebe0 2023-11-28T13:07:51.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.686+0000 7ff8eed9d700 1 -- 172.21.15.6:0/569852006 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f006c640 con 0x7ff8f0159390 2023-11-28T13:07:51.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.686+0000 7ff8eed9d700 1 --2- 172.21.15.6:0/569852006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8f0159390 0x7ff8f015d810 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00098f0 tx=0x7ff8e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218b6b340a293ebd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:51.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.686+0000 7ff8edd9b700 1 -- 172.21.15.6:0/569852006 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000b8a0 con 0x7ff8f0159390 2023-11-28T13:07:51.690 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.686+0000 7ff8edd9b700 1 -- 172.21.15.6:0/569852006 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e000ba00 con 0x7ff8f0159390 2023-11-28T13:07:51.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.686+0000 7ff8edd9b700 1 -- 172.21.15.6:0/569852006 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000bd90 con 0x7ff8f0159390 2023-11-28T13:07:51.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.687+0000 7ff8f513d700 1 -- 172.21.15.6:0/569852006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8f0159390 msgr2=0x7ff8f015d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.687+0000 7ff8f513d700 1 --2- 172.21.15.6:0/569852006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8f0159390 0x7ff8f015d810 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00098f0 tx=0x7ff8e0009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:51.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.687+0000 7ff8f513d700 1 -- 172.21.15.6:0/569852006 shutdown_connections 2023-11-28T13:07:51.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.687+0000 7ff8f513d700 1 --2- 172.21.15.6:0/569852006 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8f0159390 0x7ff8f015d810 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.687+0000 7ff8f513d700 1 -- 172.21.15.6:0/569852006 >> 172.21.15.6:0/569852006 conn(0x7ff8f006dae0 msgr2=0x7ff8f006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:51.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.687+0000 7ff8f513d700 1 -- 172.21.15.6:0/569852006 shutdown_connections 2023-11-28T13:07:51.691 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.687+0000 7ff8f513d700 1 -- 172.21.15.6:0/569852006 wait complete. 2023-11-28T13:07:51.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.688+0000 7ff8f513d700 1 Processor -- start 2023-11-28T13:07:51.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.688+0000 7ff8f513d700 1 -- start start 2023-11-28T13:07:51.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8f513d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8f00cdf80 0x7ff8f00c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8f513d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8f00cebe0 0x7ff8f00b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8f513d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8f0159390 0x7ff8f00b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.695 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8f513d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f01559d0 con 0x7ff8f00cdf80 2023-11-28T13:07:51.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8f513d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f015c800 con 0x7ff8f0159390 2023-11-28T13:07:51.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8f513d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f00c7a90 con 0x7ff8f00cebe0 2023-11-28T13:07:51.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8eed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8f00cdf80 0x7ff8f00c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8eed9d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8f00cdf80 0x7ff8f00c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35616/0 (socket says 172.21.15.6:35616) 2023-11-28T13:07:51.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8eed9d700 1 -- 172.21.15.6:0/88181662 learned_addr learned my addr 172.21.15.6:0/88181662 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:51.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8ef59e700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8f0159390 0x7ff8f00b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8eed9d700 1 -- 172.21.15.6:0/88181662 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8f00cebe0 msgr2=0x7ff8f00b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.696 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8eed9d700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7ff8f00cebe0 0x7ff8f00b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8eed9d700 1 -- 172.21.15.6:0/88181662 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8f0159390 msgr2=0x7ff8f00b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.689+0000 7ff8eed9d700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8f0159390 0x7ff8f00b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.690+0000 7ff8eed9d700 1 -- 172.21.15.6:0/88181662 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e0009550 con 0x7ff8f00cdf80 2023-11-28T13:07:51.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.690+0000 7ff8ef59e700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff8f0159390 0x7ff8f00b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:51.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.690+0000 7ff8eed9d700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8f00cdf80 0x7ff8f00c5940 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000b0e0 tx=0x7ff8e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:51.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.690+0000 7ff8e77fe700 1 -- 172.21.15.6:0/88181662 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0011a70 con 0x7ff8f00cdf80 2023-11-28T13:07:51.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.690+0000 7ff8e77fe700 1 -- 172.21.15.6:0/88181662 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e001f070 con 0x7ff8f00cdf80 2023-11-28T13:07:51.697 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.690+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f006cc00 con 0x7ff8f00cdf80 2023-11-28T13:07:51.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.690+0000 7ff8e77fe700 1 -- 172.21.15.6:0/88181662 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0011dd0 con 0x7ff8f00cdf80 2023-11-28T13:07:51.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.690+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f006c920 con 0x7ff8f00cdf80 2023-11-28T13:07:51.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.691+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8f006cee0 con 0x7ff8f00cdf80 2023-11-28T13:07:51.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.691+0000 7ff8e77fe700 1 -- 172.21.15.6:0/88181662 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff8e000f050 con 0x7ff8f00cdf80 2023-11-28T13:07:51.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.692+0000 7ff8e77fe700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8d00682f0 0x7ff8d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.692+0000 7ff8e77fe700 1 -- 172.21.15.6:0/88181662 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7ff8e0021070 con 0x7ff8f00cdf80 2023-11-28T13:07:51.698 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.692+0000 7ff8ee59c700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8d00682f0 0x7ff8d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.692+0000 7ff8e77fe700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x7ff8d006eb90 0x7ff8d0070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.692+0000 7ff8e77fe700 1 -- 172.21.15.6:0/88181662 --> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8d00716b0 con 0x7ff8d006eb90 2023-11-28T13:07:51.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.692+0000 7ff8e77fe700 1 -- 172.21.15.6:0/88181662 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8e0087e30 con 0x7ff8f00cdf80 2023-11-28T13:07:51.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.692+0000 7ff8ef59e700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x7ff8d006eb90 0x7ff8d0070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.692+0000 7ff8ee59c700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8d00682f0 0x7ff8d006a7a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8000f60 tx=0x7ff8d8007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:51.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.692+0000 7ff8ef59e700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x7ff8d006eb90 0x7ff8d0070fe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:51.699 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.693+0000 7ff8e77fe700 1 -- 172.21.15.6:0/88181662 <== osd.0 v2:172.21.15.6:6809/103502 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff8d00716b0 con 0x7ff8d006eb90 2023-11-28T13:07:51.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.708+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 --> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff8f006e740 con 0x7ff8d006eb90 2023-11-28T13:07:51.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8e77fe700 1 -- 172.21.15.6:0/88181662 <== osd.0 v2:172.21.15.6:6809/103502 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff8f006e740 con 0x7ff8d006eb90 2023-11-28T13:07:51.710 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:07:51.710 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:51.711 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:07:51.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x7ff8d006eb90 msgr2=0x7ff8d0070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8f513d700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:6809/103502,v1:172.21.15.6:6811/103502] conn(0x7ff8d006eb90 0x7ff8d0070fe0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8d00682f0 msgr2=0x7ff8d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8f513d700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7ff8d00682f0 0x7ff8d006a7a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8000f60 tx=0x7ff8d8007a40 comp rx=0 tx=0).stop 2023-11-28T13:07:51.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8f00cdf80 msgr2=0x7ff8f00c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8f513d700 1 --2- 172.21.15.6:0/88181662 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8f00cdf80 0x7ff8f00c5940 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000b0e0 tx=0x7ff8e0004750 comp rx=0 tx=0).stop 2023-11-28T13:07:51.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8ef59e700 1 -- 172.21.15.6:0/88181662 reap_dead start 2023-11-28T13:07:51.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 shutdown_connections 2023-11-28T13:07:51.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 >> 172.21.15.6:0/88181662 conn(0x7ff8f006dae0 msgr2=0x7ff8f015d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:51.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 shutdown_connections 2023-11-28T13:07:51.713 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.709+0000 7ff8f513d700 1 -- 172.21.15.6:0/88181662 wait complete. 2023-11-28T13:07:51.722 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T13:07:51.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.846+0000 7f4a3ddfc700 1 Processor -- start 2023-11-28T13:07:51.848 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.846+0000 7f4a3ddfc700 1 -- start start 2023-11-28T13:07:51.850 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.847+0000 7f4a3ddfc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 0x7f4a380cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.847+0000 7f4a3ddfc700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f4a38161090 0x7f4a380ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.847+0000 7f4a3ddfc700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4a380caeb0 0x7f4a380cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.847+0000 7f4a3ddfc700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380ce340 con 0x7f4a380caeb0 2023-11-28T13:07:51.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.847+0000 7f4a3ddfc700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380cebe0 con 0x7f4a38161090 2023-11-28T13:07:51.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.847+0000 7f4a3ddfc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380cb3f0 con 0x7f4a3815c990 2023-11-28T13:07:51.851 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.847+0000 7f4a37fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4a380caeb0 0x7f4a380cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.847+0000 7f4a37fff700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4a380caeb0 0x7f4a380cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35626/0 (socket says 172.21.15.6:35626) 2023-11-28T13:07:51.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.847+0000 7f4a37fff700 1 -- 172.21.15.6:0/3666025826 learned_addr learned my addr 172.21.15.6:0/3666025826 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:51.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a36ffd700 1 --2- 172.21.15.6:0/3666025826 >> v2:172.21.15.61:3300/0 conn(0x7f4a38161090 0x7f4a380ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a37fff700 1 -- 172.21.15.6:0/3666025826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 msgr2=0x7f4a380cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:51.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a377fe700 1 --2- 172.21.15.6:0/3666025826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 0x7f4a380cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a37fff700 1 --2- 172.21.15.6:0/3666025826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 0x7f4a380cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a37fff700 1 -- 172.21.15.6:0/3666025826 >> v2:172.21.15.61:3300/0 conn(0x7f4a38161090 msgr2=0x7f4a380ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:51.852 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a37fff700 1 --2- 172.21.15.6:0/3666025826 >> v2:172.21.15.61:3300/0 conn(0x7f4a38161090 0x7f4a380ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a37fff700 1 -- 172.21.15.6:0/3666025826 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380ce590 con 0x7f4a380caeb0 2023-11-28T13:07:51.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a377fe700 1 --2- 172.21.15.6:0/3666025826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 0x7f4a380cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:51.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a37fff700 1 --2- 172.21.15.6:0/3666025826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a380caeb0 0x7f4a380cc310 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4a3806f850 tx=0x7f4a3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839a8fc2a2406842 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:51.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a367fc700 1 -- 172.21.15.6:0/3666025826 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000e070 con 0x7f4a380caeb0 2023-11-28T13:07:51.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a367fc700 1 -- 172.21.15.6:0/3666025826 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000b9c0 con 0x7f4a380caeb0 2023-11-28T13:07:51.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a367fc700 1 -- 172.21.15.6:0/3666025826 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000bd70 con 0x7f4a380caeb0 2023-11-28T13:07:51.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3666025826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a380caeb0 msgr2=0x7f4a380cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.853 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.848+0000 7f4a3ddfc700 1 --2- 172.21.15.6:0/3666025826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a380caeb0 0x7f4a380cc310 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4a3806f850 tx=0x7f4a3000b330 comp rx=0 tx=0).stop 2023-11-28T13:07:51.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.849+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3666025826 shutdown_connections 2023-11-28T13:07:51.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.849+0000 7f4a3ddfc700 1 --2- 172.21.15.6:0/3666025826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a380caeb0 0x7f4a380cc310 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.849+0000 7f4a3ddfc700 1 --2- 172.21.15.6:0/3666025826 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 0x7f4a380cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.849+0000 7f4a3ddfc700 1 --2- 172.21.15.6:0/3666025826 >> v2:172.21.15.61:3300/0 conn(0x7f4a38161090 0x7f4a380ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.849+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3666025826 >> 172.21.15.6:0/3666025826 conn(0x7f4a380b1650 msgr2=0x7f4a380b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:51.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.849+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3666025826 shutdown_connections 2023-11-28T13:07:51.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.849+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3666025826 wait complete. 2023-11-28T13:07:51.854 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.850+0000 7f4a3ddfc700 1 Processor -- start 2023-11-28T13:07:51.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.850+0000 7f4a3ddfc700 1 -- start start 2023-11-28T13:07:51.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a3ddfc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a380caeb0 0x7f4a3815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a3ddfc700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 0x7f4a3815d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a3ddfc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a38161090 0x7f4a3815d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a3ddfc700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38147e50 con 0x7f4a38161090 2023-11-28T13:07:51.857 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a3ddfc700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3815e1b0 con 0x7f4a380caeb0 2023-11-28T13:07:51.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a3ddfc700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380b2950 con 0x7f4a3815c990 2023-11-28T13:07:51.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a38161090 0x7f4a3815d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a38161090 0x7f4a3815d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35636/0 (socket says 172.21.15.6:35636) 2023-11-28T13:07:51.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a37fff700 1 -- 172.21.15.6:0/3022844231 learned_addr learned my addr 172.21.15.6:0/3022844231 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:51.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a377fe700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a380caeb0 0x7f4a3815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a36ffd700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 0x7f4a3815d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.858 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a37fff700 1 -- 172.21.15.6:0/3022844231 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 msgr2=0x7f4a3815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a37fff700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 0x7f4a3815d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a37fff700 1 -- 172.21.15.6:0/3022844231 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a380caeb0 msgr2=0x7f4a3815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:51.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a37fff700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a380caeb0 0x7f4a3815e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:51.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a37fff700 1 -- 172.21.15.6:0/3022844231 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30009b30 con 0x7f4a38161090 2023-11-28T13:07:51.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a37fff700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a38161090 0x7f4a3815d7f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4a3806eac0 tx=0x7f4a30004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:51.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.851+0000 7f4a2ffff700 1 -- 172.21.15.6:0/3022844231 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30004880 con 0x7f4a38161090 2023-11-28T13:07:51.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.852+0000 7f4a2ffff700 1 -- 172.21.15.6:0/3022844231 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a300099b0 con 0x7f4a38161090 2023-11-28T13:07:51.859 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.852+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3022844231 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a380cf5b0 con 0x7f4a38161090 2023-11-28T13:07:51.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.852+0000 7f4a2ffff700 1 -- 172.21.15.6:0/3022844231 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3001b670 con 0x7f4a38161090 2023-11-28T13:07:51.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.852+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3022844231 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a380ce870 con 0x7f4a38161090 2023-11-28T13:07:51.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.852+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3022844231 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a380cf740 con 0x7f4a38161090 2023-11-28T13:07:51.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.853+0000 7f4a2ffff700 1 -- 172.21.15.6:0/3022844231 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4a30021030 con 0x7f4a38161090 2023-11-28T13:07:51.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.853+0000 7f4a2ffff700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4a180682f0 0x7f4a1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:51.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.854+0000 7f4a2ffff700 1 -- 172.21.15.6:0/3022844231 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f4a30086f40 con 0x7f4a38161090 2023-11-28T13:07:51.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.854+0000 7f4a377fe700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4a180682f0 0x7f4a1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:51.860 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.854+0000 7f4a377fe700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4a180682f0 0x7f4a1806a7a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006ba0 tx=0x7f4a28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:51.861 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:51.858+0000 7f4a2ffff700 1 -- 172.21.15.6:0/3022844231 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4a30053890 con 0x7f4a38161090 2023-11-28T13:07:52.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.053+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3022844231 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a380b2f70 con 0x7f4a180682f0 2023-11-28T13:07:52.055 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.054+0000 7f4a2ffff700 1 -- 172.21.15.6:0/3022844231 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4a380b2f70 con 0x7f4a180682f0 2023-11-28T13:07:52.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3022844231 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4a180682f0 msgr2=0x7f4a1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.057 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4a180682f0 0x7f4a1806a7a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006ba0 tx=0x7f4a28008040 comp rx=0 tx=0).stop 2023-11-28T13:07:52.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3022844231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a38161090 msgr2=0x7f4a3815d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a38161090 0x7f4a3815d7f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4a3806eac0 tx=0x7f4a30004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3022844231 shutdown_connections 2023-11-28T13:07:52.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4a38161090 0x7f4a3815d7f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4a380caeb0 0x7f4a3815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f4a180682f0 0x7f4a1806a7a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 --2- 172.21.15.6:0/3022844231 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f4a3815c990 0x7f4a3815d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.058 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3022844231 >> 172.21.15.6:0/3022844231 conn(0x7f4a380b1650 msgr2=0x7f4a3815c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:52.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3022844231 shutdown_connections 2023-11-28T13:07:52.059 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.056+0000 7f4a3ddfc700 1 -- 172.21.15.6:0/3022844231 wait complete. 2023-11-28T13:07:52.059 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T13:07:52.069 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T13:07:52.070 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:52.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.198+0000 7fb51ef69700 1 Processor -- start 2023-11-28T13:07:52.199 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.198+0000 7fb51ef69700 1 -- start start 2023-11-28T13:07:52.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.198+0000 7fb51ef69700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb518158ab0 0x7fb5180c5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.198+0000 7fb51ef69700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb518071180 0x7fb5180715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.201 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.198+0000 7fb51ef69700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180cdaf0 con 0x7fb5180ccf60 2023-11-28T13:07:52.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.198+0000 7fb51ef69700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5180c57d0 con 0x7fb518158ab0 2023-11-28T13:07:52.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.198+0000 7fb51ef69700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb518071b30 con 0x7fb518071180 2023-11-28T13:07:52.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb51cd05700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb518158ab0 0x7fb5180c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb51cd05700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb518158ab0 0x7fb5180c5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35640/0 (socket says 172.21.15.6:35640) 2023-11-28T13:07:52.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb51cd05700 1 -- 172.21.15.6:0/713134435 learned_addr learned my addr 172.21.15.6:0/713134435 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:52.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb517fff700 1 --2- 172.21.15.6:0/713134435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb518071180 0x7fb5180715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb51cd05700 1 -- 172.21.15.6:0/713134435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb518071180 msgr2=0x7fb5180715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:52.202 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb51cd05700 1 --2- 172.21.15.6:0/713134435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb518071180 0x7fb5180715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb51cd05700 1 -- 172.21.15.6:0/713134435 >> v1:172.21.15.61:6789/0 conn(0x7fb5180ccf60 legacy=0x7fb5180cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb51cd05700 1 -- 172.21.15.6:0/713134435 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180c41f0 con 0x7fb518158ab0 2023-11-28T13:07:52.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb51cd05700 1 --2- 172.21.15.6:0/713134435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb518158ab0 0x7fb5180c5290 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00bbc0 tx=0x7fb50c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92b3b0a85b931d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb5177fe700 1 -- 172.21.15.6:0/713134435 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00dd50 con 0x7fb518158ab0 2023-11-28T13:07:52.203 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb5177fe700 1 -- 172.21.15.6:0/713134435 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c010470 con 0x7fb518158ab0 2023-11-28T13:07:52.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.199+0000 7fb5177fe700 1 -- 172.21.15.6:0/713134435 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c010800 con 0x7fb518158ab0 2023-11-28T13:07:52.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.200+0000 7fb51ef69700 1 -- 172.21.15.6:0/713134435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb518158ab0 msgr2=0x7fb5180c5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.200+0000 7fb51ef69700 1 --2- 172.21.15.6:0/713134435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb518158ab0 0x7fb5180c5290 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00bbc0 tx=0x7fb50c00bed0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.200+0000 7fb51ef69700 1 -- 172.21.15.6:0/713134435 shutdown_connections 2023-11-28T13:07:52.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.200+0000 7fb51ef69700 1 --2- 172.21.15.6:0/713134435 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb518158ab0 0x7fb5180c5290 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.204 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.200+0000 7fb51ef69700 1 --2- 172.21.15.6:0/713134435 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb518071180 0x7fb5180715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.200+0000 7fb51ef69700 1 -- 172.21.15.6:0/713134435 >> 172.21.15.6:0/713134435 conn(0x7fb5180ce9a0 msgr2=0x7fb5180ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:52.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.200+0000 7fb51ef69700 1 -- 172.21.15.6:0/713134435 shutdown_connections 2023-11-28T13:07:52.205 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.200+0000 7fb51ef69700 1 -- 172.21.15.6:0/713134435 wait complete. 2023-11-28T13:07:52.210 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.201+0000 7fb51ef69700 1 Processor -- start 2023-11-28T13:07:52.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.201+0000 7fb51ef69700 1 -- start start 2023-11-28T13:07:52.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51ef69700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb518071180 0x7fb5181f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51ef69700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5180ccf60 0x7fb5181497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51ef69700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb518158ab0 0x7fb5180c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51ef69700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51815bee0 con 0x7fb518158ab0 2023-11-28T13:07:52.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51ef69700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51814a230 con 0x7fb518071180 2023-11-28T13:07:52.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51ef69700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb518149cf0 con 0x7fb5180ccf60 2023-11-28T13:07:52.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51d506700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb518158ab0 0x7fb5180c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51d506700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb518158ab0 0x7fb5180c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35650/0 (socket says 172.21.15.6:35650) 2023-11-28T13:07:52.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51d506700 1 -- 172.21.15.6:0/2962876400 learned_addr learned my addr 172.21.15.6:0/2962876400 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:52.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51cd05700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb518071180 0x7fb5181f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.213 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb517fff700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5180ccf60 0x7fb5181497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51d506700 1 -- 172.21.15.6:0/2962876400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5180ccf60 msgr2=0x7fb5181497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51d506700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb5180ccf60 0x7fb5181497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51d506700 1 -- 172.21.15.6:0/2962876400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb518071180 msgr2=0x7fb5181f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51d506700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb518071180 0x7fb5181f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.202+0000 7fb51d506700 1 -- 172.21.15.6:0/2962876400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c00b7f0 con 0x7fb518158ab0 2023-11-28T13:07:52.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.203+0000 7fb51d506700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb518158ab0 0x7fb5180c6db0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb51000cc80 tx=0x7fb51000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.203+0000 7fb5157fa700 1 -- 172.21.15.6:0/2962876400 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510003c00 con 0x7fb518158ab0 2023-11-28T13:07:52.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.203+0000 7fb5157fa700 1 -- 172.21.15.6:0/2962876400 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb510003d60 con 0x7fb518158ab0 2023-11-28T13:07:52.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.203+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5180c47b0 con 0x7fb518158ab0 2023-11-28T13:07:52.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.203+0000 7fb5157fa700 1 -- 172.21.15.6:0/2962876400 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51000aaa0 con 0x7fb518158ab0 2023-11-28T13:07:52.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.203+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5180c44d0 con 0x7fb518158ab0 2023-11-28T13:07:52.215 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.203+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb5180c4a90 con 0x7fb518158ab0 2023-11-28T13:07:52.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.204+0000 7fb5157fa700 1 -- 172.21.15.6:0/2962876400 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb510010050 con 0x7fb518158ab0 2023-11-28T13:07:52.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.205+0000 7fb5157fa700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4fc0682f0 0x7fb4fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.205+0000 7fb5157fa700 1 -- 172.21.15.6:0/2962876400 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb510086350 con 0x7fb518158ab0 2023-11-28T13:07:52.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.205+0000 7fb51cd05700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4fc0682f0 0x7fb4fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.205+0000 7fb5157fa700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] conn(0x7fb4fc06eb90 0x7fb4fc070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.205+0000 7fb5157fa700 1 -- 172.21.15.6:0/2962876400 --> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4fc0716b0 con 0x7fb4fc06eb90 2023-11-28T13:07:52.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.205+0000 7fb5157fa700 1 -- 172.21.15.6:0/2962876400 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb510086790 con 0x7fb518158ab0 2023-11-28T13:07:52.216 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.205+0000 7fb517fff700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] conn(0x7fb4fc06eb90 0x7fb4fc070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.205+0000 7fb51cd05700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4fc0682f0 0x7fb4fc06a7a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00bf10 tx=0x7fb50c01c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.205+0000 7fb517fff700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] conn(0x7fb4fc06eb90 0x7fb4fc070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.206+0000 7fb5157fa700 1 -- 172.21.15.6:0/2962876400 <== osd.1 v2:172.21.15.6:6800/103501 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb4fc0716b0 con 0x7fb4fc06eb90 2023-11-28T13:07:52.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.220+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 --> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb5180739f0 con 0x7fb4fc06eb90 2023-11-28T13:07:52.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb5157fa700 1 -- 172.21.15.6:0/2962876400 <== osd.1 v2:172.21.15.6:6800/103501 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb5180739f0 con 0x7fb4fc06eb90 2023-11-28T13:07:52.222 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:07:52.222 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:52.223 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:07:52.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] conn(0x7fb4fc06eb90 msgr2=0x7fb4fc070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51ef69700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:6800/103501,v1:172.21.15.6:6801/103501] conn(0x7fb4fc06eb90 0x7fb4fc070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4fc0682f0 msgr2=0x7fb4fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51ef69700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb4fc0682f0 0x7fb4fc06a7a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00bf10 tx=0x7fb50c01c040 comp rx=0 tx=0).stop 2023-11-28T13:07:52.224 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb518158ab0 msgr2=0x7fb5180c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51ef69700 1 --2- 172.21.15.6:0/2962876400 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb518158ab0 0x7fb5180c6db0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb51000cc80 tx=0x7fb51000cf90 comp rx=0 tx=0).stop 2023-11-28T13:07:52.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51d506700 1 -- 172.21.15.6:0/2962876400 reap_dead start 2023-11-28T13:07:52.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 shutdown_connections 2023-11-28T13:07:52.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 >> 172.21.15.6:0/2962876400 conn(0x7fb5180ce9a0 msgr2=0x7fb51806d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:52.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 shutdown_connections 2023-11-28T13:07:52.225 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.221+0000 7fb51ef69700 1 -- 172.21.15.6:0/2962876400 wait complete. 2023-11-28T13:07:52.234 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T13:07:52.359 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.358+0000 7fcda8acf700 1 Processor -- start 2023-11-28T13:07:52.360 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.358+0000 7fcda8acf700 1 -- start start 2023-11-28T13:07:52.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.359+0000 7fcda8acf700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcda40cc310 0x7fcda40cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.359+0000 7fcda8acf700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda40ce340 con 0x7fcda415c990 2023-11-28T13:07:52.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.359+0000 7fcda8acf700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda40cebe0 con 0x7fcda4161090 2023-11-28T13:07:52.362 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.359+0000 7fcda8acf700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda40cf780 con 0x7fcda40cc310 2023-11-28T13:07:52.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.359+0000 7fcda2d9d700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcda40cc310 0x7fcda40cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.359+0000 7fcda259c700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fcda415c990 0x7fcda40cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51208/0 (socket says 172.21.15.6:51208) 2023-11-28T13:07:52.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.359+0000 7fcda259c700 1 -- 172.21.15.6:0/882660309 learned_addr learned my addr 172.21.15.6:0/882660309 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:52.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.360+0000 7fcda2d9d700 1 -- 172.21.15.6:0/882660309 >> v1:172.21.15.61:6789/0 conn(0x7fcda4161090 legacy=0x7fcda40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.360+0000 7fcda2d9d700 1 -- 172.21.15.6:0/882660309 >> v1:172.21.15.6:6789/0 conn(0x7fcda415c990 legacy=0x7fcda40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.360+0000 7fcda159a700 1 -- 172.21.15.6:0/882660309 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079415820 0 0) 0x7fcda40ce340 con 0x7fcda415c990 2023-11-28T13:07:52.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.360+0000 7fcda2d9d700 1 -- 172.21.15.6:0/882660309 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda40cccb0 con 0x7fcda40cc310 2023-11-28T13:07:52.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.360+0000 7fcda2d9d700 1 --2- 172.21.15.6:0/882660309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda40cc310 0x7fcda40cc770 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009d90 tx=0x7fcd9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0ab54c95e24175 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.360+0000 7fcda159a700 1 -- 172.21.15.6:0/882660309 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c00e070 con 0x7fcda40cc310 2023-11-28T13:07:52.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.360+0000 7fcda159a700 1 -- 172.21.15.6:0/882660309 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c00bbd0 con 0x7fcda40cc310 2023-11-28T13:07:52.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.360+0000 7fcda159a700 1 -- 172.21.15.6:0/882660309 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c012430 con 0x7fcda40cc310 2023-11-28T13:07:52.365 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.361+0000 7fcda8acf700 1 -- 172.21.15.6:0/882660309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda40cc310 msgr2=0x7fcda40cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.361+0000 7fcda8acf700 1 --2- 172.21.15.6:0/882660309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda40cc310 0x7fcda40cc770 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009d90 tx=0x7fcd9c00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:52.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.361+0000 7fcda8acf700 1 -- 172.21.15.6:0/882660309 shutdown_connections 2023-11-28T13:07:52.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.361+0000 7fcda8acf700 1 --2- 172.21.15.6:0/882660309 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda40cc310 0x7fcda40cc770 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.366 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.361+0000 7fcda8acf700 1 -- 172.21.15.6:0/882660309 >> 172.21.15.6:0/882660309 conn(0x7fcda40b1650 msgr2=0x7fcda40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:52.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.361+0000 7fcda8acf700 1 -- 172.21.15.6:0/882660309 shutdown_connections 2023-11-28T13:07:52.367 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.361+0000 7fcda8acf700 1 -- 172.21.15.6:0/882660309 wait complete. 2023-11-28T13:07:52.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.362+0000 7fcda8acf700 1 Processor -- start 2023-11-28T13:07:52.370 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.362+0000 7fcda8acf700 1 -- start start 2023-11-28T13:07:52.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda8acf700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcda40cc310 0x7fcda40c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda8acf700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda415c990 0x7fcda40ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda8acf700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcda4161090 0x7fcda40ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.371 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda8acf700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda40c9920 con 0x7fcda415c990 2023-11-28T13:07:52.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda8acf700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda40b4430 con 0x7fcda4161090 2023-11-28T13:07:52.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda8acf700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda40b3b40 con 0x7fcda40cc310 2023-11-28T13:07:52.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda415c990 0x7fcda40ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.372 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda1d9b700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda415c990 0x7fcda40ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35664/0 (socket says 172.21.15.6:35664) 2023-11-28T13:07:52.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda1d9b700 1 -- 172.21.15.6:0/3419305242 learned_addr learned my addr 172.21.15.6:0/3419305242 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:52.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda2d9d700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcda4161090 0x7fcda40ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.373 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda259c700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcda40cc310 0x7fcda40c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.374 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda1d9b700 1 -- 172.21.15.6:0/3419305242 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcda40cc310 msgr2=0x7fcda40c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda1d9b700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcda40cc310 0x7fcda40c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda1d9b700 1 -- 172.21.15.6:0/3419305242 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcda4161090 msgr2=0x7fcda40ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda1d9b700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcda4161090 0x7fcda40ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda1d9b700 1 -- 172.21.15.6:0/3419305242 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c009a70 con 0x7fcda415c990 2023-11-28T13:07:52.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.363+0000 7fcda1d9b700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda415c990 0x7fcda40ae060 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800c450 tx=0x7fcd9800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.364+0000 7fcd92ffd700 1 -- 172.21.15.6:0/3419305242 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd980052d0 con 0x7fcda415c990 2023-11-28T13:07:52.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.364+0000 7fcd92ffd700 1 -- 172.21.15.6:0/3419305242 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd98005430 con 0x7fcda415c990 2023-11-28T13:07:52.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.364+0000 7fcda8acf700 1 -- 172.21.15.6:0/3419305242 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda40ce590 con 0x7fcda415c990 2023-11-28T13:07:52.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.364+0000 7fcd92ffd700 1 -- 172.21.15.6:0/3419305242 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd980038c0 con 0x7fcda415c990 2023-11-28T13:07:52.376 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.364+0000 7fcda8acf700 1 -- 172.21.15.6:0/3419305242 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda40cb3f0 con 0x7fcda415c990 2023-11-28T13:07:52.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.365+0000 7fcda8acf700 1 -- 172.21.15.6:0/3419305242 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda40ce870 con 0x7fcda415c990 2023-11-28T13:07:52.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.365+0000 7fcd92ffd700 1 -- 172.21.15.6:0/3419305242 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcd98019030 con 0x7fcda415c990 2023-11-28T13:07:52.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.365+0000 7fcd92ffd700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcd840682f0 0x7fcd8406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.366+0000 7fcd92ffd700 1 -- 172.21.15.6:0/3419305242 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fcd98084cb0 con 0x7fcda415c990 2023-11-28T13:07:52.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.366+0000 7fcda259c700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcd840682f0 0x7fcd8406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.366+0000 7fcda259c700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcd840682f0 0x7fcd8406a7a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcd94007b10 tx=0x7fcd94007a60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.378 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.370+0000 7fcd92ffd700 1 -- 172.21.15.6:0/3419305242 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcd980516d0 con 0x7fcda415c990 2023-11-28T13:07:52.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.561+0000 7fcda8acf700 1 -- 172.21.15.6:0/3419305242 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcda40675a0 con 0x7fcd840682f0 2023-11-28T13:07:52.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.561+0000 7fcd92ffd700 1 -- 172.21.15.6:0/3419305242 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcda40675a0 con 0x7fcd840682f0 2023-11-28T13:07:52.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.563+0000 7fcda8acf700 1 -- 172.21.15.6:0/3419305242 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcd840682f0 msgr2=0x7fcd8406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.563+0000 7fcda8acf700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcd840682f0 0x7fcd8406a7a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcd94007b10 tx=0x7fcd94007a60 comp rx=0 tx=0).stop 2023-11-28T13:07:52.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.563+0000 7fcda8acf700 1 -- 172.21.15.6:0/3419305242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda415c990 msgr2=0x7fcda40ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.563+0000 7fcda8acf700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda415c990 0x7fcda40ae060 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800c450 tx=0x7fcd9800c760 comp rx=0 tx=0).stop 2023-11-28T13:07:52.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.563+0000 7fcda8acf700 1 -- 172.21.15.6:0/3419305242 shutdown_connections 2023-11-28T13:07:52.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.564+0000 7fcda8acf700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcda4161090 0x7fcda40ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.564+0000 7fcda8acf700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fcd840682f0 0x7fcd8406a7a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.564+0000 7fcda8acf700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fcda40cc310 0x7fcda40c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.564+0000 7fcda8acf700 1 --2- 172.21.15.6:0/3419305242 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcda415c990 0x7fcda40ae060 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.564+0000 7fcda8acf700 1 -- 172.21.15.6:0/3419305242 >> 172.21.15.6:0/3419305242 conn(0x7fcda40b1650 msgr2=0x7fcda415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:52.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.564+0000 7fcda8acf700 1 -- 172.21.15.6:0/3419305242 shutdown_connections 2023-11-28T13:07:52.567 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.564+0000 7fcda8acf700 1 -- 172.21.15.6:0/3419305242 wait complete. 2023-11-28T13:07:52.567 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T13:07:52.577 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T13:07:52.577 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:52.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.704+0000 7f7d6c00e700 1 Processor -- start 2023-11-28T13:07:52.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.705+0000 7f7d6c00e700 1 -- start start 2023-11-28T13:07:52.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.705+0000 7f7d6c00e700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7d6415c990 0x7f7d640cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.708 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.705+0000 7f7d6c00e700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d640ce340 con 0x7f7d64161090 2023-11-28T13:07:52.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.705+0000 7f7d6c00e700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d640cebe0 con 0x7f7d640cc310 2023-11-28T13:07:52.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.705+0000 7f7d6c00e700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d640cb3f0 con 0x7f7d6415c990 2023-11-28T13:07:52.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.705+0000 7f7d69daa700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7d6415c990 0x7f7d640cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d69daa700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7d6415c990 0x7f7d640cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35670/0 (socket says 172.21.15.6:35670) 2023-11-28T13:07:52.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d695a9700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f7d64161090 0x7f7d640ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51222/0 (socket says 172.21.15.6:51222) 2023-11-28T13:07:52.709 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d69daa700 1 -- 172.21.15.6:0/2310230668 learned_addr learned my addr 172.21.15.6:0/2310230668 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:52.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d69daa700 1 -- 172.21.15.6:0/2310230668 >> v1:172.21.15.61:6789/0 conn(0x7f7d640cc310 legacy=0x7f7d640cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d69daa700 1 -- 172.21.15.6:0/2310230668 >> v1:172.21.15.6:6789/0 conn(0x7f7d64161090 legacy=0x7f7d640ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d69daa700 1 -- 172.21.15.6:0/2310230668 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d640ce590 con 0x7f7d6415c990 2023-11-28T13:07:52.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d68da8700 1 -- 172.21.15.6:0/2310230668 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762376382 0 0) 0x7f7d640ce340 con 0x7f7d64161090 2023-11-28T13:07:52.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d69daa700 1 --2- 172.21.15.6:0/2310230668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d6415c990 0x7f7d640cf070 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7d540098f0 tx=0x7f7d54009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962aea135e328d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d68da8700 1 -- 172.21.15.6:0/2310230668 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5400b8a0 con 0x7f7d6415c990 2023-11-28T13:07:52.710 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d68da8700 1 -- 172.21.15.6:0/2310230668 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5400ba00 con 0x7f7d6415c990 2023-11-28T13:07:52.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.706+0000 7f7d68da8700 1 -- 172.21.15.6:0/2310230668 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5400bd90 con 0x7f7d6415c990 2023-11-28T13:07:52.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.707+0000 7f7d6c00e700 1 -- 172.21.15.6:0/2310230668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d6415c990 msgr2=0x7f7d640cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.707+0000 7f7d6c00e700 1 --2- 172.21.15.6:0/2310230668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d6415c990 0x7f7d640cf070 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7d540098f0 tx=0x7f7d54009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:52.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.707+0000 7f7d6c00e700 1 -- 172.21.15.6:0/2310230668 shutdown_connections 2023-11-28T13:07:52.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.707+0000 7f7d6c00e700 1 --2- 172.21.15.6:0/2310230668 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d6415c990 0x7f7d640cf070 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.707+0000 7f7d6c00e700 1 -- 172.21.15.6:0/2310230668 >> 172.21.15.6:0/2310230668 conn(0x7f7d640b1650 msgr2=0x7f7d640b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:52.711 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.707+0000 7f7d6c00e700 1 -- 172.21.15.6:0/2310230668 shutdown_connections 2023-11-28T13:07:52.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.707+0000 7f7d6c00e700 1 -- 172.21.15.6:0/2310230668 wait complete. 2023-11-28T13:07:52.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.708+0000 7f7d6c00e700 1 Processor -- start 2023-11-28T13:07:52.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.708+0000 7f7d6c00e700 1 -- start start 2023-11-28T13:07:52.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d6c00e700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d640cc310 0x7f7d640c3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d6c00e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d6415c990 0x7f7d640b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d6c00e700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7d64161090 0x7f7d64144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d6c00e700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64155df0 con 0x7f7d640cc310 2023-11-28T13:07:52.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d6c00e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641e7c20 con 0x7f7d6415c990 2023-11-28T13:07:52.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d6c00e700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641e87a0 con 0x7f7d64161090 2023-11-28T13:07:52.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d69daa700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d640cc310 0x7f7d640c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d69daa700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d640cc310 0x7f7d640c3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35676/0 (socket says 172.21.15.6:35676) 2023-11-28T13:07:52.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d69daa700 1 -- 172.21.15.6:0/1964498028 learned_addr learned my addr 172.21.15.6:0/1964498028 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:52.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d695a9700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d6415c990 0x7f7d640b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d6a5ab700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7d64161090 0x7f7d64144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.716 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d69daa700 1 -- 172.21.15.6:0/1964498028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7d64161090 msgr2=0x7f7d64144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d69daa700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7d64161090 0x7f7d64144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d69daa700 1 -- 172.21.15.6:0/1964498028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d6415c990 msgr2=0x7f7d640b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d69daa700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d6415c990 0x7f7d640b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d69daa700 1 -- 172.21.15.6:0/1964498028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d54009550 con 0x7f7d640cc310 2023-11-28T13:07:52.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d695a9700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d6415c990 0x7f7d640b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:52.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d6a5ab700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7d64161090 0x7f7d64144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:52.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d69daa700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d640cc310 0x7f7d640c3150 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7d54014040 tx=0x7f7d54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.717 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d5a7fc700 1 -- 172.21.15.6:0/1964498028 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54011a70 con 0x7f7d640cc310 2023-11-28T13:07:52.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d5a7fc700 1 -- 172.21.15.6:0/1964498028 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5401e070 con 0x7f7d640cc310 2023-11-28T13:07:52.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.709+0000 7f7d5a7fc700 1 -- 172.21.15.6:0/1964498028 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54011dd0 con 0x7f7d640cc310 2023-11-28T13:07:52.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.710+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d640cb080 con 0x7f7d640cc310 2023-11-28T13:07:52.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.710+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d640ce870 con 0x7f7d640cc310 2023-11-28T13:07:52.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.710+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d640cf5b0 con 0x7f7d640cc310 2023-11-28T13:07:52.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.711+0000 7f7d5a7fc700 1 -- 172.21.15.6:0/1964498028 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d5400f050 con 0x7f7d640cc310 2023-11-28T13:07:52.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.711+0000 7f7d5a7fc700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7d480682f0 0x7f7d4806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.718 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.711+0000 7f7d5a7fc700 1 -- 172.21.15.6:0/1964498028 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f7d54086d40 con 0x7f7d640cc310 2023-11-28T13:07:52.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.711+0000 7f7d695a9700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7d480682f0 0x7f7d4806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.711+0000 7f7d5a7fc700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] conn(0x7f7d4806eb90 0x7f7d48070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.711+0000 7f7d5a7fc700 1 -- 172.21.15.6:0/1964498028 --> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d480716b0 con 0x7f7d4806eb90 2023-11-28T13:07:52.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.711+0000 7f7d5a7fc700 1 -- 172.21.15.6:0/1964498028 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d54089320 con 0x7f7d640cc310 2023-11-28T13:07:52.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.712+0000 7f7d6a5ab700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] conn(0x7f7d4806eb90 0x7f7d48070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.712+0000 7f7d695a9700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7d480682f0 0x7f7d4806a7a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7d6406c060 tx=0x7f7d60007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.719 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.712+0000 7f7d6a5ab700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] conn(0x7f7d4806eb90 0x7f7d48070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.720 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.712+0000 7f7d5a7fc700 1 -- 172.21.15.6:0/1964498028 <== osd.2 v2:172.21.15.6:6824/103512 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7d480716b0 con 0x7f7d4806eb90 2023-11-28T13:07:52.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.727+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 --> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7d641e88e0 con 0x7f7d4806eb90 2023-11-28T13:07:52.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.727+0000 7f7d5a7fc700 1 -- 172.21.15.6:0/1964498028 <== osd.2 v2:172.21.15.6:6824/103512 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7d5c003a00 con 0x7f7d4806eb90 2023-11-28T13:07:52.729 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:07:52.729 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:52.729 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:07:52.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.727+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] conn(0x7f7d4806eb90 msgr2=0x7f7d48070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.727+0000 7f7d6c00e700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:6824/103512,v1:172.21.15.6:6825/103512] conn(0x7f7d4806eb90 0x7f7d48070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.727+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7d480682f0 msgr2=0x7f7d4806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.727+0000 7f7d6c00e700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f7d480682f0 0x7f7d4806a7a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7d6406c060 tx=0x7f7d60007ce0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.727+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d640cc310 msgr2=0x7f7d640c3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.727+0000 7f7d6c00e700 1 --2- 172.21.15.6:0/1964498028 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7d640cc310 0x7f7d640c3150 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7d54014040 tx=0x7f7d54004750 comp rx=0 tx=0).stop 2023-11-28T13:07:52.731 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.727+0000 7f7d6a5ab700 1 -- 172.21.15.6:0/1964498028 reap_dead start 2023-11-28T13:07:52.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.728+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 shutdown_connections 2023-11-28T13:07:52.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.728+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 >> 172.21.15.6:0/1964498028 conn(0x7f7d640b1650 msgr2=0x7f7d640cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:52.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.728+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 shutdown_connections 2023-11-28T13:07:52.732 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.728+0000 7f7d6c00e700 1 -- 172.21.15.6:0/1964498028 wait complete. 2023-11-28T13:07:52.741 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T13:07:52.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.865+0000 7eff47f27700 1 Processor -- start 2023-11-28T13:07:52.867 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.865+0000 7eff47f27700 1 -- start start 2023-11-28T13:07:52.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff47f27700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff4015a6c0 0x7eff400cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff47f27700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7eff400d0020 0x7eff400ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff47f27700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7eff400b2660 0x7eff400b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff47f27700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff400b3020 con 0x7eff400d0020 2023-11-28T13:07:52.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff47f27700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff400cfb90 con 0x7eff400b2660 2023-11-28T13:07:52.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff47f27700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff400cb930 con 0x7eff4015a6c0 2023-11-28T13:07:52.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff454c2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7eff400d0020 0x7eff400ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff454c2700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7eff400d0020 0x7eff400ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35686/0 (socket says 172.21.15.6:35686) 2023-11-28T13:07:52.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff454c2700 1 -- 172.21.15.6:0/669960341 learned_addr learned my addr 172.21.15.6:0/669960341 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:52.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff464c4700 1 --2- 172.21.15.6:0/669960341 >> v2:172.21.15.61:3300/0 conn(0x7eff400b2660 0x7eff400b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.866+0000 7eff45cc3700 1 --2- 172.21.15.6:0/669960341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff4015a6c0 0x7eff400cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff454c2700 1 -- 172.21.15.6:0/669960341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff4015a6c0 msgr2=0x7eff400cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:52.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff454c2700 1 --2- 172.21.15.6:0/669960341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff4015a6c0 0x7eff400cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.872 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff454c2700 1 -- 172.21.15.6:0/669960341 >> v2:172.21.15.61:3300/0 conn(0x7eff400b2660 msgr2=0x7eff400b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:52.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff454c2700 1 --2- 172.21.15.6:0/669960341 >> v2:172.21.15.61:3300/0 conn(0x7eff400b2660 0x7eff400b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff454c2700 1 -- 172.21.15.6:0/669960341 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400cf260 con 0x7eff400d0020 2023-11-28T13:07:52.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff454c2700 1 --2- 172.21.15.6:0/669960341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff400d0020 0x7eff400ccea0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7eff40072920 tx=0x7eff30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5429c7ea35622637 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.873 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff44cc1700 1 -- 172.21.15.6:0/669960341 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30011830 con 0x7eff400d0020 2023-11-28T13:07:52.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff44cc1700 1 -- 172.21.15.6:0/669960341 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff30011990 con 0x7eff400d0020 2023-11-28T13:07:52.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff44cc1700 1 -- 172.21.15.6:0/669960341 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30011d20 con 0x7eff400d0020 2023-11-28T13:07:52.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff47f27700 1 -- 172.21.15.6:0/669960341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff400d0020 msgr2=0x7eff400ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.867+0000 7eff47f27700 1 --2- 172.21.15.6:0/669960341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff400d0020 0x7eff400ccea0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7eff40072920 tx=0x7eff30009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.868+0000 7eff47f27700 1 -- 172.21.15.6:0/669960341 shutdown_connections 2023-11-28T13:07:52.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.868+0000 7eff47f27700 1 --2- 172.21.15.6:0/669960341 >> v2:172.21.15.61:3300/0 conn(0x7eff400b2660 0x7eff400b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.868+0000 7eff47f27700 1 --2- 172.21.15.6:0/669960341 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff4015a6c0 0x7eff400cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.868+0000 7eff47f27700 1 --2- 172.21.15.6:0/669960341 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff400d0020 0x7eff400ccea0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.868+0000 7eff47f27700 1 -- 172.21.15.6:0/669960341 >> 172.21.15.6:0/669960341 conn(0x7eff400cd660 msgr2=0x7eff400cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:52.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.868+0000 7eff47f27700 1 -- 172.21.15.6:0/669960341 shutdown_connections 2023-11-28T13:07:52.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.868+0000 7eff47f27700 1 -- 172.21.15.6:0/669960341 wait complete. 2023-11-28T13:07:52.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.869+0000 7eff47f27700 1 Processor -- start 2023-11-28T13:07:52.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.869+0000 7eff47f27700 1 -- start start 2023-11-28T13:07:52.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff47f27700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff400b2660 0x7eff401f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff47f27700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff400d0020 0x7eff400ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff47f27700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4015a6c0 0x7eff4015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff47f27700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4015cf70 con 0x7eff4015a6c0 2023-11-28T13:07:52.879 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff47f27700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4015d250 con 0x7eff400d0020 2023-11-28T13:07:52.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff47f27700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4015d530 con 0x7eff400b2660 2023-11-28T13:07:52.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff464c4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4015a6c0 0x7eff4015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff464c4700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4015a6c0 0x7eff4015c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35696/0 (socket says 172.21.15.6:35696) 2023-11-28T13:07:52.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff464c4700 1 -- 172.21.15.6:0/3309643221 learned_addr learned my addr 172.21.15.6:0/3309643221 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:52.880 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff454c2700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff400d0020 0x7eff400ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff45cc3700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff400b2660 0x7eff401f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff464c4700 1 -- 172.21.15.6:0/3309643221 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff400b2660 msgr2=0x7eff401f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff464c4700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff400b2660 0x7eff401f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff464c4700 1 -- 172.21.15.6:0/3309643221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff400d0020 msgr2=0x7eff400ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:52.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff464c4700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff400d0020 0x7eff400ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:52.881 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff464c4700 1 -- 172.21.15.6:0/3309643221 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff30009550 con 0x7eff4015a6c0 2023-11-28T13:07:52.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff464c4700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4015a6c0 0x7eff4015c0a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7eff3800eed0 tx=0x7eff3800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff367fc700 1 -- 172.21.15.6:0/3309643221 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38010040 con 0x7eff4015a6c0 2023-11-28T13:07:52.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff367fc700 1 -- 172.21.15.6:0/3309643221 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3800b9f0 con 0x7eff4015a6c0 2023-11-28T13:07:52.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff47f27700 1 -- 172.21.15.6:0/3309643221 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff400cf820 con 0x7eff4015a6c0 2023-11-28T13:07:52.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff367fc700 1 -- 172.21.15.6:0/3309643221 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3800bda0 con 0x7eff4015a6c0 2023-11-28T13:07:52.882 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.870+0000 7eff47f27700 1 -- 172.21.15.6:0/3309643221 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff400cf540 con 0x7eff4015a6c0 2023-11-28T13:07:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.871+0000 7eff47f27700 1 -- 172.21.15.6:0/3309643221 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff400cd3e0 con 0x7eff4015a6c0 2023-11-28T13:07:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.872+0000 7eff367fc700 1 -- 172.21.15.6:0/3309643221 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff38014400 con 0x7eff4015a6c0 2023-11-28T13:07:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.872+0000 7eff367fc700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff240682f0 0x7eff2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.872+0000 7eff367fc700 1 -- 172.21.15.6:0/3309643221 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7eff38085f00 con 0x7eff4015a6c0 2023-11-28T13:07:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.872+0000 7eff45cc3700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff240682f0 0x7eff2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.873+0000 7eff45cc3700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff240682f0 0x7eff2406a7a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009be0 tx=0x7eff3c009210 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:52.883 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:52.877+0000 7eff367fc700 1 -- 172.21.15.6:0/3309643221 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7eff380528c0 con 0x7eff4015a6c0 2023-11-28T13:07:53.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.073+0000 7eff47f27700 1 -- 172.21.15.6:0/3309643221 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7eff4015d3c0 con 0x7eff240682f0 2023-11-28T13:07:53.074 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.073+0000 7eff367fc700 1 -- 172.21.15.6:0/3309643221 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7eff4015d3c0 con 0x7eff240682f0 2023-11-28T13:07:53.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 -- 172.21.15.6:0/3309643221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff240682f0 msgr2=0x7eff2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.076 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff240682f0 0x7eff2406a7a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009be0 tx=0x7eff3c009210 comp rx=0 tx=0).stop 2023-11-28T13:07:53.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 -- 172.21.15.6:0/3309643221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4015a6c0 msgr2=0x7eff4015c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4015a6c0 0x7eff4015c0a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7eff3800eed0 tx=0x7eff3800ef00 comp rx=0 tx=0).stop 2023-11-28T13:07:53.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 -- 172.21.15.6:0/3309643221 shutdown_connections 2023-11-28T13:07:53.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7eff4015a6c0 0x7eff4015c0a0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7eff240682f0 0x7eff2406a7a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff400b2660 0x7eff401f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.077 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 --2- 172.21.15.6:0/3309643221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff400d0020 0x7eff400ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 -- 172.21.15.6:0/3309643221 >> 172.21.15.6:0/3309643221 conn(0x7eff400cd660 msgr2=0x7eff400cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 -- 172.21.15.6:0/3309643221 shutdown_connections 2023-11-28T13:07:53.078 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.075+0000 7eff47f27700 1 -- 172.21.15.6:0/3309643221 wait complete. 2023-11-28T13:07:53.079 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T13:07:53.088 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T13:07:53.088 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:53.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.216+0000 7faf139ab700 1 Processor -- start 2023-11-28T13:07:53.217 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.216+0000 7faf139ab700 1 -- start start 2023-11-28T13:07:53.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf139ab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf0c15a6c0 0x7faf0c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf139ab700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faf0c0b2660 0x7faf0c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf139ab700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0c0cfb90 con 0x7faf0c0d0020 2023-11-28T13:07:53.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf139ab700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf0c0b3020 con 0x7faf0c0b2660 2023-11-28T13:07:53.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf139ab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0cb930 con 0x7faf0c15a6c0 2023-11-28T13:07:53.220 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf11f48700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faf0c0b2660 0x7faf0c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf11f48700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faf0c0b2660 0x7faf0c0b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35704/0 (socket says 172.21.15.6:35704) 2023-11-28T13:07:53.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf11f48700 1 -- 172.21.15.6:0/1977013698 learned_addr learned my addr 172.21.15.6:0/1977013698 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:53.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf11747700 1 --2- 172.21.15.6:0/1977013698 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf0c15a6c0 0x7faf0c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf11f48700 1 -- 172.21.15.6:0/1977013698 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf0c15a6c0 msgr2=0x7faf0c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:53.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf11f48700 1 --2- 172.21.15.6:0/1977013698 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf0c15a6c0 0x7faf0c0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf11f48700 1 -- 172.21.15.6:0/1977013698 >> v1:172.21.15.61:6789/0 conn(0x7faf0c0d0020 legacy=0x7faf0c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf11f48700 1 -- 172.21.15.6:0/1977013698 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0cf260 con 0x7faf0c0b2660 2023-11-28T13:07:53.221 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.217+0000 7faf11747700 1 --2- 172.21.15.6:0/1977013698 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf0c15a6c0 0x7faf0c0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:53.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf11f48700 1 --2- 172.21.15.6:0/1977013698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0c0b2660 0x7faf0c0b2ae0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7faf04009e50 tx=0x7faf0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e00d3bfb8ad28a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf03fff700 1 -- 172.21.15.6:0/1977013698 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0400e070 con 0x7faf0c0b2660 2023-11-28T13:07:53.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf03fff700 1 -- 172.21.15.6:0/1977013698 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0400b9c0 con 0x7faf0c0b2660 2023-11-28T13:07:53.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf03fff700 1 -- 172.21.15.6:0/1977013698 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0400bd70 con 0x7faf0c0b2660 2023-11-28T13:07:53.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf139ab700 1 -- 172.21.15.6:0/1977013698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0c0b2660 msgr2=0x7faf0c0b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf139ab700 1 --2- 172.21.15.6:0/1977013698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0c0b2660 0x7faf0c0b2ae0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7faf04009e50 tx=0x7faf0400b330 comp rx=0 tx=0).stop 2023-11-28T13:07:53.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf139ab700 1 -- 172.21.15.6:0/1977013698 shutdown_connections 2023-11-28T13:07:53.222 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf139ab700 1 --2- 172.21.15.6:0/1977013698 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0c0b2660 0x7faf0c0b2ae0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf139ab700 1 --2- 172.21.15.6:0/1977013698 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf0c15a6c0 0x7faf0c0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf139ab700 1 -- 172.21.15.6:0/1977013698 >> 172.21.15.6:0/1977013698 conn(0x7faf0c0cd660 msgr2=0x7faf0c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.218+0000 7faf139ab700 1 -- 172.21.15.6:0/1977013698 shutdown_connections 2023-11-28T13:07:53.223 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.219+0000 7faf139ab700 1 -- 172.21.15.6:0/1977013698 wait complete. 2023-11-28T13:07:53.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.220+0000 7faf139ab700 1 Processor -- start 2023-11-28T13:07:53.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.220+0000 7faf139ab700 1 -- start start 2023-11-28T13:07:53.226 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf139ab700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf0c0b2660 0x7faf0c0aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf139ab700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0c0d0020 0x7faf0c0b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf139ab700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf0c15a6c0 0x7faf0c0b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf139ab700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0cb040 con 0x7faf0c0d0020 2023-11-28T13:07:53.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf139ab700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1f3440 con 0x7faf0c0b2660 2023-11-28T13:07:53.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf139ab700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1f2460 con 0x7faf0c15a6c0 2023-11-28T13:07:53.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf10f46700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0c0d0020 0x7faf0c0b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf10f46700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0c0d0020 0x7faf0c0b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35708/0 (socket says 172.21.15.6:35708) 2023-11-28T13:07:53.227 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf10f46700 1 -- 172.21.15.6:0/2897034932 learned_addr learned my addr 172.21.15.6:0/2897034932 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:53.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf11747700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf0c0b2660 0x7faf0c0aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf11f48700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf0c15a6c0 0x7faf0c0b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf10f46700 1 -- 172.21.15.6:0/2897034932 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf0c15a6c0 msgr2=0x7faf0c0b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf10f46700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7faf0c15a6c0 0x7faf0c0b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf10f46700 1 -- 172.21.15.6:0/2897034932 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf0c0b2660 msgr2=0x7faf0c0aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf10f46700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faf0c0b2660 0x7faf0c0aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.228 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf10f46700 1 -- 172.21.15.6:0/2897034932 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04009b30 con 0x7faf0c0d0020 2023-11-28T13:07:53.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf10f46700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0c0d0020 0x7faf0c0b1840 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7faefc00c4e0 tx=0x7faefc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf01ffb700 1 -- 172.21.15.6:0/2897034932 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0052d0 con 0x7faf0c0d0020 2023-11-28T13:07:53.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf01ffb700 1 -- 172.21.15.6:0/2897034932 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc005430 con 0x7faf0c0d0020 2023-11-28T13:07:53.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.221+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c0cf820 con 0x7faf0c0d0020 2023-11-28T13:07:53.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.222+0000 7faf01ffb700 1 -- 172.21.15.6:0/2897034932 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0038c0 con 0x7faf0c0d0020 2023-11-28T13:07:53.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.222+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c0cf540 con 0x7faf0c0d0020 2023-11-28T13:07:53.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.222+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf0c0d0470 con 0x7faf0c0d0020 2023-11-28T13:07:53.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.223+0000 7faf01ffb700 1 -- 172.21.15.6:0/2897034932 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faefc019030 con 0x7faf0c0d0020 2023-11-28T13:07:53.229 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.223+0000 7faf01ffb700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faef00682f0 0x7faef006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.224+0000 7faf01ffb700 1 -- 172.21.15.6:0/2897034932 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7faefc084cf0 con 0x7faf0c0d0020 2023-11-28T13:07:53.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.224+0000 7faf11747700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faef00682f0 0x7faef006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.224+0000 7faf01ffb700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] conn(0x7faef006eb90 0x7faef0070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.224+0000 7faf01ffb700 1 -- 172.21.15.6:0/2897034932 --> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faef00716b0 con 0x7faef006eb90 2023-11-28T13:07:53.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.224+0000 7faf01ffb700 1 -- 172.21.15.6:0/2897034932 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7faefc087340 con 0x7faf0c0d0020 2023-11-28T13:07:53.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.224+0000 7faf11f48700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] conn(0x7faef006eb90 0x7faef0070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.230 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.224+0000 7faf11747700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faef00682f0 0x7faef006a7a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7faf0800c940 tx=0x7faf08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.224+0000 7faf11f48700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] conn(0x7faef006eb90 0x7faef0070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.231 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.225+0000 7faf01ffb700 1 -- 172.21.15.6:0/2897034932 <== osd.3 v2:172.21.15.6:6808/103508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faef00716b0 con 0x7faef006eb90 2023-11-28T13:07:53.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 --> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faf0c06bec0 con 0x7faef006eb90 2023-11-28T13:07:53.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf01ffb700 1 -- 172.21.15.6:0/2897034932 <== osd.3 v2:172.21.15.6:6808/103508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7faf0c06bec0 con 0x7faef006eb90 2023-11-28T13:07:53.241 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:07:53.241 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:53.242 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:07:53.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] conn(0x7faef006eb90 msgr2=0x7faef0070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:6808/103508,v1:172.21.15.6:6810/103508] conn(0x7faef006eb90 0x7faef0070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faef00682f0 msgr2=0x7faef006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7faef00682f0 0x7faef006a7a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7faf0800c940 tx=0x7faf08008040 comp rx=0 tx=0).stop 2023-11-28T13:07:53.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0c0d0020 msgr2=0x7faf0c0b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 --2- 172.21.15.6:0/2897034932 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf0c0d0020 0x7faf0c0b1840 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7faefc00c4e0 tx=0x7faefc00c7f0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf11f48700 1 -- 172.21.15.6:0/2897034932 reap_dead start 2023-11-28T13:07:53.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 shutdown_connections 2023-11-28T13:07:53.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 >> 172.21.15.6:0/2897034932 conn(0x7faf0c0cd660 msgr2=0x7faf0c0b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 shutdown_connections 2023-11-28T13:07:53.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.240+0000 7faf139ab700 1 -- 172.21.15.6:0/2897034932 wait complete. 2023-11-28T13:07:53.253 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T13:07:53.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.377+0000 7f445b04d700 1 Processor -- start 2023-11-28T13:07:53.379 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.378+0000 7f445b04d700 1 -- start start 2023-11-28T13:07:53.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.378+0000 7f445b04d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f445406bc30 0x7f445415b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.378+0000 7f445b04d700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445406c3a0 con 0x7f445406d9d0 2023-11-28T13:07:53.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.378+0000 7f445b04d700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445406c0c0 con 0x7f445406dda0 2023-11-28T13:07:53.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.378+0000 7f445b04d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445415ab60 con 0x7f445406bc30 2023-11-28T13:07:53.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.379+0000 7f4458de9700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f445406d9d0 0x7f4454150eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51268/0 (socket says 172.21.15.6:51268) 2023-11-28T13:07:53.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.379+0000 7f4458de9700 1 -- 172.21.15.6:0/3117945681 learned_addr learned my addr 172.21.15.6:0/3117945681 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:53.382 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.379+0000 7f4453fff700 1 --2- 172.21.15.6:0/3117945681 >> v2:172.21.15.61:3300/0 conn(0x7f445406bc30 0x7f445415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.379+0000 7f44537fe700 1 -- 172.21.15.6:0/3117945681 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853792389 0 0) 0x7f445406c3a0 con 0x7f445406d9d0 2023-11-28T13:07:53.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.379+0000 7f44537fe700 1 -- 172.21.15.6:0/3117945681 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44440035b0 con 0x7f445406d9d0 2023-11-28T13:07:53.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.379+0000 7f4453fff700 1 -- 172.21.15.6:0/3117945681 >> v1:172.21.15.61:6789/0 conn(0x7f445406dda0 legacy=0x7f44540c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.379+0000 7f4453fff700 1 -- 172.21.15.6:0/3117945681 >> v1:172.21.15.6:6789/0 conn(0x7f445406d9d0 legacy=0x7f4454150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.379+0000 7f44537fe700 1 -- 172.21.15.6:0/3117945681 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2243152274 0 0) 0x7f44440035b0 con 0x7f445406d9d0 2023-11-28T13:07:53.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.379+0000 7f4453fff700 1 -- 172.21.15.6:0/3117945681 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445415ae40 con 0x7f445406bc30 2023-11-28T13:07:53.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.379+0000 7f4453fff700 1 --2- 172.21.15.6:0/3117945681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f445406bc30 0x7f445415b640 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4440009840 tx=0x7f4440009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=440b338f66a060ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.383 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.380+0000 7f44537fe700 1 -- 172.21.15.6:0/3117945681 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448001670 con 0x7f445406bc30 2023-11-28T13:07:53.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.380+0000 7f44537fe700 1 -- 172.21.15.6:0/3117945681 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4440004030 con 0x7f445406bc30 2023-11-28T13:07:53.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.380+0000 7f44537fe700 1 -- 172.21.15.6:0/3117945681 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400043c0 con 0x7f445406bc30 2023-11-28T13:07:53.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.380+0000 7f445b04d700 1 -- 172.21.15.6:0/3117945681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f445406bc30 msgr2=0x7f445415b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.380+0000 7f445b04d700 1 --2- 172.21.15.6:0/3117945681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f445406bc30 0x7f445415b640 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4440009840 tx=0x7f4440009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:53.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.380+0000 7f445b04d700 1 -- 172.21.15.6:0/3117945681 shutdown_connections 2023-11-28T13:07:53.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.380+0000 7f445b04d700 1 --2- 172.21.15.6:0/3117945681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f445406bc30 0x7f445415b640 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.380+0000 7f445b04d700 1 -- 172.21.15.6:0/3117945681 >> 172.21.15.6:0/3117945681 conn(0x7f44540b0dd0 msgr2=0x7f44540b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.384 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.380+0000 7f445b04d700 1 -- 172.21.15.6:0/3117945681 shutdown_connections 2023-11-28T13:07:53.385 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.380+0000 7f445b04d700 1 -- 172.21.15.6:0/3117945681 wait complete. 2023-11-28T13:07:53.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.381+0000 7f445b04d700 1 Processor -- start 2023-11-28T13:07:53.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.381+0000 7f445b04d700 1 -- start start 2023-11-28T13:07:53.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f445b04d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f445406bc30 0x7f44540c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.387 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f445b04d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f445406d9d0 0x7f445414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f445b04d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f445406dda0 0x7f445414d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f445b04d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540c4750 con 0x7f445406dda0 2023-11-28T13:07:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f445b04d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540c4a30 con 0x7f445406bc30 2023-11-28T13:07:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f445b04d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540c4d10 con 0x7f445406d9d0 2023-11-28T13:07:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f44595ea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f445406dda0 0x7f445414d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f44595ea700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f445406dda0 0x7f445414d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35726/0 (socket says 172.21.15.6:35726) 2023-11-28T13:07:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f44595ea700 1 -- 172.21.15.6:0/1313833347 learned_addr learned my addr 172.21.15.6:0/1313833347 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:53.388 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f4458de9700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f445406bc30 0x7f44540c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.382+0000 7f4453fff700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f445406d9d0 0x7f445414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f44595ea700 1 -- 172.21.15.6:0/1313833347 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f445406d9d0 msgr2=0x7f445414d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f44595ea700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f445406d9d0 0x7f445414d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f44595ea700 1 -- 172.21.15.6:0/1313833347 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f445406bc30 msgr2=0x7f44540c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f44595ea700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f445406bc30 0x7f44540c2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f44595ea700 1 -- 172.21.15.6:0/1313833347 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400094a0 con 0x7f445406dda0 2023-11-28T13:07:53.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f44595ea700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f445406dda0 0x7f445414d7b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f444c00cb50 tx=0x7f444c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.389 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f44517fa700 1 -- 172.21.15.6:0/1313833347 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c003c00 con 0x7f445406dda0 2023-11-28T13:07:53.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f445b04d700 1 -- 172.21.15.6:0/1313833347 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445406fac0 con 0x7f445406dda0 2023-11-28T13:07:53.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f44517fa700 1 -- 172.21.15.6:0/1313833347 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c003d60 con 0x7f445406dda0 2023-11-28T13:07:53.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f44517fa700 1 -- 172.21.15.6:0/1313833347 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c00a9c0 con 0x7f445406dda0 2023-11-28T13:07:53.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.383+0000 7f445b04d700 1 -- 172.21.15.6:0/1313833347 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445415b120 con 0x7f445406dda0 2023-11-28T13:07:53.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.384+0000 7f445b04d700 1 -- 172.21.15.6:0/1313833347 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44541515c0 con 0x7f445406dda0 2023-11-28T13:07:53.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.384+0000 7f44517fa700 1 -- 172.21.15.6:0/1313833347 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f444c010050 con 0x7f445406dda0 2023-11-28T13:07:53.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.385+0000 7f44517fa700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44380682f0 0x7f443806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.390 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.385+0000 7f44517fa700 1 -- 172.21.15.6:0/1313833347 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f444c0859a0 con 0x7f445406dda0 2023-11-28T13:07:53.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.385+0000 7f4458de9700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44380682f0 0x7f443806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.385+0000 7f4458de9700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44380682f0 0x7f443806a7a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4448001eb0 tx=0x7f4448001e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.391 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.389+0000 7f44517fa700 1 -- 172.21.15.6:0/1313833347 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f444c052340 con 0x7f445406dda0 2023-11-28T13:07:53.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.580+0000 7f445b04d700 1 -- 172.21.15.6:0/1313833347 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f44540c5230 con 0x7f44380682f0 2023-11-28T13:07:53.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.580+0000 7f44517fa700 1 -- 172.21.15.6:0/1313833347 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f44540c5230 con 0x7f44380682f0 2023-11-28T13:07:53.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 -- 172.21.15.6:0/1313833347 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44380682f0 msgr2=0x7f443806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44380682f0 0x7f443806a7a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4448001eb0 tx=0x7f4448001e40 comp rx=0 tx=0).stop 2023-11-28T13:07:53.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 -- 172.21.15.6:0/1313833347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f445406dda0 msgr2=0x7f445414d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f445406dda0 0x7f445414d7b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f444c00cb50 tx=0x7f444c00ce60 comp rx=0 tx=0).stop 2023-11-28T13:07:53.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 -- 172.21.15.6:0/1313833347 shutdown_connections 2023-11-28T13:07:53.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f445406dda0 0x7f445414d7b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f445406bc30 0x7f44540c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f44380682f0 0x7f443806a7a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 --2- 172.21.15.6:0/1313833347 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f445406d9d0 0x7f445414d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 -- 172.21.15.6:0/1313833347 >> 172.21.15.6:0/1313833347 conn(0x7f44540b0dd0 msgr2=0x7f44540af130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 -- 172.21.15.6:0/1313833347 shutdown_connections 2023-11-28T13:07:53.585 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.582+0000 7f445b04d700 1 -- 172.21.15.6:0/1313833347 wait complete. 2023-11-28T13:07:53.585 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T13:07:53.595 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T13:07:53.596 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:53.724 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.723+0000 7f5e0f978700 1 Processor -- start 2023-11-28T13:07:53.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.723+0000 7f5e0f978700 1 -- start start 2023-11-28T13:07:53.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0f978700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5e0806aa70 0x7f5e080ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0f978700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5e0815c680 0x7f5e0815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.727 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0f978700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0806ed80 con 0x7f5e08158a30 2023-11-28T13:07:53.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0f978700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0806f060 con 0x7f5e0806aa70 2023-11-28T13:07:53.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0f978700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e080cd950 con 0x7f5e0815c680 2023-11-28T13:07:53.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0d714700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f5e08158a30 0x7f5e080ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51282/0 (socket says 172.21.15.6:51282) 2023-11-28T13:07:53.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0d714700 1 -- 172.21.15.6:0/62808337 learned_addr learned my addr 172.21.15.6:0/62808337 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:53.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0cf13700 1 --2- 172.21.15.6:0/62808337 >> v2:172.21.15.6:3300/0 conn(0x7f5e0806aa70 0x7f5e080ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0df15700 1 --2- 172.21.15.6:0/62808337 >> v2:172.21.15.61:3300/0 conn(0x7f5e0815c680 0x7f5e0815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0cf13700 1 -- 172.21.15.6:0/62808337 >> v2:172.21.15.61:3300/0 conn(0x7f5e0815c680 msgr2=0x7f5e0815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:53.728 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0cf13700 1 --2- 172.21.15.6:0/62808337 >> v2:172.21.15.61:3300/0 conn(0x7f5e0815c680 0x7f5e0815cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0cf13700 1 -- 172.21.15.6:0/62808337 >> v1:172.21.15.6:6789/0 conn(0x7f5e08158a30 legacy=0x7f5e080ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0cf13700 1 -- 172.21.15.6:0/62808337 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e080cdc30 con 0x7f5e0806aa70 2023-11-28T13:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.724+0000 7f5e0df15700 1 --2- 172.21.15.6:0/62808337 >> v2:172.21.15.61:3300/0 conn(0x7f5e0815c680 0x7f5e0815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5e0cf13700 1 --2- 172.21.15.6:0/62808337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0806aa70 0x7f5e080ce860 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5e080c6330 tx=0x7f5e04009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe9ea6f449844cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5dfffff700 1 -- 172.21.15.6:0/62808337 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04011830 con 0x7f5e0806aa70 2023-11-28T13:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5dfffff700 1 -- 172.21.15.6:0/62808337 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e04011990 con 0x7f5e0806aa70 2023-11-28T13:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5dfffff700 1 -- 172.21.15.6:0/62808337 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04011d20 con 0x7f5e0806aa70 2023-11-28T13:07:53.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5e0f978700 1 -- 172.21.15.6:0/62808337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0806aa70 msgr2=0x7f5e080ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5e0f978700 1 --2- 172.21.15.6:0/62808337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0806aa70 0x7f5e080ce860 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5e080c6330 tx=0x7f5e04009b20 comp rx=0 tx=0).stop 2023-11-28T13:07:53.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5e0f978700 1 -- 172.21.15.6:0/62808337 shutdown_connections 2023-11-28T13:07:53.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5e0f978700 1 --2- 172.21.15.6:0/62808337 >> v2:172.21.15.61:3300/0 conn(0x7f5e0815c680 0x7f5e0815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5e0f978700 1 --2- 172.21.15.6:0/62808337 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e0806aa70 0x7f5e080ce860 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5e0f978700 1 -- 172.21.15.6:0/62808337 >> 172.21.15.6:0/62808337 conn(0x7f5e080b0dd0 msgr2=0x7f5e080b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5e0f978700 1 -- 172.21.15.6:0/62808337 shutdown_connections 2023-11-28T13:07:53.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.725+0000 7f5e0f978700 1 -- 172.21.15.6:0/62808337 wait complete. 2023-11-28T13:07:53.733 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0f978700 1 Processor -- start 2023-11-28T13:07:53.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0f978700 1 -- start start 2023-11-28T13:07:53.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0f978700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e0806aa70 0x7f5e08148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0f978700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e08158a30 0x7f5e08163d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0f978700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e0815c680 0x7f5e080c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0f978700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e080b30a0 con 0x7f5e08158a30 2023-11-28T13:07:53.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0f978700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e080b3380 con 0x7f5e0806aa70 2023-11-28T13:07:53.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0f978700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e080b3660 con 0x7f5e0815c680 2023-11-28T13:07:53.734 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0cf13700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e08158a30 0x7f5e08163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0cf13700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e08158a30 0x7f5e08163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35738/0 (socket says 172.21.15.6:35738) 2023-11-28T13:07:53.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.727+0000 7f5e0cf13700 1 -- 172.21.15.6:0/2586018253 learned_addr learned my addr 172.21.15.6:0/2586018253 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:53.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0d714700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e0806aa70 0x7f5e08148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0cf13700 1 -- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e0815c680 msgr2=0x7f5e080c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0df15700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e0815c680 0x7f5e080c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0cf13700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5e0815c680 0x7f5e080c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0cf13700 1 -- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e0806aa70 msgr2=0x7f5e08148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.735 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0cf13700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e0806aa70 0x7f5e08148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0cf13700 1 -- 172.21.15.6:0/2586018253 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e040094a0 con 0x7f5e08158a30 2023-11-28T13:07:53.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0d714700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5e0806aa70 0x7f5e08148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:53.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0cf13700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e08158a30 0x7f5e08163d30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5e04000c00 tx=0x7f5e04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5dfdffb700 1 -- 172.21.15.6:0/2586018253 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e040038a0 con 0x7f5e08158a30 2023-11-28T13:07:53.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5dfdffb700 1 -- 172.21.15.6:0/2586018253 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0401f070 con 0x7f5e08158a30 2023-11-28T13:07:53.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5dfdffb700 1 -- 172.21.15.6:0/2586018253 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e04003c20 con 0x7f5e08158a30 2023-11-28T13:07:53.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0806f340 con 0x7f5e08158a30 2023-11-28T13:07:53.736 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.728+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e080ceda0 con 0x7f5e08158a30 2023-11-28T13:07:53.737 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.729+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e080b34f0 con 0x7f5e08158a30 2023-11-28T13:07:53.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.730+0000 7f5dfdffb700 1 -- 172.21.15.6:0/2586018253 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5e0400f050 con 0x7f5e08158a30 2023-11-28T13:07:53.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.730+0000 7f5dfdffb700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5dec0682f0 0x7f5dec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.730+0000 7f5dfdffb700 1 -- 172.21.15.6:0/2586018253 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f5e04021070 con 0x7f5e08158a30 2023-11-28T13:07:53.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.730+0000 7f5e0d714700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5dec0682f0 0x7f5dec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.730+0000 7f5dfdffb700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] conn(0x7f5dec06eb90 0x7f5dec070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.730+0000 7f5dfdffb700 1 -- 172.21.15.6:0/2586018253 --> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5dec0716b0 con 0x7f5dec06eb90 2023-11-28T13:07:53.738 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.730+0000 7f5dfdffb700 1 -- 172.21.15.6:0/2586018253 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e04089a70 con 0x7f5e08158a30 2023-11-28T13:07:53.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.731+0000 7f5e0d714700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5dec0682f0 0x7f5dec06a7a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5df8000f60 tx=0x7f5df8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.731+0000 7f5e0df15700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] conn(0x7f5dec06eb90 0x7f5dec070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.731+0000 7f5e0df15700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] conn(0x7f5dec06eb90 0x7f5dec070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.739 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.732+0000 7f5dfdffb700 1 -- 172.21.15.6:0/2586018253 <== osd.4 v2:172.21.15.61:6816/103782 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5dec0716b0 con 0x7f5dec06eb90 2023-11-28T13:07:53.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.746+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 --> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5e0806e9c0 con 0x7f5dec06eb90 2023-11-28T13:07:53.747 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.746+0000 7f5dfdffb700 1 -- 172.21.15.6:0/2586018253 <== osd.4 v2:172.21.15.61:6816/103782 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5e0806e9c0 con 0x7f5dec06eb90 2023-11-28T13:07:53.748 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:07:53.748 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:53.748 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:07:53.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] conn(0x7f5dec06eb90 msgr2=0x7f5dec070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0f978700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.61:6816/103782,v1:172.21.15.61:6817/103782] conn(0x7f5dec06eb90 0x7f5dec070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.749 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5dec0682f0 msgr2=0x7f5dec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0f978700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5dec0682f0 0x7f5dec06a7a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5df8000f60 tx=0x7f5df8009040 comp rx=0 tx=0).stop 2023-11-28T13:07:53.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e08158a30 msgr2=0x7f5e08163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0f978700 1 --2- 172.21.15.6:0/2586018253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5e08158a30 0x7f5e08163d30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5e04000c00 tx=0x7f5e04004750 comp rx=0 tx=0).stop 2023-11-28T13:07:53.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0df15700 1 -- 172.21.15.6:0/2586018253 reap_dead start 2023-11-28T13:07:53.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 shutdown_connections 2023-11-28T13:07:53.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 >> 172.21.15.6:0/2586018253 conn(0x7f5e080b0dd0 msgr2=0x7f5e080caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 shutdown_connections 2023-11-28T13:07:53.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.747+0000 7f5e0f978700 1 -- 172.21.15.6:0/2586018253 wait complete. 2023-11-28T13:07:53.762 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T13:07:53.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.886+0000 7fd6da86d700 1 Processor -- start 2023-11-28T13:07:53.887 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.886+0000 7fd6da86d700 1 -- start start 2023-11-28T13:07:53.889 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.886+0000 7fd6da86d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d4161090 0x7fd6d40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.886+0000 7fd6da86d700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fd6d40caeb0 0x7fd6d40cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.886+0000 7fd6da86d700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d40ce340 con 0x7fd6d415c990 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6da86d700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d40cebe0 con 0x7fd6d40caeb0 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6da86d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40cf780 con 0x7fd6d4161090 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d3fff700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd6d415c990 0x7fd6d40cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51298/0 (socket says 172.21.15.6:51298) 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d3fff700 1 -- 172.21.15.6:0/4019817960 learned_addr learned my addr 172.21.15.6:0/4019817960 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d8e0a700 1 --2- 172.21.15.6:0/4019817960 >> v2:172.21.15.61:3300/0 conn(0x7fd6d40caeb0 0x7fd6d40cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.890 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d37fe700 1 --2- 172.21.15.6:0/4019817960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d4161090 0x7fd6d40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d2ffd700 1 -- 172.21.15.6:0/4019817960 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529799553 0 0) 0x7fd6d40ce340 con 0x7fd6d415c990 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d2ffd700 1 -- 172.21.15.6:0/4019817960 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6bc0035b0 con 0x7fd6d415c990 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d8e0a700 1 -- 172.21.15.6:0/4019817960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d4161090 msgr2=0x7fd6d40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d8e0a700 1 --2- 172.21.15.6:0/4019817960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d4161090 0x7fd6d40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d8e0a700 1 -- 172.21.15.6:0/4019817960 >> v1:172.21.15.6:6789/0 conn(0x7fd6d415c990 legacy=0x7fd6d40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d2ffd700 1 -- 172.21.15.6:0/4019817960 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2417833874 0 0) 0x7fd6bc0035b0 con 0x7fd6d415c990 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d8e0a700 1 -- 172.21.15.6:0/4019817960 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d40cb3f0 con 0x7fd6d40caeb0 2023-11-28T13:07:53.891 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.887+0000 7fd6d8e0a700 1 --2- 172.21.15.6:0/4019817960 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d40caeb0 0x7fd6d40cc310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009d90 tx=0x7fd6cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6afdfbe665537a1f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6d2ffd700 1 -- 172.21.15.6:0/4019817960 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00e070 con 0x7fd6d40caeb0 2023-11-28T13:07:53.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6d2ffd700 1 -- 172.21.15.6:0/4019817960 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6cc003a20 con 0x7fd6d40caeb0 2023-11-28T13:07:53.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6d2ffd700 1 -- 172.21.15.6:0/4019817960 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4001670 con 0x7fd6d40caeb0 2023-11-28T13:07:53.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6da86d700 1 -- 172.21.15.6:0/4019817960 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d40caeb0 msgr2=0x7fd6d40cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6da86d700 1 --2- 172.21.15.6:0/4019817960 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d40caeb0 0x7fd6d40cc310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009d90 tx=0x7fd6cc00b330 comp rx=0 tx=0).stop 2023-11-28T13:07:53.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6da86d700 1 -- 172.21.15.6:0/4019817960 shutdown_connections 2023-11-28T13:07:53.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6da86d700 1 --2- 172.21.15.6:0/4019817960 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d40caeb0 0x7fd6d40cc310 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6da86d700 1 --2- 172.21.15.6:0/4019817960 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d4161090 0x7fd6d40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6da86d700 1 -- 172.21.15.6:0/4019817960 >> 172.21.15.6:0/4019817960 conn(0x7fd6d40b1650 msgr2=0x7fd6d40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:53.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6da86d700 1 -- 172.21.15.6:0/4019817960 shutdown_connections 2023-11-28T13:07:53.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.888+0000 7fd6da86d700 1 -- 172.21.15.6:0/4019817960 wait complete. 2023-11-28T13:07:53.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.890+0000 7fd6da86d700 1 Processor -- start 2023-11-28T13:07:53.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.890+0000 7fd6da86d700 1 -- start start 2023-11-28T13:07:53.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6da86d700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d40caeb0 0x7fd6d415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6da86d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d415c990 0x7fd6d40c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6da86d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6d4161090 0x7fd6d40c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6da86d700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40c7b80 con 0x7fd6d4161090 2023-11-28T13:07:53.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6da86d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40acdb0 con 0x7fd6d415c990 2023-11-28T13:07:53.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6da86d700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40c75c0 con 0x7fd6d40caeb0 2023-11-28T13:07:53.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6d8e0a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6d4161090 0x7fd6d40c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6d8e0a700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6d4161090 0x7fd6d40c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35754/0 (socket says 172.21.15.6:35754) 2023-11-28T13:07:53.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6d8e0a700 1 -- 172.21.15.6:0/2376264186 learned_addr learned my addr 172.21.15.6:0/2376264186 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:53.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6d3fff700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d40caeb0 0x7fd6d415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6d37fe700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d415c990 0x7fd6d40c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6d8e0a700 1 -- 172.21.15.6:0/2376264186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d40caeb0 msgr2=0x7fd6d415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6d8e0a700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d40caeb0 0x7fd6d415a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6d8e0a700 1 -- 172.21.15.6:0/2376264186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d415c990 msgr2=0x7fd6d40c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:53.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6d8e0a700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d415c990 0x7fd6d40c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:53.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.891+0000 7fd6d8e0a700 1 -- 172.21.15.6:0/2376264186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc009a70 con 0x7fd6d4161090 2023-11-28T13:07:53.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.892+0000 7fd6d8e0a700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6d4161090 0x7fd6d40c7f90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd6d406eac0 tx=0x7fd6cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.892+0000 7fd6d0ff9700 1 -- 172.21.15.6:0/2376264186 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0044c0 con 0x7fd6d4161090 2023-11-28T13:07:53.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.892+0000 7fd6d0ff9700 1 -- 172.21.15.6:0/2376264186 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6cc004640 con 0x7fd6d4161090 2023-11-28T13:07:53.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.892+0000 7fd6da86d700 1 -- 172.21.15.6:0/2376264186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d40ce870 con 0x7fd6d4161090 2023-11-28T13:07:53.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.892+0000 7fd6d0ff9700 1 -- 172.21.15.6:0/2376264186 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc012430 con 0x7fd6d4161090 2023-11-28T13:07:53.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.892+0000 7fd6da86d700 1 -- 172.21.15.6:0/2376264186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d40ce590 con 0x7fd6d4161090 2023-11-28T13:07:53.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.893+0000 7fd6da86d700 1 -- 172.21.15.6:0/2376264186 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d40cc850 con 0x7fd6d4161090 2023-11-28T13:07:53.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.893+0000 7fd6d0ff9700 1 -- 172.21.15.6:0/2376264186 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd6cc02d070 con 0x7fd6d4161090 2023-11-28T13:07:53.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.894+0000 7fd6d0ff9700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6c00682f0 0x7fd6c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:53.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.894+0000 7fd6d0ff9700 1 -- 172.21.15.6:0/2376264186 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fd6cc0879d0 con 0x7fd6d4161090 2023-11-28T13:07:53.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.894+0000 7fd6d3fff700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6c00682f0 0x7fd6c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:53.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.894+0000 7fd6d3fff700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6c00682f0 0x7fd6c006a7a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4001eb0 tx=0x7fd6c4001e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:53.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:53.898+0000 7fd6d0ff9700 1 -- 172.21.15.6:0/2376264186 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6cc054370 con 0x7fd6d4161090 2023-11-28T13:07:54.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.091+0000 7fd6da86d700 1 -- 172.21.15.6:0/2376264186 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6d40acf20 con 0x7fd6c00682f0 2023-11-28T13:07:54.093 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.092+0000 7fd6d0ff9700 1 -- 172.21.15.6:0/2376264186 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd6d40acf20 con 0x7fd6c00682f0 2023-11-28T13:07:54.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 -- 172.21.15.6:0/2376264186 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6c00682f0 msgr2=0x7fd6c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6c00682f0 0x7fd6c006a7a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4001eb0 tx=0x7fd6c4001e40 comp rx=0 tx=0).stop 2023-11-28T13:07:54.095 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 -- 172.21.15.6:0/2376264186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6d4161090 msgr2=0x7fd6d40c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6d4161090 0x7fd6d40c7f90 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd6d406eac0 tx=0x7fd6cc004ad0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 -- 172.21.15.6:0/2376264186 shutdown_connections 2023-11-28T13:07:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6d4161090 0x7fd6d40c7f90 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fd6c00682f0 0x7fd6c006a7a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fd6d40caeb0 0x7fd6d415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 --2- 172.21.15.6:0/2376264186 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6d415c990 0x7fd6d40c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 -- 172.21.15.6:0/2376264186 >> 172.21.15.6:0/2376264186 conn(0x7fd6d40b1650 msgr2=0x7fd6d415bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.096 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 -- 172.21.15.6:0/2376264186 shutdown_connections 2023-11-28T13:07:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.094+0000 7fd6da86d700 1 -- 172.21.15.6:0/2376264186 wait complete. 2023-11-28T13:07:54.097 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T13:07:54.107 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T13:07:54.107 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:54.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.236+0000 7eff04e63700 1 Processor -- start 2023-11-28T13:07:54.237 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.236+0000 7eff04e63700 1 -- start start 2023-11-28T13:07:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7eff04e63700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7eff0006b930 0x7eff000ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7eff04e63700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff000cb570 con 0x7eff000cc4b0 2023-11-28T13:07:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7eff04e63700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff000cc110 con 0x7eff0015a5a0 2023-11-28T13:07:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7eff04e63700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0006bda0 con 0x7eff0006b930 2023-11-28T13:07:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefed9d700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7eff000cc4b0 0x7eff000c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51316/0 (socket says 172.21.15.6:51316) 2023-11-28T13:07:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefed9d700 1 -- 172.21.15.6:0/493640956 learned_addr learned my addr 172.21.15.6:0/493640956 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:54.241 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207052210 0 0) 0x7eff000cb570 con 0x7eff000cc4b0 2023-11-28T13:07:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee80035b0 con 0x7eff000cc4b0 2023-11-28T13:07:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefdd9b700 1 --2- 172.21.15.6:0/493640956 >> v2:172.21.15.61:3300/0 conn(0x7eff0006b930 0x7eff000ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3859424937 0 0) 0x7efee80035b0 con 0x7eff000cc4b0 2023-11-28T13:07:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 >> v2:172.21.15.61:3300/0 conn(0x7eff0006b930 msgr2=0x7eff000ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 --2- 172.21.15.6:0/493640956 >> v2:172.21.15.61:3300/0 conn(0x7eff0006b930 0x7eff000ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 >> v1:172.21.15.61:6789/0 conn(0x7eff0015a5a0 legacy=0x7eff000cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.242 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff000ca960 con 0x7eff000cc4b0 2023-11-28T13:07:54.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7efef80019f0 con 0x7eff000cc4b0 2023-11-28T13:07:54.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 >> v1:172.21.15.6:6789/0 conn(0x7eff000cc4b0 legacy=0x7eff000c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efee8003a00 0x7efee8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 0x7efee8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efee8005200 0x7efee800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0006bda0 con 0x7efee8004850 2023-11-28T13:07:54.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.237+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee800c080 con 0x7efee8003a00 2023-11-28T13:07:54.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee800c240 con 0x7efee8005200 2023-11-28T13:07:54.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefdd9b700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 0x7efee8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefe59c700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efee8003a00 0x7efee8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefed9d700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efee8005200 0x7efee800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefdd9b700 1 -- 172.21.15.6:0/493640956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efee8005200 msgr2=0x7efee800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefdd9b700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7efee8005200 0x7efee800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefdd9b700 1 -- 172.21.15.6:0/493640956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efee8003a00 msgr2=0x7efee8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefdd9b700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efee8003a00 0x7efee8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.244 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefdd9b700 1 -- 172.21.15.6:0/493640956 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee80035b0 con 0x7efee8004850 2023-11-28T13:07:54.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefed9d700 1 -- 172.21.15.6:0/493640956 reap_dead start 2023-11-28T13:07:54.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefdd9b700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 0x7efee8004cc0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7efef4009b60 tx=0x7efee8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97cec772b209e90 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400e6c0 con 0x7efee8004850 2023-11-28T13:07:54.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef400e820 con 0x7efee8004850 2023-11-28T13:07:54.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.238+0000 7efefd59a700 1 -- 172.21.15.6:0/493640956 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400ebd0 con 0x7efee8004850 2023-11-28T13:07:54.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.239+0000 7eff04e63700 1 -- 172.21.15.6:0/493640956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 msgr2=0x7efee8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.239+0000 7eff04e63700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 0x7efee8004cc0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7efef4009b60 tx=0x7efee8005880 comp rx=0 tx=0).stop 2023-11-28T13:07:54.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.239+0000 7eff04e63700 1 -- 172.21.15.6:0/493640956 shutdown_connections 2023-11-28T13:07:54.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.239+0000 7eff04e63700 1 --2- 172.21.15.6:0/493640956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 0x7efee8004cc0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.239+0000 7eff04e63700 1 -- 172.21.15.6:0/493640956 >> 172.21.15.6:0/493640956 conn(0x7eff000b1650 msgr2=0x7eff000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.239+0000 7eff04e63700 1 -- 172.21.15.6:0/493640956 shutdown_connections 2023-11-28T13:07:54.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.239+0000 7eff04e63700 1 -- 172.21.15.6:0/493640956 wait complete. 2023-11-28T13:07:54.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.240+0000 7eff04e63700 1 Processor -- start 2023-11-28T13:07:54.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.240+0000 7eff04e63700 1 -- start start 2023-11-28T13:07:54.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7eff04e63700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 0x7eff0014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7eff04e63700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff0014c8b0 0x7eff000c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7eff04e63700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff000c3b10 0x7eff000c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7eff04e63700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff000c4600 con 0x7efee8004850 2023-11-28T13:07:54.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7eff04e63700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff000c48e0 con 0x7eff000c3b10 2023-11-28T13:07:54.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7eff04e63700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff000c4bc0 con 0x7eff0014c8b0 2023-11-28T13:07:54.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7efefe59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 0x7eff0014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7efefe59c700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 0x7eff0014c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35772/0 (socket says 172.21.15.6:35772) 2023-11-28T13:07:54.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7efefe59c700 1 -- 172.21.15.6:0/2208836635 learned_addr learned my addr 172.21.15.6:0/2208836635 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:54.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7efefdd9b700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff0014c8b0 0x7eff000c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.250 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7efefed9d700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff000c3b10 0x7eff000c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7efefe59c700 1 -- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff0014c8b0 msgr2=0x7eff000c35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7efefe59c700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7eff0014c8b0 0x7eff000c35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7efefe59c700 1 -- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff000c3b10 msgr2=0x7eff000c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7efefe59c700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff000c3b10 0x7eff000c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.241+0000 7efefe59c700 1 -- 172.21.15.6:0/2208836635 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff000caf20 con 0x7efee8004850 2023-11-28T13:07:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.242+0000 7efefe59c700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 0x7eff0014c2b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7efef000f770 tx=0x7efef000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.242+0000 7efeeeffd700 1 -- 172.21.15.6:0/2208836635 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef00052e0 con 0x7efee8004850 2023-11-28T13:07:54.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.242+0000 7efeeeffd700 1 -- 172.21.15.6:0/2208836635 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef0005440 con 0x7efee8004850 2023-11-28T13:07:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.242+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff000cac40 con 0x7efee8004850 2023-11-28T13:07:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.242+0000 7efeeeffd700 1 -- 172.21.15.6:0/2208836635 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef00038c0 con 0x7efee8004850 2023-11-28T13:07:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.242+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff000cb200 con 0x7efee8004850 2023-11-28T13:07:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.242+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff000c4a50 con 0x7efee8004850 2023-11-28T13:07:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.244+0000 7efeeeffd700 1 -- 172.21.15.6:0/2208836635 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efef0003a20 con 0x7efee8004850 2023-11-28T13:07:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.244+0000 7efeeeffd700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7efee00682f0 0x7efee006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.244+0000 7efeeeffd700 1 -- 172.21.15.6:0/2208836635 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7efef0086250 con 0x7efee8004850 2023-11-28T13:07:54.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.244+0000 7efefdd9b700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7efee00682f0 0x7efee006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.244+0000 7efeeeffd700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x7efee006eb90 0x7efee0070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.244+0000 7efeeeffd700 1 -- 172.21.15.6:0/2208836635 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efee00716b0 con 0x7efee006eb90 2023-11-28T13:07:54.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.244+0000 7efeeeffd700 1 -- 172.21.15.6:0/2208836635 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7efef0086690 con 0x7efee8004850 2023-11-28T13:07:54.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.245+0000 7efefdd9b700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7efee00682f0 0x7efee006a7a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efef40090b0 tx=0x7efef401e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.245+0000 7efefed9d700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x7efee006eb90 0x7efee0070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.245+0000 7efefed9d700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x7efee006eb90 0x7efee0070fe0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.246+0000 7efeeeffd700 1 -- 172.21.15.6:0/2208836635 <== osd.5 v2:172.21.15.61:6808/103780 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efee00716b0 con 0x7efee006eb90 2023-11-28T13:07:54.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.260+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 --> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7eff000739f0 con 0x7efee006eb90 2023-11-28T13:07:54.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7efeeeffd700 1 -- 172.21.15.6:0/2208836635 <== osd.5 v2:172.21.15.61:6808/103780 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7eff000739f0 con 0x7efee006eb90 2023-11-28T13:07:54.262 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:07:54.262 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:54.262 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:07:54.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x7efee006eb90 msgr2=0x7efee0070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7eff04e63700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.61:6808/103780,v1:172.21.15.61:6809/103780] conn(0x7efee006eb90 0x7efee0070fe0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7efee00682f0 msgr2=0x7efee006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7eff04e63700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7efee00682f0 0x7efee006a7a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efef40090b0 tx=0x7efef401e040 comp rx=0 tx=0).stop 2023-11-28T13:07:54.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 msgr2=0x7eff0014c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7eff04e63700 1 --2- 172.21.15.6:0/2208836635 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efee8004850 0x7eff0014c2b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7efef000f770 tx=0x7efef000fa80 comp rx=0 tx=0).stop 2023-11-28T13:07:54.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7efefed9d700 1 -- 172.21.15.6:0/2208836635 reap_dead start 2023-11-28T13:07:54.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 shutdown_connections 2023-11-28T13:07:54.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 >> 172.21.15.6:0/2208836635 conn(0x7eff000b1650 msgr2=0x7eff001590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 shutdown_connections 2023-11-28T13:07:54.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.261+0000 7eff04e63700 1 -- 172.21.15.6:0/2208836635 wait complete. 2023-11-28T13:07:54.274 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T13:07:54.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.399+0000 7f5cb86f5700 1 Processor -- start 2023-11-28T13:07:54.400 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.399+0000 7f5cb86f5700 1 -- start start 2023-11-28T13:07:54.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.399+0000 7f5cb86f5700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f5cb00cb230 0x7f5cb00ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.402 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.399+0000 7f5cb86f5700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb00cfb90 con 0x7f5cb015a6c0 2023-11-28T13:07:54.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.399+0000 7f5cb86f5700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb00cb680 con 0x7f5cb00b2660 2023-11-28T13:07:54.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.399+0000 7f5cb86f5700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb00cb930 con 0x7f5cb00cb230 2023-11-28T13:07:54.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.400+0000 7f5cb6491700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f5cb015a6c0 0x7f5cb00d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51334/0 (socket says 172.21.15.6:51334) 2023-11-28T13:07:54.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.400+0000 7f5cb6491700 1 -- 172.21.15.6:0/2033392828 learned_addr learned my addr 172.21.15.6:0/2033392828 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:54.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.400+0000 7f5cb5c90700 1 --2- 172.21.15.6:0/2033392828 >> v2:172.21.15.61:3300/0 conn(0x7f5cb00cb230 0x7f5cb00ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.400+0000 7f5cb548f700 1 -- 172.21.15.6:0/2033392828 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905528882 0 0) 0x7f5cb00cfb90 con 0x7f5cb015a6c0 2023-11-28T13:07:54.403 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.400+0000 7f5cb548f700 1 -- 172.21.15.6:0/2033392828 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c9c0035b0 con 0x7f5cb015a6c0 2023-11-28T13:07:54.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.400+0000 7f5cb548f700 1 -- 172.21.15.6:0/2033392828 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3182749172 0 0) 0x7f5c9c0035b0 con 0x7f5cb015a6c0 2023-11-28T13:07:54.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.400+0000 7f5cb5c90700 1 -- 172.21.15.6:0/2033392828 >> v1:172.21.15.61:6789/0 conn(0x7f5cb00b2660 legacy=0x7f5cb00b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.400+0000 7f5cb5c90700 1 -- 172.21.15.6:0/2033392828 >> v1:172.21.15.6:6789/0 conn(0x7f5cb015a6c0 legacy=0x7f5cb00d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.400+0000 7f5cb5c90700 1 -- 172.21.15.6:0/2033392828 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb00cf260 con 0x7f5cb00cb230 2023-11-28T13:07:54.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.400+0000 7f5cb5c90700 1 --2- 172.21.15.6:0/2033392828 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cb00cb230 0x7f5cb00ccea0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009840 tx=0x7f5cac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960259c810a1b13a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.401+0000 7f5cb548f700 1 -- 172.21.15.6:0/2033392828 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4001670 con 0x7f5cb00cb230 2023-11-28T13:07:54.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.401+0000 7f5cb548f700 1 -- 172.21.15.6:0/2033392828 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac004030 con 0x7f5cb00cb230 2023-11-28T13:07:54.404 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.401+0000 7f5cb548f700 1 -- 172.21.15.6:0/2033392828 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0043a0 con 0x7f5cb00cb230 2023-11-28T13:07:54.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.401+0000 7f5cb86f5700 1 -- 172.21.15.6:0/2033392828 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cb00cb230 msgr2=0x7f5cb00ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.401+0000 7f5cb86f5700 1 --2- 172.21.15.6:0/2033392828 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cb00cb230 0x7f5cb00ccea0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009840 tx=0x7f5cac009b50 comp rx=0 tx=0).stop 2023-11-28T13:07:54.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.401+0000 7f5cb86f5700 1 -- 172.21.15.6:0/2033392828 shutdown_connections 2023-11-28T13:07:54.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.401+0000 7f5cb86f5700 1 --2- 172.21.15.6:0/2033392828 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cb00cb230 0x7f5cb00ccea0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.401+0000 7f5cb86f5700 1 -- 172.21.15.6:0/2033392828 >> 172.21.15.6:0/2033392828 conn(0x7f5cb00cd660 msgr2=0x7f5cb00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.401+0000 7f5cb86f5700 1 -- 172.21.15.6:0/2033392828 shutdown_connections 2023-11-28T13:07:54.405 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.401+0000 7f5cb86f5700 1 -- 172.21.15.6:0/2033392828 wait complete. 2023-11-28T13:07:54.407 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.403+0000 7f5cb86f5700 1 Processor -- start 2023-11-28T13:07:54.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.403+0000 7f5cb86f5700 1 -- start start 2023-11-28T13:07:54.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb86f5700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cb00b2660 0x7f5cb0151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb86f5700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cb00cb230 0x7f5cb0153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb86f5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cb015a6c0 0x7f5cb00cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb86f5700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb015da40 con 0x7f5cb00cb230 2023-11-28T13:07:54.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb86f5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb0154d10 con 0x7f5cb015a6c0 2023-11-28T13:07:54.408 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb86f5700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb0154ff0 con 0x7f5cb00b2660 2023-11-28T13:07:54.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb5c90700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cb00cb230 0x7f5cb0153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb5c90700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cb00cb230 0x7f5cb0153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35790/0 (socket says 172.21.15.6:35790) 2023-11-28T13:07:54.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb5c90700 1 -- 172.21.15.6:0/1322699494 learned_addr learned my addr 172.21.15.6:0/1322699494 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:54.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb6491700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cb00b2660 0x7f5cb0151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb6c92700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cb015a6c0 0x7f5cb00cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb5c90700 1 -- 172.21.15.6:0/1322699494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cb00b2660 msgr2=0x7f5cb0151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb5c90700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cb00b2660 0x7f5cb0151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.409 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb5c90700 1 -- 172.21.15.6:0/1322699494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cb015a6c0 msgr2=0x7f5cb00cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb5c90700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cb015a6c0 0x7f5cb00cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb5c90700 1 -- 172.21.15.6:0/1322699494 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0094a0 con 0x7f5cb00cb230 2023-11-28T13:07:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb6491700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cb00b2660 0x7f5cb0151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.404+0000 7f5cb6c92700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cb015a6c0 0x7f5cb00cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.405+0000 7f5cb5c90700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cb00cb230 0x7f5cb0153a00 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009b90 tx=0x7f5cac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.405+0000 7f5ca2ffd700 1 -- 172.21.15.6:0/1322699494 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac003ea0 con 0x7f5cb00cb230 2023-11-28T13:07:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.405+0000 7f5ca2ffd700 1 -- 172.21.15.6:0/1322699494 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac01f070 con 0x7f5cb00cb230 2023-11-28T13:07:54.410 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.405+0000 7f5cb86f5700 1 -- 172.21.15.6:0/1322699494 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb00cf820 con 0x7f5cb00cb230 2023-11-28T13:07:54.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.405+0000 7f5ca2ffd700 1 -- 172.21.15.6:0/1322699494 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac011850 con 0x7f5cb00cb230 2023-11-28T13:07:54.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.405+0000 7f5cb86f5700 1 -- 172.21.15.6:0/1322699494 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb00cf540 con 0x7f5cb00cb230 2023-11-28T13:07:54.411 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.406+0000 7f5cb86f5700 1 -- 172.21.15.6:0/1322699494 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb00cd3e0 con 0x7f5cb00cb230 2023-11-28T13:07:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.407+0000 7f5ca2ffd700 1 -- 172.21.15.6:0/1322699494 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5cac017020 con 0x7f5cb00cb230 2023-11-28T13:07:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.407+0000 7f5ca2ffd700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c940682f0 0x7f5c9406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.407+0000 7f5ca2ffd700 1 -- 172.21.15.6:0/1322699494 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f5cac021070 con 0x7f5cb00cb230 2023-11-28T13:07:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.407+0000 7f5cb6491700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c940682f0 0x7f5c9406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.412 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.408+0000 7f5cb6491700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c940682f0 0x7f5c9406a7a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40011e0 tx=0x7f5ca4001da0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.413 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.411+0000 7f5ca2ffd700 1 -- 172.21.15.6:0/1322699494 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5cac054270 con 0x7f5cb00cb230 2023-11-28T13:07:54.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.602+0000 7f5cb86f5700 1 -- 172.21.15.6:0/1322699494 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cb00675a0 con 0x7f5c940682f0 2023-11-28T13:07:54.604 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.603+0000 7f5ca2ffd700 1 -- 172.21.15.6:0/1322699494 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5cb00675a0 con 0x7f5c940682f0 2023-11-28T13:07:54.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 -- 172.21.15.6:0/1322699494 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c940682f0 msgr2=0x7f5c9406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.606 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c940682f0 0x7f5c9406a7a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40011e0 tx=0x7f5ca4001da0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 -- 172.21.15.6:0/1322699494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cb00cb230 msgr2=0x7f5cb0153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cb00cb230 0x7f5cb0153a00 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009b90 tx=0x7f5cac004750 comp rx=0 tx=0).stop 2023-11-28T13:07:54.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 -- 172.21.15.6:0/1322699494 shutdown_connections 2023-11-28T13:07:54.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5cb015a6c0 0x7f5cb00cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f5c940682f0 0x7f5c9406a7a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f5cb00b2660 0x7f5cb0151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 --2- 172.21.15.6:0/1322699494 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cb00cb230 0x7f5cb0153a00 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.607 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 -- 172.21.15.6:0/1322699494 >> 172.21.15.6:0/1322699494 conn(0x7f5cb00cd660 msgr2=0x7f5cb00b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 -- 172.21.15.6:0/1322699494 shutdown_connections 2023-11-28T13:07:54.608 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.605+0000 7f5cb86f5700 1 -- 172.21.15.6:0/1322699494 wait complete. 2023-11-28T13:07:54.608 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T13:07:54.618 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T13:07:54.619 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:54.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.746+0000 7f92e7420700 1 Processor -- start 2023-11-28T13:07:54.748 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.746+0000 7f92e7420700 1 -- start start 2023-11-28T13:07:54.750 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.747+0000 7f92e7420700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f92e0161090 0x7f92e00ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.747+0000 7f92e7420700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e00caeb0 0x7f92e00cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.747+0000 7f92e7420700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e00ce340 con 0x7f92e015c990 2023-11-28T13:07:54.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.747+0000 7f92e7420700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e00cebe0 con 0x7f92e0161090 2023-11-28T13:07:54.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.747+0000 7f92e7420700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00cf780 con 0x7f92e00caeb0 2023-11-28T13:07:54.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.747+0000 7f92e49bb700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f92e0161090 0x7f92e00ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.747+0000 7f92e49bb700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f92e0161090 0x7f92e00ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35796/0 (socket says 172.21.15.6:35796) 2023-11-28T13:07:54.751 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.747+0000 7f92e49bb700 1 -- 172.21.15.6:0/1078028888 learned_addr learned my addr 172.21.15.6:0/1078028888 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:54.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.747+0000 7f92e51bc700 1 --1- 172.21.15.6:0/1078028888 >> v1:172.21.15.6:6789/0 conn(0x7f92e015c990 0x7f92e00cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51348/0 (socket says 172.21.15.6:51348) 2023-11-28T13:07:54.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92e59bd700 1 --2- 172.21.15.6:0/1078028888 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e00caeb0 0x7f92e00cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92e49bb700 1 -- 172.21.15.6:0/1078028888 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e00caeb0 msgr2=0x7f92e00cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92e49bb700 1 --2- 172.21.15.6:0/1078028888 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e00caeb0 0x7f92e00cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.752 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92e49bb700 1 -- 172.21.15.6:0/1078028888 >> v1:172.21.15.6:6789/0 conn(0x7f92e015c990 legacy=0x7f92e00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92e49bb700 1 -- 172.21.15.6:0/1078028888 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00cb3f0 con 0x7f92e0161090 2023-11-28T13:07:54.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92d7fff700 1 -- 172.21.15.6:0/1078028888 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835058649 0 0) 0x7f92e00ce340 con 0x7f92e015c990 2023-11-28T13:07:54.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92e49bb700 1 --2- 172.21.15.6:0/1078028888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e0161090 0x7f92e00ca970 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0098f0 tx=0x7f92dc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1036ee33cbc5c951 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.754 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92d7fff700 1 -- 172.21.15.6:0/1078028888 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc00b8a0 con 0x7f92e0161090 2023-11-28T13:07:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92d7fff700 1 -- 172.21.15.6:0/1078028888 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc00ba00 con 0x7f92e0161090 2023-11-28T13:07:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92d7fff700 1 -- 172.21.15.6:0/1078028888 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc00bd90 con 0x7f92e0161090 2023-11-28T13:07:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92e7420700 1 -- 172.21.15.6:0/1078028888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e0161090 msgr2=0x7f92e00ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.748+0000 7f92e7420700 1 --2- 172.21.15.6:0/1078028888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e0161090 0x7f92e00ca970 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0098f0 tx=0x7f92dc009c00 comp rx=0 tx=0).stop 2023-11-28T13:07:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.749+0000 7f92e7420700 1 -- 172.21.15.6:0/1078028888 shutdown_connections 2023-11-28T13:07:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.749+0000 7f92e7420700 1 --2- 172.21.15.6:0/1078028888 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e00caeb0 0x7f92e00cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.755 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.749+0000 7f92e7420700 1 --2- 172.21.15.6:0/1078028888 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e0161090 0x7f92e00ca970 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.749+0000 7f92e7420700 1 -- 172.21.15.6:0/1078028888 >> 172.21.15.6:0/1078028888 conn(0x7f92e00b1650 msgr2=0x7f92e00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.749+0000 7f92e7420700 1 -- 172.21.15.6:0/1078028888 shutdown_connections 2023-11-28T13:07:54.756 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.749+0000 7f92e7420700 1 -- 172.21.15.6:0/1078028888 wait complete. 2023-11-28T13:07:54.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.750+0000 7f92e7420700 1 Processor -- start 2023-11-28T13:07:54.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.750+0000 7f92e7420700 1 -- start start 2023-11-28T13:07:54.759 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e7420700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92e00caeb0 0x7f92e0151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e7420700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e015c990 0x7f92e00ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e7420700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e0161090 0x7f92e00abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e7420700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00c80f0 con 0x7f92e015c990 2023-11-28T13:07:54.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e7420700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0151590 con 0x7f92e00caeb0 2023-11-28T13:07:54.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e7420700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00c74e0 con 0x7f92e0161090 2023-11-28T13:07:54.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e49bb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e015c990 0x7f92e00ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.760 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e49bb700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e015c990 0x7f92e00ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35804/0 (socket says 172.21.15.6:35804) 2023-11-28T13:07:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e49bb700 1 -- 172.21.15.6:0/4216704702 learned_addr learned my addr 172.21.15.6:0/4216704702 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e51bc700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92e00caeb0 0x7f92e0151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e49bb700 1 -- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e0161090 msgr2=0x7f92e00abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e49bb700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f92e0161090 0x7f92e00abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e49bb700 1 -- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92e00caeb0 msgr2=0x7f92e0151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e49bb700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92e00caeb0 0x7f92e0151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e49bb700 1 -- 172.21.15.6:0/4216704702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc009550 con 0x7f92e015c990 2023-11-28T13:07:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e51bc700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92e00caeb0 0x7f92e0151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:54.761 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e49bb700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e015c990 0x7f92e00ab8c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f92e006eac0 tx=0x7f92dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92d5ffb700 1 -- 172.21.15.6:0/4216704702 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc004380 con 0x7f92e015c990 2023-11-28T13:07:54.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92d5ffb700 1 -- 172.21.15.6:0/4216704702 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc01f070 con 0x7f92e015c990 2023-11-28T13:07:54.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e00ce870 con 0x7f92e015c990 2023-11-28T13:07:54.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.751+0000 7f92d5ffb700 1 -- 172.21.15.6:0/4216704702 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0036a0 con 0x7f92e015c990 2023-11-28T13:07:54.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.752+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e00ce590 con 0x7f92e015c990 2023-11-28T13:07:54.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.752+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f92e00cc850 con 0x7f92e015c990 2023-11-28T13:07:54.762 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.753+0000 7f92d5ffb700 1 -- 172.21.15.6:0/4216704702 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f92dc02a020 con 0x7f92e015c990 2023-11-28T13:07:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.753+0000 7f92d5ffb700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c40682f0 0x7f92c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.753+0000 7f92d5ffb700 1 -- 172.21.15.6:0/4216704702 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f92dc086f70 con 0x7f92e015c990 2023-11-28T13:07:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.753+0000 7f92e51bc700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c40682f0 0x7f92c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.754+0000 7f92d5ffb700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x7f92c406eb90 0x7f92c4070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.754+0000 7f92d5ffb700 1 -- 172.21.15.6:0/4216704702 --> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f92c40716b0 con 0x7f92c406eb90 2023-11-28T13:07:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.754+0000 7f92d5ffb700 1 -- 172.21.15.6:0/4216704702 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92dc0895c0 con 0x7f92e015c990 2023-11-28T13:07:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.754+0000 7f92e59bd700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x7f92c406eb90 0x7f92c4070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.754+0000 7f92e51bc700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c40682f0 0x7f92c406a7a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f92d0000f60 tx=0x7f92d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.754+0000 7f92e59bd700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x7f92c406eb90 0x7f92c4070fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.764 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.755+0000 7f92d5ffb700 1 -- 172.21.15.6:0/4216704702 <== osd.6 v2:172.21.15.61:6800/103779 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f92c40716b0 con 0x7f92c406eb90 2023-11-28T13:07:54.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.769+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 --> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f92e00739f0 con 0x7f92c406eb90 2023-11-28T13:07:54.771 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92d5ffb700 1 -- 172.21.15.6:0/4216704702 <== osd.6 v2:172.21.15.61:6800/103779 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f92e00739f0 con 0x7f92c406eb90 2023-11-28T13:07:54.771 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:07:54.771 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:54.772 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:07:54.772 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x7f92c406eb90 msgr2=0x7f92c4070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e7420700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.61:6800/103779,v1:172.21.15.61:6801/103779] conn(0x7f92c406eb90 0x7f92c4070fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c40682f0 msgr2=0x7f92c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e7420700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f92c40682f0 0x7f92c406a7a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f92d0000f60 tx=0x7f92d0009040 comp rx=0 tx=0).stop 2023-11-28T13:07:54.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e015c990 msgr2=0x7f92e00ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e7420700 1 --2- 172.21.15.6:0/4216704702 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f92e015c990 0x7f92e00ab8c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f92e006eac0 tx=0x7f92dc004750 comp rx=0 tx=0).stop 2023-11-28T13:07:54.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e59bd700 1 -- 172.21.15.6:0/4216704702 reap_dead start 2023-11-28T13:07:54.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 shutdown_connections 2023-11-28T13:07:54.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 >> 172.21.15.6:0/4216704702 conn(0x7f92e00b1650 msgr2=0x7f92e00cf070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 shutdown_connections 2023-11-28T13:07:54.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.770+0000 7f92e7420700 1 -- 172.21.15.6:0/4216704702 wait complete. 2023-11-28T13:07:54.783 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T13:07:54.909 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.908+0000 7fb837ae2700 1 Processor -- start 2023-11-28T13:07:54.910 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.908+0000 7fb837ae2700 1 -- start start 2023-11-28T13:07:54.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb837ae2700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fb830159590 0x7fb83006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb837ae2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83015db70 0x7fb8300ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb837ae2700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83015d120 con 0x7fb8300ce890 2023-11-28T13:07:54.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb837ae2700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb83015d6e0 con 0x7fb830159590 2023-11-28T13:07:54.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb837ae2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300aef60 con 0x7fb83015db70 2023-11-28T13:07:54.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb83607f700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb8300ce890 0x7fb8300caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51368/0 (socket says 172.21.15.6:51368) 2023-11-28T13:07:54.914 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb83607f700 1 -- 172.21.15.6:0/2798719257 learned_addr learned my addr 172.21.15.6:0/2798719257 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:54.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb83587e700 1 --2- 172.21.15.6:0/2798719257 >> v2:172.21.15.61:3300/0 conn(0x7fb830159590 0x7fb83006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb83507d700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83015db70 0x7fb8300ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798676643 0 0) 0x7fb83015d120 con 0x7fb8300ce890 2023-11-28T13:07:54.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.909+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb81c0035b0 con 0x7fb8300ce890 2023-11-28T13:07:54.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1615409833 0 0) 0x7fb81c0035b0 con 0x7fb8300ce890 2023-11-28T13:07:54.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83015db70 msgr2=0x7fb8300ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83015db70 0x7fb8300ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.915 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 >> v2:172.21.15.61:3300/0 conn(0x7fb830159590 msgr2=0x7fb83006e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 --2- 172.21.15.6:0/2798719257 >> v2:172.21.15.61:3300/0 conn(0x7fb830159590 0x7fb83006e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300af240 con 0x7fb8300ce890 2023-11-28T13:07:54.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 <== mon.0 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1001986271 0 0) 0x7fb8280019f0 con 0x7fb8300ce890 2023-11-28T13:07:54.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 >> v1:172.21.15.6:6789/0 conn(0x7fb8300ce890 legacy=0x7fb8300caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83015db70 0x7fb81c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.916 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb81c003f80 0x7fb81c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb81c004d90 0x7fb81c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83015d6e0 con 0x7fb81c004d90 2023-11-28T13:07:54.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300aef60 con 0x7fb81c003f80 2023-11-28T13:07:54.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c00ba10 con 0x7fb83015db70 2023-11-28T13:07:54.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83607f700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb81c004d90 0x7fb81c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83587e700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83015db70 0x7fb81c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.917 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83507d700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb81c003f80 0x7fb81c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83607f700 1 -- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83015db70 msgr2=0x7fb81c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83607f700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83015db70 0x7fb81c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83607f700 1 -- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb81c003f80 msgr2=0x7fb81c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:54.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83607f700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb81c003f80 0x7fb81c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.910+0000 7fb83607f700 1 -- 172.21.15.6:0/2798719257 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0035b0 con 0x7fb81c004d90 2023-11-28T13:07:54.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb83607f700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb81c004d90 0x7fb81c005200 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb82800a830 tx=0x7fb81c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ace004829b18b9b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800a320 con 0x7fb81c004d90 2023-11-28T13:07:54.918 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb828012600 con 0x7fb81c004d90 2023-11-28T13:07:54.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb83487c700 1 -- 172.21.15.6:0/2798719257 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8280129b0 con 0x7fb81c004d90 2023-11-28T13:07:54.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb837ae2700 1 -- 172.21.15.6:0/2798719257 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb81c004d90 msgr2=0x7fb81c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb837ae2700 1 --2- 172.21.15.6:0/2798719257 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb81c004d90 0x7fb81c005200 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb82800a830 tx=0x7fb81c005880 comp rx=0 tx=0).stop 2023-11-28T13:07:54.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb83607f700 1 -- 172.21.15.6:0/2798719257 reap_dead start 2023-11-28T13:07:54.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb837ae2700 1 -- 172.21.15.6:0/2798719257 shutdown_connections 2023-11-28T13:07:54.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb837ae2700 1 -- 172.21.15.6:0/2798719257 >> 172.21.15.6:0/2798719257 conn(0x7fb8300b1650 msgr2=0x7fb8300b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:54.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb837ae2700 1 -- 172.21.15.6:0/2798719257 shutdown_connections 2023-11-28T13:07:54.919 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.911+0000 7fb837ae2700 1 -- 172.21.15.6:0/2798719257 wait complete. 2023-11-28T13:07:54.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.912+0000 7fb837ae2700 1 Processor -- start 2023-11-28T13:07:54.920 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.913+0000 7fb837ae2700 1 -- start start 2023-11-28T13:07:54.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.913+0000 7fb837ae2700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83014bd60 0x7fb83014c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.913+0000 7fb837ae2700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb83015bb40 0x7fb8300c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.922 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.913+0000 7fb837ae2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb83015a500 0x7fb8300b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.913+0000 7fb837ae2700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300c5bc0 con 0x7fb83015bb40 2023-11-28T13:07:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.913+0000 7fb837ae2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83015aaa0 con 0x7fb83015a500 2023-11-28T13:07:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.913+0000 7fb837ae2700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83015ad80 con 0x7fb83014bd60 2023-11-28T13:07:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.913+0000 7fb83507d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb83015bb40 0x7fb8300c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.913+0000 7fb83507d700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb83015bb40 0x7fb8300c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35826/0 (socket says 172.21.15.6:35826) 2023-11-28T13:07:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.913+0000 7fb83507d700 1 -- 172.21.15.6:0/779269820 learned_addr learned my addr 172.21.15.6:0/779269820 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb83587e700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83014bd60 0x7fb83014c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.923 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb83607f700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb83015a500 0x7fb8300b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb83507d700 1 -- 172.21.15.6:0/779269820 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83014bd60 msgr2=0x7fb83014c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb83507d700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83014bd60 0x7fb83014c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb83507d700 1 -- 172.21.15.6:0/779269820 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb83015a500 msgr2=0x7fb8300b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb83507d700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb83015a500 0x7fb8300b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb83507d700 1 -- 172.21.15.6:0/779269820 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb830159960 con 0x7fb83015bb40 2023-11-28T13:07:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb83587e700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83014bd60 0x7fb83014c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb83607f700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb83015a500 0x7fb8300b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:54.924 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb83507d700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb83015bb40 0x7fb8300c5680 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0119a0 tx=0x7fb82c011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb8267fc700 1 -- 172.21.15.6:0/779269820 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00f940 con 0x7fb83015bb40 2023-11-28T13:07:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb8267fc700 1 -- 172.21.15.6:0/779269820 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c00faa0 con 0x7fb83015bb40 2023-11-28T13:07:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb837ae2700 1 -- 172.21.15.6:0/779269820 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83015d370 con 0x7fb83015bb40 2023-11-28T13:07:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb8267fc700 1 -- 172.21.15.6:0/779269820 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00fe30 con 0x7fb83015bb40 2023-11-28T13:07:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.914+0000 7fb837ae2700 1 -- 172.21.15.6:0/779269820 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83015df40 con 0x7fb83015bb40 2023-11-28T13:07:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.915+0000 7fb837ae2700 1 -- 172.21.15.6:0/779269820 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8300675a0 con 0x7fb83015bb40 2023-11-28T13:07:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.916+0000 7fb8267fc700 1 -- 172.21.15.6:0/779269820 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb82c01d400 con 0x7fb83015bb40 2023-11-28T13:07:54.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.916+0000 7fb8267fc700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb814068320 0x7fb81406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:54.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.916+0000 7fb8267fc700 1 -- 172.21.15.6:0/779269820 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb82c0868b0 con 0x7fb83015bb40 2023-11-28T13:07:54.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.916+0000 7fb83587e700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb814068320 0x7fb81406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:54.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.917+0000 7fb83587e700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb814068320 0x7fb81406a7d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb820009bd0 tx=0x7fb820008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:54.926 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:54.920+0000 7fb8267fc700 1 -- 172.21.15.6:0/779269820 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb82c056e70 con 0x7fb83015bb40 2023-11-28T13:07:55.112 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.111+0000 7fb837ae2700 1 -- 172.21.15.6:0/779269820 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fb83006c1b0 con 0x7fb814068320 2023-11-28T13:07:55.113 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.111+0000 7fb8267fc700 1 -- 172.21.15.6:0/779269820 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb83006c1b0 con 0x7fb814068320 2023-11-28T13:07:55.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.113+0000 7fb837ae2700 1 -- 172.21.15.6:0/779269820 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb814068320 msgr2=0x7fb81406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.113+0000 7fb837ae2700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb814068320 0x7fb81406a7d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb820009bd0 tx=0x7fb820008040 comp rx=0 tx=0).stop 2023-11-28T13:07:55.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.113+0000 7fb837ae2700 1 -- 172.21.15.6:0/779269820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb83015bb40 msgr2=0x7fb8300c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.115 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.113+0000 7fb837ae2700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb83015bb40 0x7fb8300c5680 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0119a0 tx=0x7fb82c011d60 comp rx=0 tx=0).stop 2023-11-28T13:07:55.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.113+0000 7fb837ae2700 1 -- 172.21.15.6:0/779269820 shutdown_connections 2023-11-28T13:07:55.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.113+0000 7fb837ae2700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb83015a500 0x7fb8300b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.113+0000 7fb837ae2700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fb814068320 0x7fb81406a7d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.113+0000 7fb837ae2700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fb83014bd60 0x7fb83014c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.113+0000 7fb837ae2700 1 --2- 172.21.15.6:0/779269820 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb83015bb40 0x7fb8300c5680 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.113+0000 7fb837ae2700 1 -- 172.21.15.6:0/779269820 >> 172.21.15.6:0/779269820 conn(0x7fb8300b1650 msgr2=0x7fb83014ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:55.116 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.114+0000 7fb837ae2700 1 -- 172.21.15.6:0/779269820 shutdown_connections 2023-11-28T13:07:55.117 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.114+0000 7fb837ae2700 1 -- 172.21.15.6:0/779269820 wait complete. 2023-11-28T13:07:55.117 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T13:07:55.127 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T13:07:55.127 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T13:07:55.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.257+0000 7fbb5d7a6700 1 Processor -- start 2023-11-28T13:07:55.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.257+0000 7fbb5d7a6700 1 -- start start 2023-11-28T13:07:55.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb5d7a6700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbb5815c740 0x7fbb5806bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb5d7a6700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7fbb5806c470 0x7fbb5815d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb5d7a6700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb58160ad0 con 0x7fbb58160f60 2023-11-28T13:07:55.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb5d7a6700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5806c8b0 con 0x7fbb5815c740 2023-11-28T13:07:55.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb5d7a6700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5815d760 con 0x7fbb5806c470 2023-11-28T13:07:55.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb56ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbb5815c740 0x7fbb5806bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb567fc700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fbb58160f60 0x7fbb5815cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51384/0 (socket says 172.21.15.6:51384) 2023-11-28T13:07:55.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb567fc700 1 -- 172.21.15.6:0/2907732263 learned_addr learned my addr 172.21.15.6:0/2907732263 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:55.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb577fe700 1 --2- 172.21.15.6:0/2907732263 >> v2:172.21.15.61:3300/0 conn(0x7fbb5806c470 0x7fbb5815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb56ffd700 1 -- 172.21.15.6:0/2907732263 >> v2:172.21.15.61:3300/0 conn(0x7fbb5806c470 msgr2=0x7fbb5815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:55.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb56ffd700 1 --2- 172.21.15.6:0/2907732263 >> v2:172.21.15.61:3300/0 conn(0x7fbb5806c470 0x7fbb5815d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb56ffd700 1 -- 172.21.15.6:0/2907732263 >> v1:172.21.15.6:6789/0 conn(0x7fbb58160f60 legacy=0x7fbb5815cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb56ffd700 1 -- 172.21.15.6:0/2907732263 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5815d9d0 con 0x7fbb5815c740 2023-11-28T13:07:55.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.258+0000 7fbb55ffb700 1 -- 172.21.15.6:0/2907732263 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403916298 0 0) 0x7fbb58160ad0 con 0x7fbb58160f60 2023-11-28T13:07:55.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb56ffd700 1 --2- 172.21.15.6:0/2907732263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb5815c740 0x7fbb5806bf30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbb580b8fc0 tx=0x7fbb4c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff766bc3716385f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb55ffb700 1 -- 172.21.15.6:0/2907732263 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c011830 con 0x7fbb5815c740 2023-11-28T13:07:55.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb55ffb700 1 -- 172.21.15.6:0/2907732263 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c011990 con 0x7fbb5815c740 2023-11-28T13:07:55.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb55ffb700 1 -- 172.21.15.6:0/2907732263 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c011d20 con 0x7fbb5815c740 2023-11-28T13:07:55.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/2907732263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb5815c740 msgr2=0x7fbb5806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb5d7a6700 1 --2- 172.21.15.6:0/2907732263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb5815c740 0x7fbb5806bf30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fbb580b8fc0 tx=0x7fbb4c009b20 comp rx=0 tx=0).stop 2023-11-28T13:07:55.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/2907732263 shutdown_connections 2023-11-28T13:07:55.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb5d7a6700 1 --2- 172.21.15.6:0/2907732263 >> v2:172.21.15.61:3300/0 conn(0x7fbb5806c470 0x7fbb5815d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb5d7a6700 1 --2- 172.21.15.6:0/2907732263 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb5815c740 0x7fbb5806bf30 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/2907732263 >> 172.21.15.6:0/2907732263 conn(0x7fbb580d2e70 msgr2=0x7fbb580d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:55.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/2907732263 shutdown_connections 2023-11-28T13:07:55.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.259+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/2907732263 wait complete. 2023-11-28T13:07:55.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.261+0000 7fbb5d7a6700 1 Processor -- start 2023-11-28T13:07:55.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.261+0000 7fbb5d7a6700 1 -- start start 2023-11-28T13:07:55.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.261+0000 7fbb5d7a6700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb5806c470 0x7fbb581506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.261+0000 7fbb5d7a6700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb5815c740 0x7fbb5815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.261+0000 7fbb5d7a6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb58160f60 0x7fbb5815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.261+0000 7fbb5d7a6700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58150c30 con 0x7fbb5815c740 2023-11-28T13:07:55.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.261+0000 7fbb5d7a6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb580c69a0 con 0x7fbb58160f60 2023-11-28T13:07:55.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.261+0000 7fbb5d7a6700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58153450 con 0x7fbb5806c470 2023-11-28T13:07:55.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb567fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb5815c740 0x7fbb5815ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb567fc700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb5815c740 0x7fbb5815ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35840/0 (socket says 172.21.15.6:35840) 2023-11-28T13:07:55.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb567fc700 1 -- 172.21.15.6:0/350843884 learned_addr learned my addr 172.21.15.6:0/350843884 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb577fe700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb58160f60 0x7fbb5815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb56ffd700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb5806c470 0x7fbb581506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb567fc700 1 -- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb5806c470 msgr2=0x7fbb581506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb567fc700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7fbb5806c470 0x7fbb581506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb567fc700 1 -- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb58160f60 msgr2=0x7fbb5815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb567fc700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb58160f60 0x7fbb5815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb567fc700 1 -- 172.21.15.6:0/350843884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c0094a0 con 0x7fbb5815c740 2023-11-28T13:07:55.269 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb567fc700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb5815c740 0x7fbb5815ee70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c520 tx=0x7fbb4800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb3f7fe700 1 -- 172.21.15.6:0/350843884 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48004c80 con 0x7fbb5815c740 2023-11-28T13:07:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb3f7fe700 1 -- 172.21.15.6:0/350843884 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb48004de0 con 0x7fbb5815c740 2023-11-28T13:07:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb3f7fe700 1 -- 172.21.15.6:0/350843884 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480072d0 con 0x7fbb5815c740 2023-11-28T13:07:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb58160760 con 0x7fbb5815c740 2023-11-28T13:07:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.262+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb58160480 con 0x7fbb5815c740 2023-11-28T13:07:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.263+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb580739f0 con 0x7fbb5815c740 2023-11-28T13:07:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.264+0000 7fbb3f7fe700 1 -- 172.21.15.6:0/350843884 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb4801a020 con 0x7fbb5815c740 2023-11-28T13:07:55.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.264+0000 7fbb3f7fe700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb440682f0 0x7fbb4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.264+0000 7fbb3f7fe700 1 -- 172.21.15.6:0/350843884 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fbb48085e30 con 0x7fbb5815c740 2023-11-28T13:07:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.264+0000 7fbb56ffd700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb440682f0 0x7fbb4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.264+0000 7fbb3f7fe700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] conn(0x7fbb4406eb90 0x7fbb44070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.264+0000 7fbb3f7fe700 1 -- 172.21.15.6:0/350843884 --> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb440716b0 con 0x7fbb4406eb90 2023-11-28T13:07:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.264+0000 7fbb3f7fe700 1 -- 172.21.15.6:0/350843884 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb48086270 con 0x7fbb5815c740 2023-11-28T13:07:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.265+0000 7fbb577fe700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] conn(0x7fbb4406eb90 0x7fbb44070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.265+0000 7fbb56ffd700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb440682f0 0x7fbb4406a7a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0090b0 tx=0x7fbb4c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.265+0000 7fbb577fe700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] conn(0x7fbb4406eb90 0x7fbb44070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.266+0000 7fbb3f7fe700 1 -- 172.21.15.6:0/350843884 <== osd.7 v2:172.21.15.61:6824/103806 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbb440716b0 con 0x7fbb4406eb90 2023-11-28T13:07:55.281 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.280+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 --> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbb580721a0 con 0x7fbb4406eb90 2023-11-28T13:07:55.282 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.281+0000 7fbb3f7fe700 1 -- 172.21.15.6:0/350843884 <== osd.7 v2:172.21.15.61:6824/103806 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbb580721a0 con 0x7fbb4406eb90 2023-11-28T13:07:55.282 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-28T13:07:55.283 INFO:teuthology.orchestra.run.smithi006.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T13:07:55.283 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-28T13:07:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] conn(0x7fbb4406eb90 msgr2=0x7fbb44070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb5d7a6700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.61:6824/103806,v1:172.21.15.61:6825/103806] conn(0x7fbb4406eb90 0x7fbb44070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb440682f0 msgr2=0x7fbb4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb5d7a6700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7fbb440682f0 0x7fbb4406a7a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0090b0 tx=0x7fbb4c008030 comp rx=0 tx=0).stop 2023-11-28T13:07:55.284 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb5815c740 msgr2=0x7fbb5815ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb5d7a6700 1 --2- 172.21.15.6:0/350843884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbb5815c740 0x7fbb5815ee70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c520 tx=0x7fbb4800c830 comp rx=0 tx=0).stop 2023-11-28T13:07:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb577fe700 1 -- 172.21.15.6:0/350843884 reap_dead start 2023-11-28T13:07:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 shutdown_connections 2023-11-28T13:07:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 >> 172.21.15.6:0/350843884 conn(0x7fbb580d2e70 msgr2=0x7fbb5815a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 shutdown_connections 2023-11-28T13:07:55.285 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.282+0000 7fbb5d7a6700 1 -- 172.21.15.6:0/350843884 wait complete. 2023-11-28T13:07:55.295 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T13:07:55.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.420+0000 7f3409493700 1 Processor -- start 2023-11-28T13:07:55.421 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.420+0000 7f3409493700 1 -- start start 2023-11-28T13:07:55.423 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.420+0000 7f3409493700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f340415a230 0x7f340415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.420+0000 7f3409493700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f34040cd4a0 0x7f34040b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.420+0000 7f3409493700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 0x7f3404160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.420+0000 7f3409493700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340415a600 con 0x7f340415a230 2023-11-28T13:07:55.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.420+0000 7f3409493700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34040b5290 con 0x7f34040cd4a0 2023-11-28T13:07:55.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.420+0000 7f3409493700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340406abb0 con 0x7f34040b4e10 2023-11-28T13:07:55.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3402ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f340415a230 0x7f340415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3402ffd700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f340415a230 0x7f340415d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35848/0 (socket says 172.21.15.6:35848) 2023-11-28T13:07:55.424 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3402ffd700 1 -- 172.21.15.6:0/1243762455 learned_addr learned my addr 172.21.15.6:0/1243762455 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:55.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f34037fe700 1 --2- 172.21.15.6:0/1243762455 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 0x7f3404160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f34027fc700 1 --2- 172.21.15.6:0/1243762455 >> v2:172.21.15.61:3300/0 conn(0x7f34040cd4a0 0x7f34040b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3402ffd700 1 -- 172.21.15.6:0/1243762455 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 msgr2=0x7f3404160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:55.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3402ffd700 1 --2- 172.21.15.6:0/1243762455 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 0x7f3404160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3402ffd700 1 -- 172.21.15.6:0/1243762455 >> v2:172.21.15.61:3300/0 conn(0x7f34040cd4a0 msgr2=0x7f34040b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:55.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3402ffd700 1 --2- 172.21.15.6:0/1243762455 >> v2:172.21.15.61:3300/0 conn(0x7f34040cd4a0 0x7f34040b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3402ffd700 1 -- 172.21.15.6:0/1243762455 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340406a280 con 0x7f340415a230 2023-11-28T13:07:55.425 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f34037fe700 1 --2- 172.21.15.6:0/1243762455 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 0x7f3404160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T13:07:55.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3402ffd700 1 --2- 172.21.15.6:0/1243762455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 0x7f340415d6b0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f34040d2e20 tx=0x7f33f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65033eebdc24626e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3401ffb700 1 -- 172.21.15.6:0/1243762455 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400b870 con 0x7f340415a230 2023-11-28T13:07:55.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3401ffb700 1 -- 172.21.15.6:0/1243762455 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f400b9d0 con 0x7f340415a230 2023-11-28T13:07:55.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.421+0000 7f3401ffb700 1 -- 172.21.15.6:0/1243762455 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400bd60 con 0x7f340415a230 2023-11-28T13:07:55.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.422+0000 7f3409493700 1 -- 172.21.15.6:0/1243762455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 msgr2=0x7f340415d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.422+0000 7f3409493700 1 --2- 172.21.15.6:0/1243762455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 0x7f340415d6b0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f34040d2e20 tx=0x7f33f4009bd0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.426 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.422+0000 7f3409493700 1 -- 172.21.15.6:0/1243762455 shutdown_connections 2023-11-28T13:07:55.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.422+0000 7f3409493700 1 --2- 172.21.15.6:0/1243762455 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 0x7f3404160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.422+0000 7f3409493700 1 --2- 172.21.15.6:0/1243762455 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 0x7f340415d6b0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.422+0000 7f3409493700 1 --2- 172.21.15.6:0/1243762455 >> v2:172.21.15.61:3300/0 conn(0x7f34040cd4a0 0x7f34040b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.422+0000 7f3409493700 1 -- 172.21.15.6:0/1243762455 >> 172.21.15.6:0/1243762455 conn(0x7f34040cf8d0 msgr2=0x7f34040cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:55.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.422+0000 7f3409493700 1 -- 172.21.15.6:0/1243762455 shutdown_connections 2023-11-28T13:07:55.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.422+0000 7f3409493700 1 -- 172.21.15.6:0/1243762455 wait complete. 2023-11-28T13:07:55.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.423+0000 7f3409493700 1 Processor -- start 2023-11-28T13:07:55.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.423+0000 7f3409493700 1 -- start start 2023-11-28T13:07:55.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f3409493700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 0x7f34041f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.430 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f3409493700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34040cd4a0 0x7f34040c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f3409493700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 0x7f34040c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f3409493700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404146570 con 0x7f340415a230 2023-11-28T13:07:55.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f3409493700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404150a80 con 0x7f34040cd4a0 2023-11-28T13:07:55.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f3409493700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34040c9740 con 0x7f34040b4e10 2023-11-28T13:07:55.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f34037fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 0x7f34040c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f34037fe700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 0x7f34040c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35858/0 (socket says 172.21.15.6:35858) 2023-11-28T13:07:55.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f34037fe700 1 -- 172.21.15.6:0/3696874484 learned_addr learned my addr 172.21.15.6:0/3696874484 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:55.431 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f34027fc700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34040cd4a0 0x7f34040c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f3402ffd700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 0x7f34041f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f34037fe700 1 -- 172.21.15.6:0/3696874484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 msgr2=0x7f34041f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f34037fe700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 0x7f34041f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f34037fe700 1 -- 172.21.15.6:0/3696874484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34040cd4a0 msgr2=0x7f34040c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f34037fe700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34040cd4a0 0x7f34040c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.424+0000 7f34037fe700 1 -- 172.21.15.6:0/3696874484 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4009550 con 0x7f340415a230 2023-11-28T13:07:55.432 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.425+0000 7f34037fe700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 0x7f34040c9200 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f33fc007e40 tx=0x7f33fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.425+0000 7f33eb7fe700 1 -- 172.21.15.6:0/3696874484 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc00de90 con 0x7f340415a230 2023-11-28T13:07:55.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.425+0000 7f33eb7fe700 1 -- 172.21.15.6:0/3696874484 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc00f040 con 0x7f340415a230 2023-11-28T13:07:55.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.425+0000 7f3409493700 1 -- 172.21.15.6:0/3696874484 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340406a840 con 0x7f340415a230 2023-11-28T13:07:55.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.425+0000 7f33eb7fe700 1 -- 172.21.15.6:0/3696874484 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc003c60 con 0x7f340415a230 2023-11-28T13:07:55.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.425+0000 7f3409493700 1 -- 172.21.15.6:0/3696874484 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340406a560 con 0x7f340415a230 2023-11-28T13:07:55.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.426+0000 7f3409493700 1 -- 172.21.15.6:0/3696874484 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34040cd920 con 0x7f340415a230 2023-11-28T13:07:55.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.426+0000 7f33eb7fe700 1 -- 172.21.15.6:0/3696874484 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f33fc01c030 con 0x7f340415a230 2023-11-28T13:07:55.433 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.427+0000 7f33eb7fe700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33f00682f0 0x7f33f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.427+0000 7f33eb7fe700 1 -- 172.21.15.6:0/3696874484 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f33fc085ad0 con 0x7f340415a230 2023-11-28T13:07:55.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.427+0000 7f3402ffd700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33f00682f0 0x7f33f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.427+0000 7f3402ffd700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33f00682f0 0x7f33f006a7a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f33f4014040 tx=0x7f33f4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.434 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.431+0000 7f33eb7fe700 1 -- 172.21.15.6:0/3696874484 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f33fc0524f0 con 0x7f340415a230 2023-11-28T13:07:55.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.622+0000 7f3409493700 1 -- 172.21.15.6:0/3696874484 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f34040675a0 con 0x7f33f00682f0 2023-11-28T13:07:55.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.622+0000 7f33eb7fe700 1 -- 172.21.15.6:0/3696874484 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f34040675a0 con 0x7f33f00682f0 2023-11-28T13:07:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 -- 172.21.15.6:0/3696874484 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33f00682f0 msgr2=0x7f33f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33f00682f0 0x7f33f006a7a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f33f4014040 tx=0x7f33f4008030 comp rx=0 tx=0).stop 2023-11-28T13:07:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 -- 172.21.15.6:0/3696874484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 msgr2=0x7f34040c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 0x7f34040c9200 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f33fc007e40 tx=0x7f33fc00d3b0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 -- 172.21.15.6:0/3696874484 shutdown_connections 2023-11-28T13:07:55.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f340415a230 0x7f34040c9200 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f33f00682f0 0x7f33f006a7a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f34040b4e10 0x7f34041f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 --2- 172.21.15.6:0/3696874484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f34040cd4a0 0x7f34040c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.628 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 -- 172.21.15.6:0/3696874484 >> 172.21.15.6:0/3696874484 conn(0x7f34040cf8d0 msgr2=0x7f34040cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 -- 172.21.15.6:0/3696874484 shutdown_connections 2023-11-28T13:07:55.629 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.624+0000 7f3409493700 1 -- 172.21.15.6:0/3696874484 wait complete. 2023-11-28T13:07:55.630 INFO:teuthology.orchestra.run.smithi006.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T13:07:55.638 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:07:55.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.761+0000 7f9201719700 1 Processor -- start 2023-11-28T13:07:55.763 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.761+0000 7f9201719700 1 -- start start 2023-11-28T13:07:55.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.762+0000 7f9201719700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc06e400 0x7f91fc15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.766 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.762+0000 7f9201719700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f91fc0ce890 0x7f91fc0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.762+0000 7f9201719700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91fc15e0b0 con 0x7f91fc159590 2023-11-28T13:07:55.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.762+0000 7f9201719700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91fc0cf250 con 0x7f91fc0ce890 2023-11-28T13:07:55.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.762+0000 7f9201719700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc15d120 con 0x7f91fc06e400 2023-11-28T13:07:55.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.762+0000 7f91faffd700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f91fc159590 0x7f91fc0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51414/0 (socket says 172.21.15.6:51414) 2023-11-28T13:07:55.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.762+0000 7f91faffd700 1 -- 172.21.15.6:0/3261339836 learned_addr learned my addr 172.21.15.6:0/3261339836 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:07:55.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91fb7fe700 1 --2- 172.21.15.6:0/3261339836 >> v2:172.21.15.61:3300/0 conn(0x7f91fc0ce890 0x7f91fc0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91fa7fc700 1 --2- 172.21.15.6:0/3261339836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc06e400 0x7f91fc15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.767 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91f9ffb700 1 -- 172.21.15.6:0/3261339836 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892840113 0 0) 0x7f91f0001110 con 0x7f91fc159590 2023-11-28T13:07:55.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91f9ffb700 1 -- 172.21.15.6:0/3261339836 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e40035b0 con 0x7f91fc159590 2023-11-28T13:07:55.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91fb7fe700 1 -- 172.21.15.6:0/3261339836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc06e400 msgr2=0x7f91fc15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:07:55.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91fb7fe700 1 --2- 172.21.15.6:0/3261339836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc06e400 0x7f91fc15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91f9ffb700 1 -- 172.21.15.6:0/3261339836 <== mon.0 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1733313893 0 0) 0x7f91e40035b0 con 0x7f91fc159590 2023-11-28T13:07:55.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91fb7fe700 1 -- 172.21.15.6:0/3261339836 >> v1:172.21.15.6:6789/0 conn(0x7f91fc159590 legacy=0x7f91fc0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91fb7fe700 1 -- 172.21.15.6:0/3261339836 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc15d6e0 con 0x7f91fc0ce890 2023-11-28T13:07:55.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91fa7fc700 1 --2- 172.21.15.6:0/3261339836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc06e400 0x7f91fc15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:07:55.768 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91fb7fe700 1 --2- 172.21.15.6:0/3261339836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91fc0ce890 0x7f91fc0ced10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009d90 tx=0x7f91f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecd082642702a1f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91f9ffb700 1 -- 172.21.15.6:0/3261339836 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f400e070 con 0x7f91fc0ce890 2023-11-28T13:07:55.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.763+0000 7f91f9ffb700 1 -- 172.21.15.6:0/3261339836 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f4003a20 con 0x7f91fc0ce890 2023-11-28T13:07:55.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.764+0000 7f91f9ffb700 1 -- 172.21.15.6:0/3261339836 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0001800 con 0x7f91fc0ce890 2023-11-28T13:07:55.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.764+0000 7f9201719700 1 -- 172.21.15.6:0/3261339836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91fc0ce890 msgr2=0x7f91fc0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.764+0000 7f9201719700 1 --2- 172.21.15.6:0/3261339836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91fc0ce890 0x7f91fc0ced10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009d90 tx=0x7f91f400b330 comp rx=0 tx=0).stop 2023-11-28T13:07:55.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.764+0000 7f9201719700 1 -- 172.21.15.6:0/3261339836 shutdown_connections 2023-11-28T13:07:55.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.764+0000 7f9201719700 1 --2- 172.21.15.6:0/3261339836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91fc0ce890 0x7f91fc0ced10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.769 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.764+0000 7f9201719700 1 --2- 172.21.15.6:0/3261339836 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc06e400 0x7f91fc15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.764+0000 7f9201719700 1 -- 172.21.15.6:0/3261339836 >> 172.21.15.6:0/3261339836 conn(0x7f91fc0b1650 msgr2=0x7f91fc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:55.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.764+0000 7f9201719700 1 -- 172.21.15.6:0/3261339836 shutdown_connections 2023-11-28T13:07:55.770 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.764+0000 7f9201719700 1 -- 172.21.15.6:0/3261339836 wait complete. 2023-11-28T13:07:55.773 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.766+0000 7f9201719700 1 Processor -- start 2023-11-28T13:07:55.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.766+0000 7f9201719700 1 -- start start 2023-11-28T13:07:55.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f9201719700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91fc06e400 0x7f91fc0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f9201719700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91fc0ce890 0x7f91fc152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f9201719700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc159590 0x7f91fc0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.774 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f9201719700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc159e90 con 0x7f91fc06e400 2023-11-28T13:07:55.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f9201719700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc1534b0 con 0x7f91fc0ce890 2023-11-28T13:07:55.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f9201719700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc15a070 con 0x7f91fc159590 2023-11-28T13:07:55.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91faffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91fc06e400 0x7f91fc0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91faffd700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91fc06e400 0x7f91fc0c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35868/0 (socket says 172.21.15.6:35868) 2023-11-28T13:07:55.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91faffd700 1 -- 172.21.15.6:0/2650417864 learned_addr learned my addr 172.21.15.6:0/2650417864 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:07:55.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91fa7fc700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91fc0ce890 0x7f91fc152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.775 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91fb7fe700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc159590 0x7f91fc0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91faffd700 1 -- 172.21.15.6:0/2650417864 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc159590 msgr2=0x7f91fc0c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91faffd700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc159590 0x7f91fc0c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91faffd700 1 -- 172.21.15.6:0/2650417864 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91fc0ce890 msgr2=0x7f91fc152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91faffd700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91fc0ce890 0x7f91fc152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.776 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91faffd700 1 -- 172.21.15.6:0/2650417864 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec0094a0 con 0x7f91fc06e400 2023-11-28T13:07:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91fa7fc700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91fc0ce890 0x7f91fc152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91fb7fe700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc159590 0x7f91fc0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T13:07:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.767+0000 7f91faffd700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91fc06e400 0x7f91fc0c78a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c750 tx=0x7f91f000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.768+0000 7f91e37fe700 1 -- 172.21.15.6:0/2650417864 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0004c80 con 0x7f91fc06e400 2023-11-28T13:07:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.768+0000 7f91e37fe700 1 -- 172.21.15.6:0/2650417864 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f0004de0 con 0x7f91fc06e400 2023-11-28T13:07:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.768+0000 7f9201719700 1 -- 172.21.15.6:0/2650417864 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f4009a70 con 0x7f91fc06e400 2023-11-28T13:07:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.768+0000 7f91e37fe700 1 -- 172.21.15.6:0/2650417864 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f00072f0 con 0x7f91fc06e400 2023-11-28T13:07:55.777 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.768+0000 7f9201719700 1 -- 172.21.15.6:0/2650417864 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc0af240 con 0x7f91fc06e400 2023-11-28T13:07:55.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.769+0000 7f9201719700 1 -- 172.21.15.6:0/2650417864 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91fc0aef60 con 0x7f91fc06e400 2023-11-28T13:07:55.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.769+0000 7f91e37fe700 1 -- 172.21.15.6:0/2650417864 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f91f001a020 con 0x7f91fc06e400 2023-11-28T13:07:55.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.769+0000 7f91e37fe700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91e80682f0 0x7f91e806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:07:55.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.770+0000 7f91e37fe700 1 -- 172.21.15.6:0/2650417864 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f91f00856e0 con 0x7f91fc06e400 2023-11-28T13:07:55.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.770+0000 7f91fa7fc700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91e80682f0 0x7f91e806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:07:55.778 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.770+0000 7f91fa7fc700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91e80682f0 0x7f91e806a7a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0c64b0 tx=0x7f91ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:07:55.779 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.774+0000 7f91e37fe700 1 -- 172.21.15.6:0/2650417864 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f91f0052080 con 0x7f91fc06e400 2023-11-28T13:07:55.965 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.964+0000 7f9201719700 1 -- 172.21.15.6:0/2650417864 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91fc15a1e0 con 0x7f91e80682f0 2023-11-28T13:07:55.967 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.966+0000 7f91e37fe700 1 -- 172.21.15.6:0/2650417864 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73720 (secure 0 0 0) 0x7f91fc15a1e0 con 0x7f91e80682f0 2023-11-28T13:07:55.969 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:07:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 -- 172.21.15.6:0/2650417864 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91e80682f0 msgr2=0x7f91e806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91e80682f0 0x7f91e806a7a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f91fc0c64b0 tx=0x7f91ec008040 comp rx=0 tx=0).stop 2023-11-28T13:07:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 -- 172.21.15.6:0/2650417864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91fc06e400 msgr2=0x7f91fc0c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:07:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91fc06e400 0x7f91fc0c78a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c750 tx=0x7f91f000cb10 comp rx=0 tx=0).stop 2023-11-28T13:07:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 -- 172.21.15.6:0/2650417864 shutdown_connections 2023-11-28T13:07:55.970 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f91e80682f0 0x7f91e806a7a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f91fc159590 0x7f91fc0c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91fc06e400 0x7f91fc0c78a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 --2- 172.21.15.6:0/2650417864 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91fc0ce890 0x7f91fc152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:07:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 -- 172.21.15.6:0/2650417864 >> 172.21.15.6:0/2650417864 conn(0x7f91fc0b1650 msgr2=0x7f91fc0af580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:07:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 -- 172.21.15.6:0/2650417864 shutdown_connections 2023-11-28T13:07:55.971 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:07:55.968+0000 7f9201719700 1 -- 172.21.15.6:0/2650417864 wait complete. 2023-11-28T13:07:55.972 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:07:55.982 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-11-28T13:07:55.052825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82304,"kb_used_data":7936,"kb_used_omap":0,"kb_used_meta":74304,"kb_avail":754892416,"statfs":{"total":773094113280,"available":773009833984,"internally_reserved":0,"allocated":8126464,"data_stored":4449425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":76087296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.852827"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T13:07:46.407696+0000","last_change":"2023-11-28T13:00:15.994077+0000","last_active":"2023-11-28T13:07:46.407696+0000","last_peered":"2023-11-28T13:07:46.407696+0000","last_clean":"2023-11-28T13:07:46.407696+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:07:46.407696+0000","last_undegraded":"2023-11-28T13:07:46.407696+0000","last_fullsized":"2023-11-28T13:07:46.407696+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:48:04.434492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":63,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:47.927381+0000","last_change":"2023-11-28T13:00:15.954328+0000","last_active":"2023-11-28T13:07:47.927381+0000","last_peered":"2023-11-28T13:07:47.927381+0000","last_clean":"2023-11-28T13:07:47.927381+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:07:47.927381+0000","last_undegraded":"2023-11-28T13:07:47.927381+0000","last_fullsized":"2023-11-28T13:07:47.927381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:52.942218+0000","last_change":"2023-11-28T13:07:52.942197+0000","last_active":"2023-11-28T13:07:52.942218+0000","last_peered":"2023-11-28T13:07:52.942218+0000","last_clean":"2023-11-28T13:07:52.942218+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:07:52.942218+0000","last_undegraded":"2023-11-28T13:07:52.942218+0000","last_fullsized":"2023-11-28T13:07:52.942218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:52.942118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:52.942118+0000","last_clean_scrub_stamp":"2023-11-28T13:07:52.942118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:30.095059+0000","scrub_duration":0.0041266669999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":217,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T13:07:46.201338+0000","last_change":"2023-11-28T13:00:16.962425+0000","last_active":"2023-11-28T13:07:46.201338+0000","last_peered":"2023-11-28T13:07:46.201338+0000","last_clean":"2023-11-28T13:07:46.201338+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:07:46.201338+0000","last_undegraded":"2023-11-28T13:07:46.201338+0000","last_fullsized":"2023-11-28T13:07:46.201338+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:15.941055+0000","last_clean_scrub_stamp":"2023-11-28T13:00:15.941055+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:17:50.222286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":68,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T13:07:46.200868+0000","last_change":"2023-11-28T13:00:15.993111+0000","last_active":"2023-11-28T13:07:46.200868+0000","last_peered":"2023-11-28T13:07:46.200868+0000","last_clean":"2023-11-28T13:07:46.200868+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:07:46.200868+0000","last_undegraded":"2023-11-28T13:07:46.200868+0000","last_fullsized":"2023-11-28T13:07:46.200868+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":68,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T13:07:46.200763+0000","last_change":"2023-11-28T13:00:15.993105+0000","last_active":"2023-11-28T13:07:46.200763+0000","last_peered":"2023-11-28T13:07:46.200763+0000","last_clean":"2023-11-28T13:07:46.200763+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:07:46.200763+0000","last_undegraded":"2023-11-28T13:07:46.200763+0000","last_fullsized":"2023-11-28T13:07:46.200763+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":60,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T13:07:46.408416+0000","last_change":"2023-11-28T13:00:15.993035+0000","last_active":"2023-11-28T13:07:46.408416+0000","last_peered":"2023-11-28T13:07:46.408416+0000","last_clean":"2023-11-28T13:07:46.408416+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:07:46.408416+0000","last_undegraded":"2023-11-28T13:07:46.408416+0000","last_fullsized":"2023-11-28T13:07:46.408416+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":59,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T13:07:46.409315+0000","last_change":"2023-11-28T13:00:15.993374+0000","last_active":"2023-11-28T13:07:46.409315+0000","last_peered":"2023-11-28T13:07:46.409315+0000","last_clean":"2023-11-28T13:07:46.409315+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:07:46.409315+0000","last_undegraded":"2023-11-28T13:07:46.409315+0000","last_fullsized":"2023-11-28T13:07:46.409315+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:15:28.109132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":59,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T13:07:46.411096+0000","last_change":"2023-11-28T13:00:15.958060+0000","last_active":"2023-11-28T13:07:46.411096+0000","last_peered":"2023-11-28T13:07:46.411096+0000","last_clean":"2023-11-28T13:07:46.411096+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:07:46.411096+0000","last_undegraded":"2023-11-28T13:07:46.411096+0000","last_fullsized":"2023-11-28T13:07:46.411096+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:00:14.933661+0000","last_clean_scrub_stamp":"2023-11-28T13:00:14.933661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:33:45.215979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11164,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":9728,"kb_avail":94360676,"statfs":{"total":96636764160,"available":96625332224,"internally_reserved":0,"allocated":1462272,"data_stored":999203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9961472},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.49199999999999999},"min":{"1min":0.49399999999999999,"5min":0.373,"15min":0.24099999999999999},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.877},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.53700000000000003},"min":{"1min":0.46899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.52000000000000002},"min":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.313},"max":{"1min":0.87,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.55000000000000004},"min":{"1min":0.50700000000000001,"5min":0.34999999999999998,"15min":0.33000000000000002},"max":{"1min":0.77200000000000002,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60599999999999998,"15min":0.55000000000000004},"min":{"1min":0.53500000000000003,"5min":0.375,"15min":0.36899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.628,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.875,"15min":0.875},"last":0.53100000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.56100000000000005},"min":{"1min":0.49199999999999999,"5min":0.42699999999999999,"15min":0.39300000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.59099999999999997},"min":{"1min":0.49099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82199999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64800000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81899999999999995,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.61799999999999999,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.28499999999999998},"max":{"1min":0.87,"5min":7.9939999999999998,"15min":7.9939999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68200000000000005,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84799999999999998,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.66800000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82799999999999996,"5min":4.133,"15min":4.133},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89000000000000001,"5min":8.3000000000000007,"15min":8.3000000000000007},"last":0.751}]}]},{"osd":6,"up_from":13,"seq":55834574944,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":868352,"data_stored":408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56000000000000005,"15min":0.496},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.629},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57299999999999995,"15min":0.51400000000000001},"min":{"1min":0.34300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41399999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.30499999999999999},"max":{"1min":0.75700000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.42999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.79500000000000004,"15min":0.87},"last":0.51600000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65800000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57399999999999995,"15min":0.53200000000000003},"min":{"1min":0.434,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60799999999999998,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.49299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.71499999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52500000000000002,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.84599999999999997,"15min":0.85399999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55100000000000005,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73299999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.65300000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.53300000000000003,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68899999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.69499999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77800000000000002,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11032,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360808,"statfs":{"total":96636764160,"available":96625467392,"internally_reserved":0,"allocated":1458176,"data_stored":998198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.32800000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.373},"min":{"1min":0.29299999999999998,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.26200000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39800000000000002,"15min":0.36499999999999999},"min":{"1min":0.26200000000000001,"5min":0.252,"15min":0.25},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.78500000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.437,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23699999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.41799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41599999999999998,"15min":0.38400000000000001},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.24099999999999999},"max":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.69699999999999995},"last":0.35799999999999998}]},{"osd":4,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79200000000000004,"5min":0.93200000000000005,"15min":1.1259999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.629,"15min":0.58599999999999997},"min":{"1min":0.50600000000000001,"5min":0.46500000000000002,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61899999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61099999999999999,"15min":0.58099999999999996},"min":{"1min":0.48899999999999999,"5min":0.41299999999999998,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.48099999999999998,"5min":0.42699999999999999,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.98299999999999998,"15min":1.157},"last":0.68600000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.41799999999999998,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.54300000000000004,"5min":0.46200000000000002,"15min":0.39000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0169999999999999,"15min":1.1759999999999999},"last":0.64200000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.625},"min":{"1min":0.52000000000000002,"5min":0.40500000000000003,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.624},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.41599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56699999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361708,"statfs":{"total":96636764160,"available":96626388992,"internally_reserved":0,"allocated":864256,"data_stored":407849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.35399999999999998},"min":{"1min":0.223,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.61399999999999999,"5min":0.623,"15min":0.623},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40899999999999997,"15min":0.36399999999999999},"min":{"1min":0.30199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.311}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41199999999999998,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.443,"15min":0.39500000000000002},"min":{"1min":0.33700000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.66000000000000003},"last":0.375}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66700000000000004,"5min":0.746,"15min":0.746},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47699999999999998,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66900000000000004,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.65800000000000003}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.42799999999999999,"15min":0.42699999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.61399999999999999},"min":{"1min":0.505,"5min":0.44400000000000001,"15min":0.42999999999999999},"max":{"1min":0.83599999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69299999999999995}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.44900000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":1.5509999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.65200000000000002},"min":{"1min":0.57399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.996},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66200000000000003,"15min":0.63},"min":{"1min":0.55500000000000005,"5min":0.45200000000000001,"15min":0.41499999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.98299999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.46999999999999997,"15min":0.40200000000000002},"max":{"1min":0.90200000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574944,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":868352,"data_stored":408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.379,"15min":0.36199999999999999},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.20399999999999999},"max":{"1min":0.46000000000000002,"5min":0.628,"15min":0.70999999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.378},"min":{"1min":0.27800000000000002,"5min":0.24199999999999999,"15min":0.23300000000000001},"max":{"1min":0.47699999999999998,"5min":0.60899999999999999,"15min":0.72299999999999998},"last":0.27100000000000002}]},{"osd":1,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.41799999999999998,"15min":0.40200000000000002},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.24099999999999999},"max":{"1min":0.48199999999999998,"5min":0.63200000000000001,"15min":0.65700000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.443,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.47699999999999998,"5min":0.63200000000000001,"15min":0.69399999999999995},"last":0.30399999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40600000000000003,"15min":0.38100000000000001},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51200000000000001,"5min":0.65400000000000003,"15min":0.73099999999999998},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.47299999999999998,"15min":0.44400000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.53900000000000003,"5min":0.71199999999999997,"15min":0.748},"last":0.377}]},{"osd":4,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.61699999999999999},"min":{"1min":0.46000000000000002,"5min":0.39600000000000002,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":0.93500000000000005,"15min":0.96199999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.624},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.36699999999999999},"max":{"1min":0.874,"5min":0.96299999999999997,"15min":1.0649999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.63400000000000001},"min":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.40300000000000002},"max":{"1min":0.92800000000000005,"5min":0.94099999999999995,"15min":0.96899999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.44600000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.40999999999999998}]},{"osd":6,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.52700000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88400000000000001,"5min":1.022,"15min":1.022},"last":0.435},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67300000000000004,"15min":0.63900000000000001},"min":{"1min":0.499,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.46000000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69599999999999995,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86299999999999999,"5min":1.022,"15min":1.0309999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.65200000000000002},"min":{"1min":0.48099999999999998,"5min":0.42499999999999999,"15min":0.38400000000000001},"max":{"1min":0.84299999999999997,"5min":1.004,"15min":1.004},"last":0.51100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10072,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361768,"statfs":{"total":96636764160,"available":96626450432,"internally_reserved":0,"allocated":868352,"data_stored":408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.189},"max":{"1min":0.47099999999999997,"5min":0.61199999999999999,"15min":0.75800000000000001},"last":0.19700000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39400000000000002,"15min":0.35299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.48999999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.245}]},{"osd":1,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.55800000000000005,"5min":0.64300000000000002,"15min":1.1279999999999999},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.40600000000000003},"min":{"1min":0.318,"5min":0.252,"15min":0.22600000000000001},"max":{"1min":0.52900000000000003,"5min":0.63200000000000001,"15min":0.82799999999999996},"last":0.23200000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.52700000000000002},"min":{"1min":0.314,"5min":0.27000000000000002,"15min":0.24399999999999999},"max":{"1min":0.54600000000000004,"5min":0.59399999999999997,"15min":5.0540000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.432,"15min":0.45100000000000001},"min":{"1min":0.25800000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.67500000000000004,"15min":1.9390000000000001},"last":0.315}]},{"osd":4,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.64800000000000002},"min":{"1min":0.45400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":1.6919999999999999},"last":0.434},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68999999999999995,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.97099999999999997,"15min":1.2},"last":0.443}]},{"osd":5,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.625,"15min":0.57199999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.92500000000000004,"15min":1.169},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65200000000000002,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45200000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.90500000000000003,"5min":0.94599999999999995,"15min":1.6519999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93000000000000005,"5min":1.01,"15min":1.23},"last":0.42599999999999999}]},{"osd":7,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.65400000000000003},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86799999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69499999999999995,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90000000000000002,"5min":1.018,"15min":1.7250000000000001},"last":0.41699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10136,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361704,"statfs":{"total":96636764160,"available":96626384896,"internally_reserved":0,"allocated":868352,"data_stored":408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32000000000000001},"max":{"1min":0.77300000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.873,"15min":0.873},"last":0.442}]},{"osd":1,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60899999999999999,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.34000000000000002},"max":{"1min":0.81000000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.53200000000000003},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35199999999999998},"max":{"1min":0.79000000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46899999999999997}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.36899999999999999,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.85999999999999999},"last":0.497}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.55600000000000005},"min":{"1min":0.45600000000000002,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.879},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.58899999999999997},"min":{"1min":0.442,"5min":0.41199999999999998,"15min":0.372},"max":{"1min":0.81000000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69999999999999996}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51500000000000001,"15min":0.56200000000000006},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66700000000000004,"5min":0.79800000000000004,"15min":0.95199999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54000000000000004,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72299999999999998,"5min":0.77300000000000002,"15min":0.79400000000000004},"last":0.55200000000000005}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.58399999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80000000000000004,"5min":0.82599999999999996,"15min":0.84999999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60199999999999998,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81899999999999995,"5min":0.88500000000000001,"15min":1.117},"last":0.67200000000000004}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59099999999999997,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.74099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9880,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361960,"statfs":{"total":96636764160,"available":96626647040,"internally_reserved":0,"allocated":868352,"data_stored":408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.52300000000000002},"min":{"1min":0.39900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86099999999999999,"5min":0.875,"15min":0.875},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73899999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60799999999999998,"15min":0.55700000000000005},"min":{"1min":0.45600000000000002,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57999999999999996,"15min":0.52300000000000002},"min":{"1min":0.41799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59899999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.50700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63600000000000001,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.41699999999999998,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.59599999999999997},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65700000000000003,"15min":0.64800000000000002},"min":{"1min":0.52200000000000002,"5min":0.41599999999999998,"15min":0.39200000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.624}]},{"osd":4,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.95999999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.61099999999999999},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.84599999999999997,"15min":1.0029999999999999},"last":0.52000000000000002}]},{"osd":6,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69999999999999996,"5min":0.755,"15min":0.85899999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.85399999999999998,"5min":0.91900000000000004,"15min":1.0349999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56599999999999995,"15min":0.59799999999999998},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.89800000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60299999999999998,"15min":0.65000000000000002},"min":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.89100000000000001,"15min":1.052},"last":0.61199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:07:56.031 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-11-28T13:00:14.933661+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=14, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=50, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:56.031 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-11-28T13:00:14.933661+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=14, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=50, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:56.031 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T13:00:15.941055+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=15, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=50, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:56.031 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-11-28T13:00:14.933661+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=14, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=50, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:56.031 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T13:00:14.933661+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=14, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=50, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:56.032 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-11-28T13:00:14.933661+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=14, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=50, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:56.032 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-11-28T13:00:14.933661+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=14, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=50, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:56.032 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-11-28T13:00:14.933661+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=0, tm_sec=14, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=13, tm_min=7, tm_sec=50, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T13:07:56.032 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T13:07:57.648 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:07:57.650 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:08:16.033 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T13:08:16.175 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.172+0000 7f960a543700 1 Processor -- start 2023-11-28T13:08:16.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.172+0000 7f960a543700 1 -- start start 2023-11-28T13:08:16.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.173+0000 7f960a543700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f96041592b0 0x7f96040cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.173+0000 7f960a543700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9604070960 0x7f9604070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.173+0000 7f960a543700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96040712f0 con 0x7f96040ca880 2023-11-28T13:08:16.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.173+0000 7f960a543700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960415a4a0 con 0x7f96041592b0 2023-11-28T13:08:16.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.173+0000 7f960a543700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960415a780 con 0x7f9604070960 2023-11-28T13:08:16.176 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.173+0000 7f9603fff700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f96041592b0 0x7f96040cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.173+0000 7f96037fe700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9604070960 0x7f9604070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.173+0000 7f9608ae0700 1 --1- >> v1:172.21.15.61:6789/0 conn(0x7f96040ca880 0x7f96040caca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.61:6789/0 says I am v1:172.21.15.6:41462/0 (socket says 172.21.15.6:41462) 2023-11-28T13:08:16.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.173+0000 7f9608ae0700 1 -- 172.21.15.6:0/728727370 learned_addr learned my addr 172.21.15.6:0/728727370 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:08:16.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f9602ffd700 1 -- 172.21.15.6:0/728727370 <== mon.1 v1:172.21.15.61:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763663225 0 0) 0x7f95fc001490 con 0x7f96040ca880 2023-11-28T13:08:16.177 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f9602ffd700 1 -- 172.21.15.6:0/728727370 --> v1:172.21.15.61:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ec0035b0 con 0x7f96040ca880 2023-11-28T13:08:16.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f9603fff700 1 -- 172.21.15.6:0/728727370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9604070960 msgr2=0x7f9604070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:16.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f9603fff700 1 --2- 172.21.15.6:0/728727370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9604070960 0x7f9604070db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f9603fff700 1 -- 172.21.15.6:0/728727370 >> v1:172.21.15.61:6789/0 conn(0x7f96040ca880 legacy=0x7f96040caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.178 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f9603fff700 1 -- 172.21.15.6:0/728727370 --> v2:172.21.15.61:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96040ca080 con 0x7f96041592b0 2023-11-28T13:08:16.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f96037fe700 1 --2- 172.21.15.6:0/728727370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9604070960 0x7f9604070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T13:08:16.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f9603fff700 1 --2- 172.21.15.6:0/728727370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96041592b0 0x7f96040cc5b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009840 tx=0x7f95f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c889ac6a6d768c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f9602ffd700 1 -- 172.21.15.6:0/728727370 <== mon.1 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4011830 con 0x7f96041592b0 2023-11-28T13:08:16.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f9602ffd700 1 -- 172.21.15.6:0/728727370 <== mon.1 v2:172.21.15.61:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f4011990 con 0x7f96041592b0 2023-11-28T13:08:16.179 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.174+0000 7f9602ffd700 1 -- 172.21.15.6:0/728727370 <== mon.1 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4011d20 con 0x7f96041592b0 2023-11-28T13:08:16.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.175+0000 7f960a543700 1 -- 172.21.15.6:0/728727370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96041592b0 msgr2=0x7f96040cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.175+0000 7f960a543700 1 --2- 172.21.15.6:0/728727370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96041592b0 0x7f96040cc5b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009840 tx=0x7f95f4009b50 comp rx=0 tx=0).stop 2023-11-28T13:08:16.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.175+0000 7f960a543700 1 -- 172.21.15.6:0/728727370 shutdown_connections 2023-11-28T13:08:16.183 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.175+0000 7f960a543700 1 --2- 172.21.15.6:0/728727370 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96041592b0 0x7f96040cc5b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.175+0000 7f960a543700 1 --2- 172.21.15.6:0/728727370 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f9604070960 0x7f9604070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.175+0000 7f960a543700 1 -- 172.21.15.6:0/728727370 >> 172.21.15.6:0/728727370 conn(0x7f96040ce9e0 msgr2=0x7f96040cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:16.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.175+0000 7f960a543700 1 -- 172.21.15.6:0/728727370 shutdown_connections 2023-11-28T13:08:16.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.175+0000 7f960a543700 1 -- 172.21.15.6:0/728727370 wait complete. 2023-11-28T13:08:16.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.176+0000 7f960a543700 1 Processor -- start 2023-11-28T13:08:16.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.176+0000 7f960a543700 1 -- start start 2023-11-28T13:08:16.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f960a543700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9604070960 0x7f960414dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.184 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f960a543700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96040ca880 0x7f96040cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f960a543700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96041592b0 0x7f960415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f960a543700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040c32b0 con 0x7f96041592b0 2023-11-28T13:08:16.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f960a543700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96040c3520 con 0x7f9604070960 2023-11-28T13:08:16.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f960a543700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96040c3800 con 0x7f96040ca880 2023-11-28T13:08:16.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f9608ae0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96041592b0 0x7f960415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f9608ae0700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96041592b0 0x7f960415ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35886/0 (socket says 172.21.15.6:35886) 2023-11-28T13:08:16.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f9608ae0700 1 -- 172.21.15.6:0/2006551226 learned_addr learned my addr 172.21.15.6:0/2006551226 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:08:16.185 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f9603fff700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9604070960 0x7f960414dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f96037fe700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96040ca880 0x7f96040cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f9608ae0700 1 -- 172.21.15.6:0/2006551226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96040ca880 msgr2=0x7f96040cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f9608ae0700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96040ca880 0x7f96040cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f9608ae0700 1 -- 172.21.15.6:0/2006551226 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9604070960 msgr2=0x7f960414dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f9608ae0700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9604070960 0x7f960414dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.177+0000 7f9608ae0700 1 -- 172.21.15.6:0/2006551226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f80094a0 con 0x7f96041592b0 2023-11-28T13:08:16.186 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.178+0000 7f9608ae0700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96041592b0 0x7f960415ac30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00cb40 tx=0x7f95fc00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.178+0000 7f9600ff9700 1 -- 172.21.15.6:0/2006551226 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00a9e0 con 0x7f96041592b0 2023-11-28T13:08:16.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.178+0000 7f9600ff9700 1 -- 172.21.15.6:0/2006551226 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc00ab40 con 0x7f96041592b0 2023-11-28T13:08:16.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.178+0000 7f960a543700 1 -- 172.21.15.6:0/2006551226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f40094a0 con 0x7f96041592b0 2023-11-28T13:08:16.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.178+0000 7f9600ff9700 1 -- 172.21.15.6:0/2006551226 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc012400 con 0x7f96041592b0 2023-11-28T13:08:16.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.178+0000 7f960a543700 1 -- 172.21.15.6:0/2006551226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96040c9d90 con 0x7f96041592b0 2023-11-28T13:08:16.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.179+0000 7f960a543700 1 -- 172.21.15.6:0/2006551226 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96040ca360 con 0x7f96041592b0 2023-11-28T13:08:16.187 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.179+0000 7f9600ff9700 1 -- 172.21.15.6:0/2006551226 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f95fc019030 con 0x7f96041592b0 2023-11-28T13:08:16.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.180+0000 7f9600ff9700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95f0068320 0x7f95f006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.180+0000 7f9600ff9700 1 -- 172.21.15.6:0/2006551226 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f95fc0852d0 con 0x7f96041592b0 2023-11-28T13:08:16.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.180+0000 7f9603fff700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95f0068320 0x7f95f006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.188 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.180+0000 7f9603fff700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95f0068320 0x7f95f006a7d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000c00 tx=0x7f95f401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.189 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.184+0000 7f9600ff9700 1 -- 172.21.15.6:0/2006551226 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f95fc04b5e0 con 0x7f96041592b0 2023-11-28T13:08:16.375 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.373+0000 7f960a543700 1 -- 172.21.15.6:0/2006551226 --> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f960415d1d0 con 0x7f95f0068320 2023-11-28T13:08:16.377 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.376+0000 7f9600ff9700 1 -- 172.21.15.6:0/2006551226 <== mgr.4107 v2:172.21.15.6:6832/103231 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73920 (secure 0 0 0) 0x7f960415d1d0 con 0x7f95f0068320 2023-11-28T13:08:16.378 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:08:16.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 -- 172.21.15.6:0/2006551226 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95f0068320 msgr2=0x7f95f006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95f0068320 0x7f95f006a7d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f95f4000c00 tx=0x7f95f401c040 comp rx=0 tx=0).stop 2023-11-28T13:08:16.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 -- 172.21.15.6:0/2006551226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96041592b0 msgr2=0x7f960415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96041592b0 0x7f960415ac30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00cb40 tx=0x7f95fc00ce50 comp rx=0 tx=0).stop 2023-11-28T13:08:16.380 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 -- 172.21.15.6:0/2006551226 shutdown_connections 2023-11-28T13:08:16.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96041592b0 0x7f960415ac30 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9604070960 0x7f960414dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f95f0068320 0x7f95f006a7d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 --2- 172.21.15.6:0/2006551226 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f96040ca880 0x7f96040cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 -- 172.21.15.6:0/2006551226 >> 172.21.15.6:0/2006551226 conn(0x7f96040ce9e0 msgr2=0x7f96040b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:16.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 -- 172.21.15.6:0/2006551226 shutdown_connections 2023-11-28T13:08:16.381 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.378+0000 7f960a543700 1 -- 172.21.15.6:0/2006551226 wait complete. 2023-11-28T13:08:16.382 INFO:teuthology.orchestra.run.smithi006.stderr:dumped all 2023-11-28T13:08:16.392 INFO:teuthology.orchestra.run.smithi006.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2023-11-28T13:08:15.056798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82304,"kb_used_data":7936,"kb_used_omap":0,"kb_used_meta":74304,"kb_avail":754892416,"statfs":{"total":773094113280,"available":773009833984,"internally_reserved":0,"allocated":8126464,"data_stored":4449425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":76087296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002169"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:55.905762+0000","last_change":"2023-11-28T13:07:55.905732+0000","last_active":"2023-11-28T13:07:55.905762+0000","last_peered":"2023-11-28T13:07:55.905762+0000","last_clean":"2023-11-28T13:07:55.905762+0000","last_became_active":"2023-11-28T13:00:15.993498+0000","last_became_peered":"2023-11-28T13:00:15.993498+0000","last_unstale":"2023-11-28T13:07:55.905762+0000","last_undegraded":"2023-11-28T13:07:55.905762+0000","last_fullsized":"2023-11-28T13:07:55.905762+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.905637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.905637+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.905637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:50.663438+0000","scrub_duration":0.003576606,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:54.408307+0000","last_change":"2023-11-28T13:07:54.408284+0000","last_active":"2023-11-28T13:07:54.408307+0000","last_peered":"2023-11-28T13:07:54.408307+0000","last_clean":"2023-11-28T13:07:54.408307+0000","last_became_active":"2023-11-28T13:00:15.954016+0000","last_became_peered":"2023-11-28T13:00:15.954016+0000","last_unstale":"2023-11-28T13:07:54.408307+0000","last_undegraded":"2023-11-28T13:07:54.408307+0000","last_fullsized":"2023-11-28T13:07:54.408307+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:54.408179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:54.408179+0000","last_clean_scrub_stamp":"2023-11-28T13:07:54.408179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:39:50.945576+0000","scrub_duration":0.0036219860000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:52.942218+0000","last_change":"2023-11-28T13:07:52.942197+0000","last_active":"2023-11-28T13:07:52.942218+0000","last_peered":"2023-11-28T13:07:52.942218+0000","last_clean":"2023-11-28T13:07:52.942218+0000","last_became_active":"2023-11-28T13:00:15.958238+0000","last_became_peered":"2023-11-28T13:00:15.958238+0000","last_unstale":"2023-11-28T13:07:52.942218+0000","last_undegraded":"2023-11-28T13:07:52.942218+0000","last_fullsized":"2023-11-28T13:07:52.942218+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:52.942118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:52.942118+0000","last_clean_scrub_stamp":"2023-11-28T13:07:52.942118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:02:30.095059+0000","scrub_duration":0.0041266669999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"71'119","reported_seq":234,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:59.400073+0000","last_change":"2023-11-28T13:07:59.400045+0000","last_active":"2023-11-28T13:07:59.400073+0000","last_peered":"2023-11-28T13:07:59.400073+0000","last_clean":"2023-11-28T13:07:59.400073+0000","last_became_active":"2023-11-28T13:00:16.961515+0000","last_became_peered":"2023-11-28T13:00:16.961515+0000","last_unstale":"2023-11-28T13:07:59.400073+0000","last_undegraded":"2023-11-28T13:07:59.400073+0000","last_fullsized":"2023-11-28T13:07:59.400073+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'119","last_scrub_stamp":"2023-11-28T13:07:59.399940+0000","last_deep_scrub":"71'119","last_deep_scrub_stamp":"2023-11-28T13:07:59.399940+0000","last_clean_scrub_stamp":"2023-11-28T13:07:59.399940+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:05:54.157741+0000","scrub_duration":0.0072944100000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":83,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:56.428374+0000","last_change":"2023-11-28T13:07:56.428340+0000","last_active":"2023-11-28T13:07:56.428374+0000","last_peered":"2023-11-28T13:07:56.428374+0000","last_clean":"2023-11-28T13:07:56.428374+0000","last_became_active":"2023-11-28T13:00:15.992798+0000","last_became_peered":"2023-11-28T13:00:15.992798+0000","last_unstale":"2023-11-28T13:07:56.428374+0000","last_undegraded":"2023-11-28T13:07:56.428374+0000","last_fullsized":"2023-11-28T13:07:56.428374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.428231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.428231+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.428231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:37:35.711341+0000","scrub_duration":0.003793785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:58.424837+0000","last_change":"2023-11-28T13:07:58.424806+0000","last_active":"2023-11-28T13:07:58.424837+0000","last_peered":"2023-11-28T13:07:58.424837+0000","last_clean":"2023-11-28T13:07:58.424837+0000","last_became_active":"2023-11-28T13:00:15.992501+0000","last_became_peered":"2023-11-28T13:00:15.992501+0000","last_unstale":"2023-11-28T13:07:58.424837+0000","last_undegraded":"2023-11-28T13:07:58.424837+0000","last_fullsized":"2023-11-28T13:07:58.424837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:58.424696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:58.424696+0000","last_clean_scrub_stamp":"2023-11-28T13:07:58.424696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:59:24.984569+0000","scrub_duration":0.0041058659999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:59.532393+0000","last_change":"2023-11-28T13:07:59.532361+0000","last_active":"2023-11-28T13:07:59.532393+0000","last_peered":"2023-11-28T13:07:59.532393+0000","last_clean":"2023-11-28T13:07:59.532393+0000","last_became_active":"2023-11-28T13:00:15.992699+0000","last_became_peered":"2023-11-28T13:00:15.992699+0000","last_unstale":"2023-11-28T13:07:59.532393+0000","last_undegraded":"2023-11-28T13:07:59.532393+0000","last_fullsized":"2023-11-28T13:07:59.532393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:59.532255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:59.532255+0000","last_clean_scrub_stamp":"2023-11-28T13:07:59.532255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:04:11.181348+0000","scrub_duration":0.0038972159999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:55.454742+0000","last_change":"2023-11-28T13:07:55.454711+0000","last_active":"2023-11-28T13:07:55.454742+0000","last_peered":"2023-11-28T13:07:55.454742+0000","last_clean":"2023-11-28T13:07:55.454742+0000","last_became_active":"2023-11-28T13:00:15.993002+0000","last_became_peered":"2023-11-28T13:00:15.993002+0000","last_unstale":"2023-11-28T13:07:55.454742+0000","last_undegraded":"2023-11-28T13:07:55.454742+0000","last_fullsized":"2023-11-28T13:07:55.454742+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:55.454610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:55.454610+0000","last_clean_scrub_stamp":"2023-11-28T13:07:55.454610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:13:01.461292+0000","scrub_duration":0.0035680429999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T13:07:56.437534+0000","last_change":"2023-11-28T13:07:56.437481+0000","last_active":"2023-11-28T13:07:56.437534+0000","last_peered":"2023-11-28T13:07:56.437534+0000","last_clean":"2023-11-28T13:07:56.437534+0000","last_became_active":"2023-11-28T13:00:15.957797+0000","last_became_peered":"2023-11-28T13:00:15.957797+0000","last_unstale":"2023-11-28T13:07:56.437534+0000","last_undegraded":"2023-11-28T13:07:56.437534+0000","last_fullsized":"2023-11-28T13:07:56.437534+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T13:07:56.437365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T13:07:56.437365+0000","last_clean_scrub_stamp":"2023-11-28T13:07:56.437365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:45.269069+0000","scrub_duration":0.0043014500000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574949,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11164,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":9728,"kb_avail":94360676,"statfs":{"total":96636764160,"available":96625332224,"internally_reserved":0,"allocated":1462272,"data_stored":999203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9961472},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.58599999999999997,"15min":0.49199999999999999},"min":{"1min":0.49399999999999999,"5min":0.373,"15min":0.24099999999999999},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.877},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.53700000000000003},"min":{"1min":0.46899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":2.5110000000000001,"15min":2.5110000000000001},"last":0.48899999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.52000000000000002},"min":{"1min":0.42499999999999999,"5min":0.38300000000000001,"15min":0.313},"max":{"1min":0.87,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.55000000000000004},"min":{"1min":0.50700000000000001,"5min":0.34999999999999998,"15min":0.33000000000000002},"max":{"1min":0.77200000000000002,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":0.628}]},{"osd":2,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60599999999999998,"15min":0.55000000000000004},"min":{"1min":0.53500000000000003,"5min":0.375,"15min":0.36899999999999999},"max":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.628,"15min":0.54100000000000004},"min":{"1min":0.45200000000000001,"5min":0.35899999999999999,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.875,"15min":0.875},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61799999999999999,"15min":0.56100000000000005},"min":{"1min":0.49199999999999999,"5min":0.42699999999999999,"15min":0.39300000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63300000000000001,"15min":0.59099999999999997},"min":{"1min":0.49099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82199999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64700000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81899999999999995,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":0.46300000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.61799999999999999,"15min":0.59299999999999997},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.28499999999999998},"max":{"1min":0.87,"5min":7.9939999999999998,"15min":7.9939999999999998},"last":0.442},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68200000000000005,"15min":0.66300000000000003},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84799999999999998,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.68799999999999994}]},{"osd":6,"last update":"Tue Nov 28 13:07:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.627,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82799999999999996,"5min":4.133,"15min":4.133},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.68899999999999995,"15min":0.65700000000000003},"min":{"1min":0.45100000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89000000000000001,"5min":8.3000000000000007,"15min":8.3000000000000007},"last":0.52000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574948,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":868352,"data_stored":408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56000000000000005,"15min":0.496},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70599999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57299999999999995,"15min":0.51400000000000001},"min":{"1min":0.34300000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.68300000000000005}]},{"osd":1,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.30499999999999999},"max":{"1min":0.75700000000000001,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.55200000000000005},"min":{"1min":0.42999999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76000000000000001,"5min":0.79500000000000004,"15min":0.87},"last":0.754}]},{"osd":2,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65800000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57399999999999995,"15min":0.53200000000000003},"min":{"1min":0.434,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.71999999999999997}]},{"osd":3,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.60799999999999998,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71199999999999997,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61699999999999999,"15min":0.59299999999999997},"min":{"1min":0.49299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.71499999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.63300000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.52500000000000002,"15min":0.54200000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.75,"5min":0.84599999999999997,"15min":0.85399999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.55100000000000005,"15min":0.56799999999999995},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73299999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.53300000000000003,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.68899999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.60099999999999998,"15min":0.61299999999999999},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.86599999999999999,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56399999999999995}]},{"osd":7,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77800000000000002,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41899999999999998,"5min":0.40799999999999997,"15min":0.372},"max":{"1min":0.71499999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.77200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11032,"kb_used_data":1424,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360808,"statfs":{"total":96636764160,"available":96625467392,"internally_reserved":0,"allocated":1458176,"data_stored":998198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.32800000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.373},"min":{"1min":0.29299999999999998,"5min":0.25700000000000001,"15min":0.253},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.27800000000000002}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.39800000000000002,"15min":0.36499999999999999},"min":{"1min":0.26200000000000001,"5min":0.252,"15min":0.25},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.78500000000000003},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.437,"15min":0.39900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.23699999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.36099999999999999}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.45800000000000002,"15min":0.41799999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.28699999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41599999999999998,"15min":0.38400000000000001},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.24099999999999999},"max":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.69699999999999995},"last":0.436}]},{"osd":4,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79200000000000004,"5min":0.93200000000000005,"15min":1.1259999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.629,"15min":0.58599999999999997},"min":{"1min":0.50600000000000001,"5min":0.46500000000000002,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61199999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61099999999999999,"15min":0.58099999999999996},"min":{"1min":0.48899999999999999,"5min":0.41299999999999998,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.48099999999999998,"5min":0.42699999999999999,"15min":0.38},"max":{"1min":0.88800000000000001,"5min":0.98299999999999998,"15min":1.157},"last":0.48999999999999999}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.41799999999999998,"15min":0.39600000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68899999999999995,"15min":0.67500000000000004},"min":{"1min":0.54300000000000004,"5min":0.46200000000000002,"15min":0.39000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0169999999999999,"15min":1.1759999999999999},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.625},"min":{"1min":0.52000000000000002,"5min":0.40500000000000003,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.624},"min":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.41599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.51600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574948,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361708,"statfs":{"total":96636764160,"available":96626388992,"internally_reserved":0,"allocated":864256,"data_stored":407849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.35399999999999998},"min":{"1min":0.223,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.61399999999999999,"5min":0.623,"15min":0.623},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40899999999999997,"15min":0.36399999999999999},"min":{"1min":0.30199999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57799999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.35299999999999998}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41199999999999998,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.443,"15min":0.39500000000000002},"min":{"1min":0.33700000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.66000000000000003},"last":0.316}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40500000000000003,"15min":0.36099999999999999},"min":{"1min":0.27700000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66700000000000004,"5min":0.746,"15min":0.746},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47699999999999998,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.38300000000000001}]},{"osd":4,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66900000000000004,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":3.7559999999999998,"15min":3.7559999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.57799999999999996},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.372},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64300000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.52900000000000003,"5min":0.42799999999999999,"15min":0.42699999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.61399999999999999},"min":{"1min":0.505,"5min":0.44400000000000001,"15min":0.42999999999999999},"max":{"1min":0.83599999999999997,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.78300000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.44900000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":1.5509999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.68600000000000005,"15min":0.65200000000000002},"min":{"1min":0.57399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.996},"last":0.751}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66200000000000003,"15min":0.63},"min":{"1min":0.55500000000000005,"5min":0.45200000000000001,"15min":0.41499999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.98299999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.65000000000000002},"min":{"1min":0.47299999999999998,"5min":0.46999999999999997,"15min":0.40200000000000002},"max":{"1min":0.90200000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":868352,"data_stored":408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.379,"15min":0.36199999999999999},"min":{"1min":0.23400000000000001,"5min":0.21299999999999999,"15min":0.20399999999999999},"max":{"1min":0.46000000000000002,"5min":0.628,"15min":0.70999999999999996},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.378},"min":{"1min":0.27800000000000002,"5min":0.24199999999999999,"15min":0.23300000000000001},"max":{"1min":0.47699999999999998,"5min":0.60899999999999999,"15min":0.72299999999999998},"last":0.29899999999999999}]},{"osd":1,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.41799999999999998,"15min":0.40200000000000002},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.24099999999999999},"max":{"1min":0.48199999999999998,"5min":0.63200000000000001,"15min":0.65700000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.443,"15min":0.40200000000000002},"min":{"1min":0.30299999999999999,"5min":0.28699999999999998,"15min":0.255},"max":{"1min":0.47699999999999998,"5min":0.63200000000000001,"15min":0.69399999999999995},"last":0.44900000000000001}]},{"osd":3,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40600000000000003,"15min":0.38100000000000001},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51200000000000001,"5min":0.65400000000000003,"15min":0.73099999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.47299999999999998,"15min":0.44400000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.29599999999999999},"max":{"1min":0.53900000000000003,"5min":0.71199999999999997,"15min":0.748},"last":0.40100000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.61699999999999999},"min":{"1min":0.46000000000000002,"5min":0.39600000000000002,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":0.93500000000000005,"15min":0.96199999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66500000000000004,"15min":0.624},"min":{"1min":0.53800000000000003,"5min":0.438,"15min":0.36699999999999999},"max":{"1min":0.874,"5min":0.96299999999999997,"15min":1.0649999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.63400000000000001},"min":{"1min":0.51100000000000001,"5min":0.45100000000000001,"15min":0.40300000000000002},"max":{"1min":0.92800000000000005,"5min":0.94099999999999995,"15min":0.96899999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.44600000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.66600000000000004}]},{"osd":6,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.52700000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.88400000000000001,"5min":1.022,"15min":1.022},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67300000000000004,"15min":0.63900000000000001},"min":{"1min":0.499,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Nov 28 13:07:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.69599999999999995,"15min":0.67100000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86299999999999999,"5min":1.022,"15min":1.0309999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.65200000000000002},"min":{"1min":0.48099999999999998,"5min":0.42499999999999999,"15min":0.38400000000000001},"max":{"1min":0.84299999999999997,"5min":1.004,"15min":1.004},"last":0.73999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10072,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361768,"statfs":{"total":96636764160,"available":96626450432,"internally_reserved":0,"allocated":868352,"data_stored":408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34599999999999997,"15min":0.34999999999999998},"min":{"1min":0.23899999999999999,"5min":0.19800000000000001,"15min":0.189},"max":{"1min":0.47099999999999997,"5min":0.61199999999999999,"15min":0.75800000000000001},"last":0.627},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39400000000000002,"15min":0.35299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20399999999999999},"max":{"1min":0.48999999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.59999999999999998}]},{"osd":1,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.373},"min":{"1min":0.28199999999999997,"5min":0.245,"15min":0.245},"max":{"1min":0.55800000000000005,"5min":0.64300000000000002,"15min":1.1279999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.40600000000000003},"min":{"1min":0.318,"5min":0.252,"15min":0.22600000000000001},"max":{"1min":0.52900000000000003,"5min":0.63200000000000001,"15min":0.82799999999999996},"last":0.68400000000000005}]},{"osd":2,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.52700000000000002},"min":{"1min":0.314,"5min":0.27000000000000002,"15min":0.24399999999999999},"max":{"1min":0.54600000000000004,"5min":0.59399999999999997,"15min":5.0540000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.432,"15min":0.45100000000000001},"min":{"1min":0.25800000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56499999999999995,"5min":0.67500000000000004,"15min":1.9390000000000001},"last":0.64500000000000002}]},{"osd":4,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.64800000000000002},"min":{"1min":0.45400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93799999999999994,"5min":0.98599999999999999,"15min":1.6919999999999999},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68999999999999995,"15min":0.64500000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88800000000000001,"5min":0.97099999999999997,"15min":1.2},"last":0.98999999999999999}]},{"osd":5,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.625,"15min":0.57199999999999995},"min":{"1min":0.41299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.92500000000000004,"15min":1.169},"last":0.87},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65200000000000002,"15min":0.61099999999999999},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81100000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.96999999999999997}]},{"osd":6,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42299999999999999},"max":{"1min":0.90500000000000003,"5min":0.94599999999999995,"15min":1.6519999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93000000000000005,"5min":1.01,"15min":1.23},"last":0.90400000000000003}]},{"osd":7,"last update":"Tue Nov 28 13:07:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.65400000000000003},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86799999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":1.04},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69499999999999995,"15min":0.66800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90000000000000002,"5min":1.018,"15min":1.7250000000000001},"last":1.0880000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10136,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361704,"statfs":{"total":96636764160,"available":96626384896,"internally_reserved":0,"allocated":868352,"data_stored":408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59599999999999997,"15min":0.505},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32000000000000001},"max":{"1min":0.77300000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59399999999999997,"15min":0.54600000000000004},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79100000000000004,"5min":0.873,"15min":0.873},"last":0.754}]},{"osd":1,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60899999999999999,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.377,"15min":0.34000000000000002},"max":{"1min":0.81000000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.53200000000000003},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.35199999999999998},"max":{"1min":0.79000000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68000000000000005}]},{"osd":2,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.36899999999999999,"15min":0.35099999999999998},"max":{"1min":0.76800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63200000000000001,"15min":0.60299999999999998},"min":{"1min":0.47499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.85999999999999999},"last":0.79300000000000004}]},{"osd":3,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60399999999999998,"15min":0.55600000000000005},"min":{"1min":0.45600000000000002,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.879},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.58899999999999997},"min":{"1min":0.442,"5min":0.41199999999999998,"15min":0.372},"max":{"1min":0.81000000000000005,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.77800000000000002}]},{"osd":5,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51500000000000001,"15min":0.56200000000000006},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66700000000000004,"5min":0.79800000000000004,"15min":0.95199999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54000000000000004,"15min":0.57399999999999995},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72299999999999998,"5min":0.77300000000000002,"15min":0.79400000000000004},"last":0.76600000000000001}]},{"osd":6,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54600000000000004,"15min":0.58399999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80000000000000004,"5min":0.82599999999999996,"15min":0.84999999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60199999999999998,"15min":0.64800000000000002},"min":{"1min":0.441,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.81899999999999995,"5min":0.88500000000000001,"15min":1.117},"last":0.71299999999999997}]},{"osd":7,"last update":"Tue Nov 28 13:07:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59099999999999997,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.78300000000000003,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.81799999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9880,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361960,"statfs":{"total":96636764160,"available":96626647040,"internally_reserved":0,"allocated":868352,"data_stored":408835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59199999999999997,"15min":0.52300000000000002},"min":{"1min":0.39900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86099999999999999,"5min":0.875,"15min":0.875},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.56999999999999995},"min":{"1min":0.38200000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66700000000000004}]},{"osd":1,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60799999999999998,"15min":0.55700000000000005},"min":{"1min":0.45600000000000002,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.57999999999999996,"15min":0.52300000000000002},"min":{"1min":0.41799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.63100000000000001}]},{"osd":2,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.61099999999999999,"15min":0.57799999999999996},"min":{"1min":0.50700000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63600000000000001,"15min":0.61099999999999999},"min":{"1min":0.443,"5min":0.41699999999999998,"15min":0.38600000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.56200000000000006}]},{"osd":3,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.622,"15min":0.59599999999999997},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65700000000000003,"15min":0.64800000000000002},"min":{"1min":0.52200000000000002,"5min":0.41599999999999998,"15min":0.39200000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.70999999999999996}]},{"osd":4,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52700000000000002,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.95999999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52100000000000002,"15min":0.61099999999999999},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78700000000000003,"5min":0.84599999999999997,"15min":1.0029999999999999},"last":0.53000000000000003}]},{"osd":6,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51300000000000001,"15min":0.54700000000000004},"min":{"1min":0.35699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69999999999999996,"5min":0.755,"15min":0.85899999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.66000000000000003},"min":{"1min":0.379,"5min":0.36299999999999999,"15min":0.34999999999999998},"max":{"1min":0.85399999999999998,"5min":0.91900000000000004,"15min":1.0349999999999999},"last":0.44}]},{"osd":7,"last update":"Tue Nov 28 13:07:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.56599999999999995,"15min":0.59799999999999998},"min":{"1min":0.40300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.89800000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60299999999999998,"15min":0.65000000000000002},"min":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.38600000000000001},"max":{"1min":0.879,"5min":0.89100000000000001,"15min":1.052},"last":0.64900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T13:08:16.395 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T13:08:16.519 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.518+0000 7f3986a75700 1 Processor -- start 2023-11-28T13:08:16.520 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.518+0000 7f3986a75700 1 -- start start 2023-11-28T13:08:16.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3986a75700 1 --2- >> v2:172.21.15.61:3300/0 conn(0x7f39800cd140 0x7f398006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3986a75700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f398006d140 0x7f398006b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3986a75700 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39800ae730 con 0x7f398015a640 2023-11-28T13:08:16.522 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3986a75700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39800cb210 con 0x7f398006d140 2023-11-28T13:08:16.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3986a75700 1 -- --> v2:172.21.15.61:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39800c3e10 con 0x7f39800cd140 2023-11-28T13:08:16.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3985012700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f398006d140 0x7f398006b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3984811700 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f398015a640 0x7f39800c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:51442/0 (socket says 172.21.15.6:51442) 2023-11-28T13:08:16.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3984811700 1 -- 172.21.15.6:0/215712703 learned_addr learned my addr 172.21.15.6:0/215712703 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-28T13:08:16.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f397ffff700 1 --2- 172.21.15.6:0/215712703 >> v2:172.21.15.61:3300/0 conn(0x7f39800cd140 0x7f398006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3985012700 1 -- 172.21.15.6:0/215712703 >> v2:172.21.15.61:3300/0 conn(0x7f39800cd140 msgr2=0x7f398006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:16.523 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3985012700 1 --2- 172.21.15.6:0/215712703 >> v2:172.21.15.61:3300/0 conn(0x7f39800cd140 0x7f398006b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3985012700 1 -- 172.21.15.6:0/215712703 >> v1:172.21.15.6:6789/0 conn(0x7f398015a640 legacy=0x7f39800c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f3985012700 1 -- 172.21.15.6:0/215712703 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398006e9e0 con 0x7f398006d140 2023-11-28T13:08:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.519+0000 7f397f7fe700 1 -- 172.21.15.6:0/215712703 <== mon.0 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438197975 0 0) 0x7f39800ae730 con 0x7f398015a640 2023-11-28T13:08:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f3985012700 1 --2- 172.21.15.6:0/215712703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 0x7f398006b730 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f39800d0c50 tx=0x7f397800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c1fc5944361be9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f397f7fe700 1 -- 172.21.15.6:0/215712703 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397800e070 con 0x7f398006d140 2023-11-28T13:08:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f397f7fe700 1 -- 172.21.15.6:0/215712703 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397800bbd0 con 0x7f398006d140 2023-11-28T13:08:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f397f7fe700 1 -- 172.21.15.6:0/215712703 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978012430 con 0x7f398006d140 2023-11-28T13:08:16.524 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f3986a75700 1 -- 172.21.15.6:0/215712703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 msgr2=0x7f398006b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f3986a75700 1 --2- 172.21.15.6:0/215712703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 0x7f398006b730 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f39800d0c50 tx=0x7f397800b330 comp rx=0 tx=0).stop 2023-11-28T13:08:16.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f3986a75700 1 -- 172.21.15.6:0/215712703 shutdown_connections 2023-11-28T13:08:16.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f3986a75700 1 --2- 172.21.15.6:0/215712703 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 0x7f398006b730 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f3986a75700 1 --2- 172.21.15.6:0/215712703 >> v2:172.21.15.61:3300/0 conn(0x7f39800cd140 0x7f398006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f3986a75700 1 -- 172.21.15.6:0/215712703 >> 172.21.15.6:0/215712703 conn(0x7f39800cc630 msgr2=0x7f39800cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:16.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f3986a75700 1 -- 172.21.15.6:0/215712703 shutdown_connections 2023-11-28T13:08:16.525 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.520+0000 7f3986a75700 1 -- 172.21.15.6:0/215712703 wait complete. 2023-11-28T13:08:16.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.521+0000 7f3986a75700 1 Processor -- start 2023-11-28T13:08:16.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.521+0000 7f3986a75700 1 -- start start 2023-11-28T13:08:16.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.522+0000 7f3986a75700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 0x7f3980150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.528 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.522+0000 7f3986a75700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39800cd140 0x7f398015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.522+0000 7f3986a75700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f398015a640 0x7f398015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.522+0000 7f3986a75700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39801f41d0 con 0x7f398006d140 2023-11-28T13:08:16.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.522+0000 7f3986a75700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39801efc70 con 0x7f398015a640 2023-11-28T13:08:16.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.522+0000 7f3986a75700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39801efe60 con 0x7f39800cd140 2023-11-28T13:08:16.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.522+0000 7f3984811700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 0x7f3980150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.522+0000 7f3984811700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 0x7f3980150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35896/0 (socket says 172.21.15.6:35896) 2023-11-28T13:08:16.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.522+0000 7f3984811700 1 -- 172.21.15.6:0/2992046936 learned_addr learned my addr 172.21.15.6:0/2992046936 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:08:16.529 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f3985012700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f398015a640 0x7f398015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f397ffff700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39800cd140 0x7f398015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f3984811700 1 -- 172.21.15.6:0/2992046936 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39800cd140 msgr2=0x7f398015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f3984811700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39800cd140 0x7f398015d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f3984811700 1 -- 172.21.15.6:0/2992046936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f398015a640 msgr2=0x7f398015d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f3984811700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f398015a640 0x7f398015d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f3984811700 1 -- 172.21.15.6:0/2992046936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3978009a70 con 0x7f398006d140 2023-11-28T13:08:16.530 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f3984811700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 0x7f3980150450 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f39740011e0 tx=0x7f397400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f397d7fa700 1 -- 172.21.15.6:0/2992046936 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974004c80 con 0x7f398006d140 2023-11-28T13:08:16.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f397d7fa700 1 -- 172.21.15.6:0/2992046936 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3974004e00 con 0x7f398006d140 2023-11-28T13:08:16.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f397d7fa700 1 -- 172.21.15.6:0/2992046936 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974007310 con 0x7f398006d140 2023-11-28T13:08:16.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f3986a75700 1 -- 172.21.15.6:0/2992046936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39800aec60 con 0x7f398006d140 2023-11-28T13:08:16.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.523+0000 7f3986a75700 1 -- 172.21.15.6:0/2992046936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39800ae980 con 0x7f398006d140 2023-11-28T13:08:16.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.524+0000 7f3986a75700 1 -- 172.21.15.6:0/2992046936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39800aef40 con 0x7f398006d140 2023-11-28T13:08:16.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.524+0000 7f397d7fa700 1 -- 172.21.15.6:0/2992046936 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f397401a020 con 0x7f398006d140 2023-11-28T13:08:16.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.525+0000 7f397d7fa700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39640682f0 0x7f396406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.531 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.525+0000 7f397d7fa700 1 -- 172.21.15.6:0/2992046936 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f3974085770 con 0x7f398006d140 2023-11-28T13:08:16.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.525+0000 7f397ffff700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39640682f0 0x7f396406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.525+0000 7f397ffff700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39640682f0 0x7f396406a7a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3980158630 tx=0x7f396c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.532 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.529+0000 7f397d7fa700 1 -- 172.21.15.6:0/2992046936 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3974052110 con 0x7f398006d140 2023-11-28T13:08:16.721 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.720+0000 7f3986a75700 1 -- 172.21.15.6:0/2992046936 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f39800739f0 con 0x7f398006d140 2023-11-28T13:08:16.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.723+0000 7f397d7fa700 1 -- 172.21.15.6:0/2992046936 <== mon.0 v2:172.21.15.6:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3974052110 con 0x7f398006d140 2023-11-28T13:08:16.725 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.724+0000 7f397d7fa700 1 -- 172.21.15.6:0/2992046936 <== mon.0 v2:172.21.15.6:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3974055730 con 0x7f398006d140 2023-11-28T13:08:16.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 -- 172.21.15.6:0/2992046936 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39640682f0 msgr2=0x7f396406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39640682f0 0x7f396406a7a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3980158630 tx=0x7f396c008040 comp rx=0 tx=0).stop 2023-11-28T13:08:16.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 -- 172.21.15.6:0/2992046936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 msgr2=0x7f3980150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 0x7f3980150450 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f39740011e0 tx=0x7f397400cae0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 -- 172.21.15.6:0/2992046936 shutdown_connections 2023-11-28T13:08:16.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f398015a640 0x7f398015d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.729 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f398006d140 0x7f3980150450 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f39640682f0 0x7f396406a7a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 --2- 172.21.15.6:0/2992046936 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f39800cd140 0x7f398015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 -- 172.21.15.6:0/2992046936 >> 172.21.15.6:0/2992046936 conn(0x7f39800cc630 msgr2=0x7f39800cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 -- 172.21.15.6:0/2992046936 shutdown_connections 2023-11-28T13:08:16.730 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.727+0000 7f3986a75700 1 -- 172.21.15.6:0/2992046936 wait complete. 2023-11-28T13:08:16.741 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T13:08:16.741 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T13:08:16.865 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.864+0000 7f42dc2b7700 1 Processor -- start 2023-11-28T13:08:16.866 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.864+0000 7f42dc2b7700 1 -- start start 2023-11-28T13:08:16.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.865+0000 7f42dc2b7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42d4159590 0x7f42d406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.865+0000 7f42dc2b7700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f42d415db70 0x7f42d40ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.868 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.865+0000 7f42dc2b7700 1 -- --> v1:172.21.15.61:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d415d120 con 0x7f42d40ce890 2023-11-28T13:08:16.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.865+0000 7f42dc2b7700 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d415d6e0 con 0x7f42d415db70 2023-11-28T13:08:16.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.865+0000 7f42dc2b7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40aef60 con 0x7f42d4159590 2023-11-28T13:08:16.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.865+0000 7f42d9852700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f42d415db70 0x7f42d40ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.865+0000 7f42d9852700 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f42d415db70 0x7f42d40ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35904/0 (socket says 172.21.15.6:35904) 2023-11-28T13:08:16.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.865+0000 7f42d9852700 1 -- 172.21.15.6:0/1865918687 learned_addr learned my addr 172.21.15.6:0/1865918687 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:08:16.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.865+0000 7f42da053700 1 --2- 172.21.15.6:0/1865918687 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42d4159590 0x7f42d406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.866+0000 7f42d9852700 1 -- 172.21.15.6:0/1865918687 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42d4159590 msgr2=0x7f42d406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T13:08:16.869 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.866+0000 7f42d9852700 1 --2- 172.21.15.6:0/1865918687 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42d4159590 0x7f42d406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.866+0000 7f42d9852700 1 -- 172.21.15.6:0/1865918687 >> v1:172.21.15.61:6789/0 conn(0x7f42d40ce890 legacy=0x7f42d40caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.866+0000 7f42d9852700 1 -- 172.21.15.6:0/1865918687 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d40af240 con 0x7f42d415db70 2023-11-28T13:08:16.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.866+0000 7f42d9852700 1 --2- 172.21.15.6:0/1865918687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d415db70 0x7f42d40ae730 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f42d00098f0 tx=0x7f42d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bcf6d648f1dffc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.866+0000 7f42d9051700 1 -- 172.21.15.6:0/1865918687 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000b8a0 con 0x7f42d415db70 2023-11-28T13:08:16.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.866+0000 7f42d9051700 1 -- 172.21.15.6:0/1865918687 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f42d000ba00 con 0x7f42d415db70 2023-11-28T13:08:16.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.866+0000 7f42d9051700 1 -- 172.21.15.6:0/1865918687 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000bdb0 con 0x7f42d415db70 2023-11-28T13:08:16.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.867+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1865918687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d415db70 msgr2=0x7f42d40ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.870 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.867+0000 7f42dc2b7700 1 --2- 172.21.15.6:0/1865918687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d415db70 0x7f42d40ae730 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f42d00098f0 tx=0x7f42d0009c00 comp rx=0 tx=0).stop 2023-11-28T13:08:16.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.867+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1865918687 shutdown_connections 2023-11-28T13:08:16.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.867+0000 7f42dc2b7700 1 --2- 172.21.15.6:0/1865918687 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42d4159590 0x7f42d406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.867+0000 7f42dc2b7700 1 --2- 172.21.15.6:0/1865918687 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d415db70 0x7f42d40ae730 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.867+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1865918687 >> 172.21.15.6:0/1865918687 conn(0x7f42d40b1650 msgr2=0x7f42d40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:16.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.867+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1865918687 shutdown_connections 2023-11-28T13:08:16.871 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.867+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1865918687 wait complete. 2023-11-28T13:08:16.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.868+0000 7f42dc2b7700 1 Processor -- start 2023-11-28T13:08:16.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.868+0000 7f42dc2b7700 1 -- start start 2023-11-28T13:08:16.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42dc2b7700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d40ce890 0x7f42d40c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.874 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42dc2b7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42d4159590 0x7f42d40c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42dc2b7700 1 --2- >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42d415db70 0x7f42d4153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42dc2b7700 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40c7390 con 0x7f42d40ce890 2023-11-28T13:08:16.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42dc2b7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40c9b60 con 0x7f42d4159590 2023-11-28T13:08:16.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42dc2b7700 1 -- --> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40c6cf0 con 0x7f42d415db70 2023-11-28T13:08:16.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42da053700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d40ce890 0x7f42d40c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42da053700 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d40ce890 0x7f42d40c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:35910/0 (socket says 172.21.15.6:35910) 2023-11-28T13:08:16.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42da053700 1 -- 172.21.15.6:0/1889614799 learned_addr learned my addr 172.21.15.6:0/1889614799 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-28T13:08:16.875 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42d9852700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42d4159590 0x7f42d40c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42da854700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42d415db70 0x7f42d4153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42da053700 1 -- 172.21.15.6:0/1889614799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42d415db70 msgr2=0x7f42d4153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42da053700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42d415db70 0x7f42d4153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42da053700 1 -- 172.21.15.6:0/1889614799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42d4159590 msgr2=0x7f42d40c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:16.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42da053700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42d4159590 0x7f42d40c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:16.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42da053700 1 -- 172.21.15.6:0/1889614799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d0009550 con 0x7f42d40ce890 2023-11-28T13:08:16.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42da053700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d40ce890 0x7f42d40c77b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f42c400e810 tx=0x7f42c400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.876 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42caffd700 1 -- 172.21.15.6:0/1889614799 <== mon.0 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c400c070 con 0x7f42d40ce890 2023-11-28T13:08:16.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42caffd700 1 -- 172.21.15.6:0/1889614799 <== mon.0 v2:172.21.15.6:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f42c4013610 con 0x7f42d40ce890 2023-11-28T13:08:16.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1889614799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d40aec70 con 0x7f42d40ce890 2023-11-28T13:08:16.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.869+0000 7f42caffd700 1 -- 172.21.15.6:0/1889614799 <== mon.0 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c40139e0 con 0x7f42d40ce890 2023-11-28T13:08:16.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.870+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1889614799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d415d370 con 0x7f42d40ce890 2023-11-28T13:08:16.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.870+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1889614799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d415dfe0 con 0x7f42d40ce890 2023-11-28T13:08:16.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.871+0000 7f42caffd700 1 -- 172.21.15.6:0/1889614799 <== mon.0 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f42c4023070 con 0x7f42d40ce890 2023-11-28T13:08:16.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.871+0000 7f42caffd700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b80682f0 0x7f42b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T13:08:16.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.871+0000 7f42caffd700 1 -- 172.21.15.6:0/1889614799 <== mon.0 v2:172.21.15.6:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f42c4085890 con 0x7f42d40ce890 2023-11-28T13:08:16.877 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.871+0000 7f42d9852700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b80682f0 0x7f42b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T13:08:16.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.872+0000 7f42d9852700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b80682f0 0x7f42b806a7a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009c40 tx=0x7f42d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T13:08:16.878 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:16.876+0000 7f42caffd700 1 -- 172.21.15.6:0/1889614799 <== mon.0 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f42c4052230 con 0x7f42d40ce890 2023-11-28T13:08:17.118 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.117+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1889614799 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42d40c6b50 con 0x7f42d40ce890 2023-11-28T13:08:17.122 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.117+0000 7f42caffd700 1 -- 172.21.15.6:0/1889614799 <== mon.0 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3520 (secure 0 0 0) 0x7f42d40c6b50 con 0x7f42d40ce890 2023-11-28T13:08:17.122 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-28T13:08:17.123 INFO:teuthology.orchestra.run.smithi006.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6836/1198233336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":1198233336},{"type":"v1","addr":"172.21.15.6:6836","nonce":1198233336}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6430,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2761758707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2761758707},{"type":"v1","addr":"172.21.15.61:6835","nonce":2761758707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6837/235204249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6835","nonce":235204249},{"type":"v1","addr":"172.21.15.6:6837","nonce":235204249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6468,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6833/2675481701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":2675481701},{"type":"v1","addr":"172.21.15.61:6833","nonce":2675481701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2023-11-28T13:08:17.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1889614799 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b80682f0 msgr2=0x7f42b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:17.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b80682f0 0x7f42b806a7a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009c40 tx=0x7f42d0008030 comp rx=0 tx=0).stop 2023-11-28T13:08:17.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1889614799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d40ce890 msgr2=0x7f42d40c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T13:08:17.124 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d40ce890 0x7f42d40c77b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f42c400e810 tx=0x7f42c400eb20 comp rx=0 tx=0).stop 2023-11-28T13:08:17.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1889614799 shutdown_connections 2023-11-28T13:08:17.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.6:6832/103231,v1:172.21.15.6:6833/103231] conn(0x7f42b80682f0 0x7f42b806a7a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:17.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f42d415db70 0x7f42d4153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:17.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f42d40ce890 0x7f42d40c77b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:17.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 --2- 172.21.15.6:0/1889614799 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42d4159590 0x7f42d40c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T13:08:17.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1889614799 >> 172.21.15.6:0/1889614799 conn(0x7f42d40b1650 msgr2=0x7f42d406d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T13:08:17.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1889614799 shutdown_connections 2023-11-28T13:08:17.125 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-28T13:08:17.120+0000 7f42dc2b7700 1 -- 172.21.15.6:0/1889614799 wait complete. 2023-11-28T13:08:17.126 INFO:teuthology.orchestra.run.smithi006.stderr:dumped fsmap epoch 35 2023-11-28T13:08:17.133 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T13:08:17.133 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T13:08:17.134 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T13:08:17.134 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.142 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T13:08:17.143 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T13:08:17.143 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.191 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T13:08:17.191 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T13:08:17.191 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.202 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T13:08:17.202 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T13:08:17.202 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.254 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T13:08:17.255 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T13:08:17.255 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T13:08:17.255 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.285 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T13:08:17.285 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T13:08:17.285 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.358 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T13:08:17.358 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T13:08:17.358 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.438 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T13:08:17.438 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T13:08:17.438 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.519 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T13:08:17.520 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T13:08:17.520 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.565 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T13:08:17.565 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T13:08:17.565 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.608 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T13:08:17.608 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T13:08:17.608 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.655 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T13:08:17.655 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T13:08:17.655 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.700 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T13:08:17.700 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T13:08:17.700 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T13:08:17.700 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.726 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T13:08:17.727 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T13:08:17.727 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.752 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T13:08:17.752 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T13:08:17.752 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T13:08:17.752 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.779 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T13:08:17.779 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T13:08:17.780 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.803 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T13:08:17.804 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T13:08:17.804 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T13:08:17.856 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T13:08:17.857 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T13:08:17.857 DEBUG:teuthology.orchestra.run.smithi006:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-28T13:08:17.894 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:08:17.895 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T13:08:18.142 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:08:18.142 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T13:08:18.287 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:08:18.288 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T13:08:18.377 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:08:18.377 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T13:08:18.467 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:08:18.468 DEBUG:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T13:08:18.665 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:08:18.666 DEBUG:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T13:08:18.774 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:08:18.774 DEBUG:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T13:08:18.839 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:08:18.840 DEBUG:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T13:08:18.905 INFO:tasks.ceph:Archiving mon data... 2023-11-28T13:08:18.907 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973/data/mon.a.tgz 2023-11-28T13:08:18.907 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2023-11-28T13:08:18.927 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.2pDVZk3FUF 2023-11-28T13:08:18.927 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.2pDVZk3FUF 2023-11-28T13:08:19.074 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.2pDVZk3FUF 2023-11-28T13:08:19.163 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.2pDVZk3FUF is 312KB 2023-11-28T13:08:19.259 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.2pDVZk3FUF 2023-11-28T13:08:19.276 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973/data/mon.b.tgz 2023-11-28T13:08:19.276 DEBUG:teuthology.orchestra.run.smithi061:> mktemp 2023-11-28T13:08:19.296 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.gPumS0Aq3e 2023-11-28T13:08:19.296 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.gPumS0Aq3e 2023-11-28T13:08:19.470 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.gPumS0Aq3e 2023-11-28T13:08:19.561 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.gPumS0Aq3e is 320KB 2023-11-28T13:08:19.686 DEBUG:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.gPumS0Aq3e 2023-11-28T13:08:19.708 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973/data/mon.c.tgz 2023-11-28T13:08:19.709 DEBUG:teuthology.orchestra.run.smithi061:> mktemp 2023-11-28T13:08:19.770 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.DIJ26DeiKZ 2023-11-28T13:08:19.770 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.DIJ26DeiKZ 2023-11-28T13:08:19.903 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.DIJ26DeiKZ 2023-11-28T13:08:19.993 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.DIJ26DeiKZ is 320KB 2023-11-28T13:08:20.067 DEBUG:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.DIJ26DeiKZ 2023-11-28T13:08:20.084 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T13:08:20.084 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T13:08:20.087 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T13:08:20.154 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T13:08:20.155 DEBUG:teuthology.orchestra.run.smithi006:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T13:08:20.162 DEBUG:teuthology.orchestra.run.smithi061:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T13:08:20.202 INFO:teuthology.orchestra.run.smithi006.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T13:08:20.229 INFO:teuthology.orchestra.run.smithi061.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T13:08:20.235 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T13:08:20.448 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973/remote/smithi006/crash 2023-11-28T13:08:20.493 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T13:08:20.562 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973/remote/smithi061/crash 2023-11-28T13:08:20.577 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T13:08:20.606 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T13:08:20.607 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:08:20.623 DEBUG:teuthology.orchestra.run.smithi061:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T13:08:20.703 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T13:08:20.734 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T13:08:20.761 INFO:tasks.ceph:Compressing logs... 2023-11-28T13:08:20.761 DEBUG:teuthology.orchestra.run.smithi006:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:08:20.776 DEBUG:teuthology.orchestra.run.smithi061:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:08:25.904 INFO:tasks.ceph:Archiving logs... 2023-11-28T13:08:25.904 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973/remote/smithi006/log 2023-11-28T13:08:25.905 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T13:08:26.881 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973/remote/smithi061/log 2023-11-28T13:08:26.882 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T13:08:27.422 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T13:08:27.431 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T13:08:27.431 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T13:08:27.434 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T13:08:27.533 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T13:08:27.542 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T13:08:27.543 DEBUG:teuthology.orchestra.run.smithi006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T13:08:27.547 DEBUG:teuthology.orchestra.run.smithi061:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T13:08:27.564 INFO:teuthology.orchestra.run.smithi006.stderr:bash: ntpq: command not found 2023-11-28T13:08:27.566 INFO:teuthology.orchestra.run.smithi061.stderr:bash: ntpq: command not found 2023-11-28T13:08:27.569 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T13:08:27.569 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2023-11-28T13:08:27.570 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +914us[ +914us] +/- 98ms 2023-11-28T13:08:27.570 INFO:teuthology.orchestra.run.smithi006.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -356us[ -339us] +/- 66ms 2023-11-28T13:08:27.570 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -278us[ -262us] +/- 81ms 2023-11-28T13:08:27.570 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:08:27.571 INFO:teuthology.orchestra.run.smithi061.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T13:08:27.571 INFO:teuthology.orchestra.run.smithi061.stdout:=============================================================================== 2023-11-28T13:08:27.571 INFO:teuthology.orchestra.run.smithi061.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 41 +910us[ +924us] +/- 98ms 2023-11-28T13:08:27.572 INFO:teuthology.orchestra.run.smithi061.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -353us[ -340us] +/- 66ms 2023-11-28T13:08:27.572 INFO:teuthology.orchestra.run.smithi061.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -288us[ -275us] +/- 81ms 2023-11-28T13:08:27.572 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T13:08:27.573 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T13:08:27.634 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T13:08:27.634 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T13:08:27.670 DEBUG:teuthology.orchestra.run.smithi006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T13:08:27.789 DEBUG:teuthology.orchestra.run.smithi061:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T13:08:27.900 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T13:08:27.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:27.937 DEBUG:teuthology.orchestra.run.smithi061:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T13:08:27.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:27.972 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T13:08:27.981 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T13:08:28.038 INFO:teuthology.task.internal:Duration was 1299.376706 seconds 2023-11-28T13:08:28.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T13:08:28.070 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T13:08:28.070 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T13:08:28.072 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T13:08:28.122 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T13:08:28.123 INFO:teuthology.orchestra.run.smithi061.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T13:08:28.594 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T13:08:28.594 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi006.front.sepia.ceph.com 2023-11-28T13:08:28.594 DEBUG:teuthology.orchestra.run.smithi006:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T13:08:28.621 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi061.front.sepia.ceph.com 2023-11-28T13:08:28.621 DEBUG:teuthology.orchestra.run.smithi061:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T13:08:28.647 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T13:08:28.647 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:08:28.664 DEBUG:teuthology.orchestra.run.smithi061:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T13:08:28.733 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T13:08:28.733 DEBUG:teuthology.orchestra.run.smithi006:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T13:08:28.735 DEBUG:teuthology.orchestra.run.smithi061:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T13:08:28.828 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T13:08:28.840 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T13:08:28.840 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T13:08:28.869 DEBUG:teuthology.orchestra.run.smithi061:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T13:08:28.898 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T13:08:28.909 DEBUG:teuthology.orchestra.run.smithi006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:08:28.911 DEBUG:teuthology.orchestra.run.smithi061:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:08:28.937 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = core 2023-11-28T13:08:28.966 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern = core 2023-11-28T13:08:28.982 DEBUG:teuthology.orchestra.run.smithi006:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:08:29.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:29.008 DEBUG:teuthology.orchestra.run.smithi061:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T13:08:29.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T13:08:29.039 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T13:08:29.051 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T13:08:29.051 DEBUG:teuthology.misc:Transferring archived files from smithi006:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973/remote/smithi006 2023-11-28T13:08:29.052 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T13:08:29.092 DEBUG:teuthology.misc:Transferring archived files from smithi061:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973/remote/smithi061 2023-11-28T13:08:29.092 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T13:08:29.130 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T13:08:29.131 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T13:08:29.133 DEBUG:teuthology.orchestra.run.smithi061:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T13:08:29.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T13:08:29.194 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T13:08:29.194 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T13:08:29.229 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T13:08:29.229 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T13:08:29.231 DEBUG:teuthology.orchestra.run.smithi061:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T13:08:29.246 INFO:teuthology.orchestra.run.smithi006.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 13:08 /home/ubuntu/cephtest 2023-11-28T13:08:29.248 INFO:teuthology.orchestra.run.smithi061.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 13:08 /home/ubuntu/cephtest 2023-11-28T13:08:29.250 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T13:08:29.263 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T13:08:29.329 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T13:08:29.356 DEBUG:teuthology.nuke:shortname: smithi006 2023-11-28T13:08:29.356 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T13:08:29.381 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:41:37.001073', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T13:08:29.385 DEBUG:teuthology.nuke:shortname: smithi061 2023-11-28T13:08:29.385 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T13:08:29.416 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi061.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_12:38:30-fs-wip-neesingh-testing-231123-distro-default-smithi/7469973', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 12:41:37.002518', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T13:08:29.447 INFO:teuthology.orchestra.console.smithi006:Power off 2023-11-28T13:08:29.448 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T13:08:29.472 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off 2023-11-28T13:08:29.472 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:29.490 INFO:teuthology.orchestra.console.smithi061:Power off 2023-11-28T13:08:29.491 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T13:08:29.501 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-11-28T13:08:29.513 DEBUG:teuthology.orchestra.console.smithi061:power off output: Chassis Power Control: Down/Off 2023-11-28T13:08:29.514 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:29.536 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is on 2023-11-28T13:08:33.501 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:33.538 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:33.613 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-11-28T13:08:33.650 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is on 2023-11-28T13:08:37.618 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:37.651 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:37.729 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-11-28T13:08:37.762 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is on 2023-11-28T13:08:41.733 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:41.763 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T13:08:41.844 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2023-11-28T13:08:41.874 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is off 2023-11-28T13:08:41.945 INFO:teuthology.orchestra.console.smithi006:Power off completed 2023-11-28T13:08:41.975 INFO:teuthology.orchestra.console.smithi061:Power off completed 2023-11-28T13:08:42.079 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1299.3767058849335 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=780162d5c8414688a4c601747ab2870f status: fail success: false 2023-11-28T13:08:42.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T13:08:42.157 INFO:teuthology.run:FAIL